Sparse CLIP: Co-Optimizing Interpretability and Performance in Contrastive Learning
Authors: Chuan Qin
Constantin Venhoff
Sonia Joseph
Fanyi Xiao
Stefan Scherer
Venue: ICLR 2026 Poster (2026)
Keywords: contrastive learning
multimodal learning
interpretability
Primary area: interpretability and explainable AI

Abstract: Contrastive Language-Image Pre-training (CLIP) has become a cornerstone in vision-language representation learning, powering diverse downstream tasks and serving as the default vision backbone in multimodal large language models (MLLMs). Despite its success, CLIP's dense and opaque latent representations pose significant interpretability challenges. A common assumption is that interpretability and performance are in tension: enforcing sparsity during training degrades accuracy, motivating recent post-hoc approaches such as Sparse Autoencoders (SAEs). However, these post-hoc approaches often suffer from degraded downstream performance and loss of CLIP's inherent multimodal capabilities, with most learned features remaining unimodal.

We propose a simple yet effective approach that integrates sparsity directly into CLIP training, yielding representations that are both interpretable and performant. Compared to SAEs, our Sparse CLIP representations preserve strong downstream task performance, achieve superior interpretability, and retain multimodal capabilities. We show that multimodal sparse features enable straightforward semantic concept alignment and reveal training dynamics of how cross-modal knowledge emerges. Finally, as a proof of concept, we train a vision-language model on sparse CLIP representations that enables interpretable, vision-based steering capabilities. Our findings challenge conventional wisdom that interpretability requires sacrificing accuracy and demonstrate that interpretability and performance can be co-optimized, offering a promising design principle for future models.

Published as a conference paper at ICLR 2026
SPARSE CLIP: CO-OPTIMIZING INTERPRETABILITY AND PERFORMANCE IN CONTRASTIVE LEARNING
Chuan Qin'
Constantin Venhoff
Sonia Joseph'
Fanyi Xiao!
Stefan Scherer'
Meta
University of Oxford
ABSTRACT
Contrastive Language-Image Pre-training (CLIP) has become a cornerstone in vision-language representation learning, powering diverse downstream tasks and serving as the default vision backbone in multimodal large language models (MLLMs). Despite its success, CLIP's dense and opaque latent representations pose significant interpretability challenges. A common assumption is that inter- pretability and performance are in tension: enforeing sparsity during training degrades accuracy, motivating recent post-hoc approaches such Sparse Autoen- coders (SAEs). However, these post-hoc approaches often suffer from degraded downstream performance and loss of CLIP's inherent multimodal capabilities, with most learned features remaining unimodal.
We propose simple yet effective approach that integrates sparsity directly into CLIP training, yiclding representations that are both interpretable and performant. Compared to SAEs, Sparse CLIP representations preserve strong downstream task performance, achieve superior interpretability, and retain multimodal capabil- ities. We show that multimodal sparse features enable straightforward semantic concept alignment and reveal training dynamics of how cross-modal knowledge emerges. Finally, proof of concept, train a vision-language model sparse CLIP representations that enables interpretable, vision- based steering capabili- ties. Our findings challenge conventional wisdom that interpretability requires sacrificing accuracy and demonstrate that interpretability and performance can be co-optimized, offering promising design principle for future models.
INTRODUCTION
Contrastive Language- Image Pre-training (CLIP) (Radford et al. 2021) has achieved significant suc- cess vision-language representation learning by linking images and text through contrastive training on web-crawled image-text pairs. Its transfcrability and generalizability have been demonstrated across a wide range of downstream tasks, including but not limited to zero-shot image classification and image-text retrieval. Even beyond, CLIP has arguably become the default choice for the vision backbone multimodal large language models (MLLMs). Given the heavy reliance of these applica- tions on the prior knowledge in CLIP embeddings, it is essential to improve understanding to the representations that CLIP generated, which is a dense latent space not directly interpretable.
Recently, researchers have begun applying Sparse Autoencoders (SAEs) well-established tool for mechanistically interpreting large language models (LLMs) to CLIP vision encoders, aiming to gain deeper insights into the concepts captured by pretrained CLIP models (Cunningham et al.) 2023] Joseph et al. (0255]a). a). Typically, an SAE introduces a bottleneck layer into the residual stream of pretrained CLIP encoder. By enforeing sparsity during the training of this high-dimensional bottleneck, the SAE disentangles the original CLIP features into sparse representations, which can potentially be associated with specific concepts.
While SAFs enhance interpretability, several limitations constrain their practical utility. First, re- searches on LLM has shown that sparse SAE features often underperform on downstream tasks compared to their dense counterparts. For example, [Kantamneni et al. (2025) shows that for probing
"Contribution made during internship Mcta.

---

Published as a conference paper at ICLR 2026
task, SAE latents fail to achieve consistent overall advantage over dense counterparts, and Farrell et al. (2024) has had similar result on a different unlearning task. Second, they lose the multi-modal capabilities of CLIP. Most CLIP SAEs were trained only the residual stream of vision tower to get interpretable vision features. And when Papadimitriou et al. (2025) trained SAEs on the multimodal representation space of pretrained CLIP models, they found that most features were unimodal, activated exclusively for image or text inputs.
These limitations reflect broader conventional view currently in the field interpretability: that interpretability and accuracy are fundamentally in tension. In particular, enforcing sparsity during training is widely assumed to degrade downstream task performance, which has motivated the focus on post-hoc sparse methods (Olshausen & Field 1996] Higgins et al.) 2017 Koh et al. [220] Hendrycks Hiscoti 2025). Although some work suggests that this trade-off may not inevitable (Rudin) 2019), the prevailing assumption has remained that training-time sparsity compromises accuracy.
In this work, challenge this assumption. We demonstrate that by projecting dense features into high-dimensional space and enforcing sparsity during CLIP training, we can transform dense CLIP representations into interpretable features similar to those produced by SAEs. However, unlike SAE features, these "Sparse CLIP features" maintain strong performance on downstream tasks, and naturally preserve CLIP's similarity loss. Furthermore, we show that this inherent multi-modal nature enables easy alignment of features with concepts. Building this, analyzed these multi-modal representations, revealing some intriguing multi-modal concepts that the model is able to learn, and more interestingly, how they emerge and evolve during training. Finally, as downstream task for applications, we trained VLM using the sparse CLIP representations, and demonstrated vision-based steering using interpretable features.
nature, as they directly trained using modal cosine
We summarize our contributions as follows:
We introduce novel but simple technique for learning sparse CLIP representations that maintain competitive performance while achieving interpretable multimodal features.
We demonstrate how multimodal alignment unlocks transparent analysis of CLIP's encoded knowledge, revealing feature evolution dynamics throughout training.
We demonstrate real-world application of our approach by implementing interpretable vision-hased steering in vasion-language models.
SPARSE CLIP TRAINING
2.1 ADDING SPARSITY TO CLIP PRETRAINING
Taking broader perspective, there exists family of concept extraction methods designed to improve the interpretability of machine learning models. Fell et al. (2023) proposed that all concept extraction methods can be unified within the framework of dictionary learning. To faithfully interpret an activation matrix A E Rxx where n is the size of dataset and m is the width of the activation, the goal is to find two low-rank matrices (U, such that UVT. this framework, V E Rmak serves as the dictionary while U E Baxk represents the activation A in terms of the dictionary atoms
Different concept extraction methods approach this decomposition through various techniques. Some methods utilize traditional tools like K-means and PCA, others employ Non-negative Matrix Factorization (NMF), while Sparse Autoencoders (SAEs) have recently garnered significant attention due to their ability to be trained using Stochastic Gradient Descent (SGD) and their scalability to unseen data. We focus specifically NMF and SAE in our discussion, in the dictionary learning framework from Fel et af. (2023), they can be unified as:
(U20VV20 (U", VV"" arg min 1 stt. U (AA), U1/o KK (SAE)
(NMF)
(I)
2

---

Published as a conference paper at ICLR 2026
R
R R+-0mm)
RMA
12
prijer
project
R
f Rm
Rm
VelendTast
Veterieer
CLIP
Sparse CLIP
SAE
Figure 1: Comparison between Sparse CLIP and two existing methods, CLIP and SAE.
We initially explored adding sparsity CLIP pretraining by incorporating traditional SAE compo- nents such as reconstruction loss and top-K activation into the CLIP training process. However, we quickly discovered that effective sparsity could be achieved with surprisingly minimal modifications to the original CLIP training procedure, requiring only two key changes:
1. Non-negative constraints: Adding ReLU activation after the final projection layer.
2. Dimension expansion: Significantly increasing the dimensionality of the final projection layer in CLIP.
Our findings turned out be no coincidence, HaoChen et al. (022)) demonstrated that contrastive learning, when expressed in its spectral form, equivalent Matrix Factorization (MF) objective. Building upon this foundation, Wang et al. (2024) proved that non-negative contrastive learning equivalent to Non- Non-negative Matrix Factorization (NMF), with this equivalence extending to the multimodal contrastive learning domain. Therefore, the theorctical foundations established in these papers provide justification for why modifications particularly the introduction of non-negative constraints successfully induce sparsity in CLIP training. And putting this equivalence together with equation [ we start view non-negative CLIP and SAE different decomposition approaches under the same dictionary learning framework.
However, non-negativity constraints alone are insufficient for learning satisfactory dictionary. Our small-scale experiments demonstrate that dimension expansion is critical for enabling sparse representations achieve competitive performance on downstream tasks. This observation aligns with dictionary learning theory, where sufficiently large dictionary size is required for effective representation learning. the best of our knowledge, this work is the first to apply both non- negativity constraints and dimension expansion to CLIP training.
We demonstrate the comparison between CLIP training, sparse CLIP training and SAE in figure After adding the geigh-dimensionalppriieee Sparse CL.PP close to SAE without decoder, but still trained with constrastive loss. the next two sections, we will describe our training details and show benchmark results comparison of these three architectures.
and non-negative constraint (Rel.U) into vanilla CLIP.
2.2 PROOF OF CONCEPT SMALL SCALE TRAINING
use OpenCLIP [Iharco et al. 202 well-established open-source CLIP training repository, as our experimental foundation. To identify the optimal training recipe for sparse CLIP, we conduct small-scale ablation studies using a VIT-B/32 model and 15M image-text pairs sampled from the MetaCLIP dataset (Xu et al. 2024b). We evaluate performance using zero-shot classification accuracy on ImageNet- 1k (Deng et al. 2009) and LO sparsity metrics. Detailed experimental setup is provided in Appendix A.I These small-scale experiments yielded several interesting observations:

---

Published as a conference paper at ICLR 2026
TALL
(a) Impact of non-negativity and embedding dimensionality (log scale) on model perfor- sparsity injection model performance. Logit scale (or "temper- mance and sparsity.
(b) Comparison of (c) Impact of logit scale cap sparsity and
methods.
ature") ") is a learnable scaler for activations.
Figure 2: Small-scal ablation study. We evaluated different Sparse CLIP design choices by training ViT-B/32 models from scratch on a 15M sample dataset.
Observation I: In sparse CLIP training, both performance and sparsity increase with dimension scaling, but only under non-negativity constraints. Tigure[ajshoww how non-negativity constraints and dimension expansion- our two primary CLIP training modifications-afii model performance and representation sparsity. The left panel reveals that increasing embedding dimensionality improves performance only when non-negativity constraints dimension expansion, non-negativity constraints alone degrade performance (leftmost point on blue curve). The right panel shows that activated features plateau embedding dimensionality increases, but only under non-negativity constraints wonstrintss-withoutt them, all features remain activated regardless of dimensionality.
enforced via ReLU activation. Without
Observation 2: ReLU enables sparsity to form more gradually, improving model performance. Fig- ure 26 compares how different sparsity injection methods affect both sparsity levels and performance. We tested three sparsity-inducing techniques while keeping all other training parameters identical: LI loss, TopK activation (K=512) Gao al. (2024). and ReLU activation. Unlike typical SAE training, we apply these methods directly with CLIP loss rather than reconstruction loss. The top panel shows that and TopK methods dramatically reduce LO sparsity from training onset, poten- tially limiting model capacity and impairing learning effectiveness. The bottom panel confirms this hypothesis: ReLU achieves significantly superior zero-shot accuracy, demonstrating that gradual sparsity development preserves the model's 's learning capacity during training.
Observation Sparsity can be controlled by tuning the logit scale cap. CLIP uses a learnable logit scale (temperature) parameter that multiplies cosine similarity scores before applying softmax/cross- entropy loss. This parameter controls softmax distribution sharpness, enabling the model to amplify differences between positive and negative pairs and learn from increasingly challenging examples during training. We discovered that sparsity levels can be effectively controlled by adjusting the logit scale cap. As shown in Figure 2c (left panel), reducing the logit scale cap consistently decreases LO sparsity in activations. While we lack a complete theoretical explanation, this provides a practical sparsity control mechanism. However, optimal sparsity exists within specific range. The right panel demonstrates this trade-off: when the logit scale cap drops to 20, zero-shot performance declines substantially, indicating that excessive sparsity reduction harms the model's representational capacity.
After conducting comprehensive ablation studies, we demonstrated the promising potential of sparse CLIP training on smaller models and datasets. These results motivated to scale up our experiments with scaler-scale training runs.
2.3 SCALING
Based on the findings smaller models and datasets, scale up both model size and data to train an interpretable CLIP model with performance comparable to mainstream models. With hyperparameter settings mostly consistent with our smaller-scale experiments, complete 2.2 billion sample MetaCLIP corpus for approximately 6 epochs. We apply a dimension expansion factor of 7 yielding a representation dimension of 55,296. Based our observation
ViT-L/14 and train with the
An expansion factor of 72 is the maximum can achieve for VIT-L/14 given 80GB GPU memory constraints. This limitation discussed further in Section 5
4

---

Published as a conference paper at ICLR 2026
Zero Shot Clasilicaioo
Zero Shot Fine Grained Clusifictiion
Class
ImageNNe
ImageNer-22
ObpectNe
InageNos-A
BragseNoeR
ImageNNec S
Ang. Fine.
Foods
Commiies
Scooes
Satellne
Model
OpenAl VIT-8/32
57.5
68.8
60.1
53.5
29.4
77.8
56.4
63.3
85.5
73.3
89.6
86.1
24.0
17.3
67.8
62.5
Prisma VIE-R/32
56.6
67.5
50.2
52.4
28.0
76.8
65.7
02.1
84.5
09.5
89.1
85.0
22.9
16.8
66.0
68.3
WIF-/14 baseline
75.1
78.0
71.5
75.9
67.8
90.6
67.1
73.3
93.4
79.2
93.1
92.0
50.8
33.4
74.2
70.4
VIT-L/14 S. (0.66%)
75.6
77.1
70.2
77.2
71.9
91.0
66.5
74.0
94.0
79.0
93.1
92.0
53.7
34.7
74.0
72.0
ViT-L/14 S.+ (0.47%)
75.1
77.0
69.6
76.8
70.9
90.7
66.1
73.2
94.0
76.1
92.9
91.8
48.5
34.1
73.9
72.9
Table 1: Zero-shot image classification performance of Sparse CLIP models compared to their standard (non-sparse) CLIP counterparts.
BBax Classification
Zero Shot Retrivaal
Model
Sparsity
Ace
Acces
IRe1
Ikee
TRØ1
TR
VIT-L/14 OpenAl
100%
52.3
84.1
36.5
61.0
56.4
79.3
VIT
baseline
100%
53.3
52.2
45.5
70.1
62.7
83.2
Sparse
0.66%
63.5
43.7
68.5
VII-L/14 Sparse+
0.475
56.0
$5.3
41.8
66.4
57.0
80.5
Table 2: Results on additional CLIP downstream tasks. BBox classification simulates CLIP usage in open-vocabulary detection pipelines by classifying ground-truth bounding boxes. Zero-shot retrieval evaluates bidirectional image-caption matching, both on COCO validation set.
that logit scale cap affects sparsity, we train two models with logit scale caps of 50 and 40, producing models with different sparsity levels (0.66% vs. 0.47%). We name them ViT-L/14 Sparse and ViT-L/14 Sparse+. Following Bolya et al. (225).. we evaluate both models against the non-sparse baseline model on comprehensive zero-shot classification benchmarks (Table IT and additional tasks (Table 2 to validate generalization ability.
Sparse CLIP Performance: As shown in Table I our ViT-L/14 Sparse model outperforms the dense baseline on both standard zero-shot classification tasks (+0.5% average across 6 benchmarks) and fine-grained classification tasks (+0.7% average across benchmarks). The ViT-I/14 Sparse+ model performs slightly worse than ViT-L/14 Sparse but remains nearly par with the baseline. On additional downstream tasks (Table D. both ViT-L/14 Sparse and Sparse+ significantly outperform the baseline bounding box classification, yet consistently underperform We hypothesize that the combination of low LO sparsity and the reduced logit scale cap encourages Sparse CLIP focus predominantly on the dominant subject within each image. This behavior may be disadvantageous for COCO retrieval, where captions typically describe multiple subjects simultaneously.
zero-shot retrieval.
Comparison with SAE: For comparison, we evaluate Prisma SAE (Joseph et al. 2025b) on the same zero-shot classification benchmarks (Table Since Prisma SAE's sparse features were trained only on the vision branch, we evaluate using reconstructed dense features. Although trained solely on ImageNet, Prisma SAE generalizes well across domains but suffers 1 1-2% performance loss due to imperfect reconstruction. In contrast, our Sparse CLIP models match or surpass baseline performance while maintaining extreme sparsity.
3 MULTI-MODAL REPRESENTATIONS
Having demonstrated that Sparse CLIP models maintain competitive performance while achieving high sparsity, we now evaluate the interpretability enabled by their sparse representations.
3.1 INTERPRETABILITY EVALUATION
We first address important question: How does Sparse CLIP interpretability compare SAEs? We hypothesize that enforcing sparsity natively during training produces more interpretable features than post-hoc SAE approaches, and we demonstrate this through quantitative metrics.
5

---

Published as a conference paper at ICLR 2026
ImsageNet- train (1.2M)
MetoCLIP subset (800))
Model
Clarity
Active
Clarity Active P%
1.
dataset baseline
0.485
0.422
Prisma clsde11
0.519
40.1%
916.0
0.009
34.8%
1007. .9
Dunjotzs" SAL" 0.521
41.0%
0.450
40.5%
10k
Sporse 0.549
408.5
0.642
89.1%
385.5
VIT-L/14 Sparse+
0.659
80.6%
344.3
0.667
87.7%
281.2
(a) Sparse CLIP embeddings vs. existing open-weights CLIP SAEs across Clarity, feature activation, and LO metrics. ("Memory con- straints limit caching to 10k features per image; Daujotas' SAE exceeds this threshold, preventing exact LO calculation.)
(b) Distribution of Sparse CLIP features by modality score (image activation ra- tio) and activation density (magnitude- weighted frequency).
Figure 3: Quantitative interpretability evaluations of Sparse CLIP embeddings.
Standard SAE evaluation metrics such as Reconstruction Error and Fraction of Variance Unexplained (FVU) are not applicable to Sparse CLIP representations, as they lack decoder for reconstruction. Instead, we employ the Clarity metric proposed by Dreyer et al. (2025). which measures the interpretability of visual embeddings a manner agnostic to dimensionality, architecture, and training procedure.
Clarity quantifies feature interpretability by measuring the average pairwise cosine similarity between images that activate each feature, averaged across all active features. Formally, let I; {o E D a,(x)) > 7 denote the set of images activating feature above threshold т, let Factive = {i: [ 2
Mmin be the set of features with at least Amin activating images, and let sim (m((u,)) v) ||||||||||| denote cosine similarity. The Clarity of visual embedding on dataset D is:
Clarity
Σ FEFFa
Σ RJ. JER
sim(e(z)),e(xx))
(2)
Factivvl
III - 1)
We evaluate Clarity two datasets: the ImageNet- 1k training set (~1. 1.2M images) and subset of MetaCLIP containing 800k images. We employ the original OpenAI CLIP vision encoder as e(x)
and set 7 0.001 and mmin 2 to capture more activated features. For completeness, we also report the percentage of activated features and LO, since Clarity only measures activated features. Table BableBalpon presentss the results. Compared to open- open-weights CLIP SAEs (Joseph al. [022a]] Daujotas) 2024). Sparse CLIP embeddings consistently outperform across all three metrics. While the ViT-L/14 Sparse+ model achieves a lower at the cost of slightly reduced performance, surpasses ViT-L/14 Sparse Clarity, making it primary focus for subsequent interpretability evaluations.
Next, we evaluate whether Sparse CLIP learns genuinely multimodal concepts-feattee that activate for both image and text inputs. Papadimitriou et al. (2025) previously trained SAEs pretrained CLIP's multimodal representations but found most features were unimodal, activating exclusively for either images text. Using the same evaluation methodology we observe a dramatically different outcome. As shown in Figure 3b) Sparse CLIP representations dominated by truly multimodal features. This stark contrast with post-hoe SAE approaches demonstrates a key advantage of native multimodal sparse training-the ability to learn genuinely cross-modal concepts.
ViT-L/14 Sparse+ features,
3.2 ASSIGNING CONCEPT TO FEATURES
Interpreting vision features has long challenged researchers, typically requiring separate classification models or L.M assistance. Sparse CLIP offers a direct solution: we can name features using vocabulary words that trigger maximum activations. If our features are truly multimodal, these ward-based labels should accurately reflect the visual concepts each feature encodes.
To validate this approach, we constructed comprehensive vocabulary with minimal conceptual overlap. We selected the first word from each WordNet synonym set (McCrae et al.) 2019) and merged these with the 20K most common English words (Oikarinen & Weng 2023) which covers informal ones from Internet. This combination ensures broad coverage of both formal and colloquial English, yielding a final vocabulary of 98,619 words.

---

Published as a conference paper at ICLR 2026
CATS
COO
CATS
Wrotham
48997
PV7322
a11
Postare 67828
TART
144
Rpach 100, FETIEE
COME
0331
-1.
Figure 4: Examples features of ViT-L/14 Sparse+. The top activated words and images for these multimodal features are highly corre- lated, so we can directly name their visual con- cept based text.
Figure 5: Evolution of "dog rose" feature. Top activated words and images different stage of training. Activations values are normalized.
We analyzed each ViT-L/14 Sparse+ feature by identifying the top-K words and images that produced the highest activations, using this 98,619-word vocabulary and 80K randomly sampled MetaCLIP images. We found high correlations between text and visual concepts for most features, indicating that Sparse CLIP multimodal features indeed activate on semantically similar content across both shows example features with their top-activated words and images. The "dog" and "cat" features at the first two rows demonstrate robust visual understanding, with top images spanning realistic photos, abstract logos, and OCR text. The "British" feature captures cultural symbols including currency, buildings, and food, while its text activations consist primarily of city names-a a pattern observed in many geographical concepts. Additional examples include an exclusive "David Beckham" feature and mixed "dog rose/rose hips" feature. More interesting example features can be found in Appendix A.2
modalities. Figure f
3.3 CONCEPT EMERGENCE AND EVOLUTION
The last three features in Figure seem to suggest that there may be patterns in how multimodal concepts emerge. Do they begin as separate single-modal features that merge later? Also, will their learnt concepts evolve during training?
While many studies have proposed methods to interpret what concepts CLIP models have learned, few have investigated how concepts emerge and evolve during training. This gap is understandable given the fact that intermediate training checkpoints are typically unavailable for open-source models. However, we found training Sparse CLIP from scratch with native interpretability provides a unique opportunity to examine concept emergence and evolution in an easy-to-visual viiiullll multimodal setting.
We analyzed VYT-L/14 Sparse+ checkpoints 1%. 3%, 10%, 30%, and 100% training completion, visualizing top-activated words and images for each feature at every stage. This reveals feature lifecycles-hov how concepts emerge, develop, and mature throughout training. Our analysis provides insights into not only what CLIP learns, but how it acquires knowledge from large-scale image-text data through contrastive learning.
Observation 1: Concepts emerge multimodal features early in training. Figure 6 shows the modality distribution at 1% training completion, which closely resembles the final distribution most features activate for both images and text from very early stages. The primary difference is activation density: due to lower ttay-trainnng sparsity, densities are an order of magnitude higher than at completion. Examining specific features at 1% completion (Figure 7 confirms strong correlation
7

---

Published as a conference paper at ICLR 2026
Figure 6: Modality distribution of
Figure 7: Same example features figure 4 but at 1% training completion, which after the first training epoch.
Sparse CLIP features similar Fig- ure 3b but at I % training completion.
Vision Encoder
MMMU
AI2D
TextVQA
POPE
ViT-L/14 baseline
bT--eliebseeline
39.6
67.2
48.9
80.8
VIT-L/14 Sparse
40.8
66.4
51.3
82.0
ViT-L/14 Sparse+
41.7
70.6
48.5
80.7
Table 3: Image QA benchmark results. We train VLMs with different CLIP vision encoders, and compare the VLM performance on image QA benchmarks that covers different areas.
between image and text activations. However, some early concepts differ substantially from their final forms, leading to our second observation.
Observation Feature concepts evolve and sometimes completely transform during training. Ana- lyzing top activation trajectories uncovers how concepts evolve during training. Consider the "dog rose" feature (Figure 5) initially activating for red fruits and random creatures, narrows to rose hips mid-training before crystallizing into a dog rose detector that retains hip activations clear evidence of emergent multimodal alignment. We observe similarly striking transformations: feature 40397 pro- gresses "goatee" - "Ryan Gosling" + "David Beckham"; feature 10884 shifts from eye makeup to conjunctivitis; feature 38195 195 evolves from generic soccer players to "Lionel Messi" (Appendix (A.2). These patterns raise fundamental questions about whether transformations are knowledge-dreg driven or noise-driven, their correlation with performance, and their cattrollabiitty-pppnn promising avenues for future work.
APPLICATIONS
CLIP's widespread adoption makes interpretable multimodal representations valuable for applications like controllable image generation, open-vocabulary detection, and image search. To demonstrate our approach's viability, we focus on Vision-Language Models (VLMs) due their prominence and fundamental reliance on CLIP's cross-modal capabilities.
We trained minimal VLM using ViT-L/14 Sparse+ as the vision encoder and Llama 3.1 8B Instruct as the language model, connected via 2-layer MLP adapter. Our VLM training followed staged approach: first pretraining the adapter with frozen vision encoder (Sparse CLIP models) and the LLM (LLama 3.1 8B instruct), then fine-tuning the adapter and LLM together. First stages uses SA-1B-1M (Kirillov
2023) for 2 epoch and second stage used 2.5M image-QU QA pairs from PLM image dataset (Cho et al.) 2025) for 4 epochs. Table Jshows our sparse VLM achieves comparable performance across four benchmarks covering perception, diagrams, hallucination, and general image-based QA.
Our primary demonstration focuses on vision-based steering @osephh al., 2025a). As illustrated in Figure Sa manipulating sparse representation activations prior to the adapter layer enables precise output control-ttransfommin transforming "dog" to "cat" concepts in generated text, or suppressing sensitive concepts like "password" for security applications.
We systematically evaluate steering effectiveness on the complete ImageNet- lk validation set (50k images) using the prompt: "What's the main subject in this image? Your answer needs to be short". Our steering protocol for each image involves two operations: (1) suppressing the ground truth label by zeroing its top-K activated features in the VIT-L/14 Sparse+ text encoder, while (2) boosting
8

---

Published as a conference paper at ICLR 2026
Examplo 1
Example 2
Oaginal biw 68:2
Paerwers pereraar
Gim Orginal bew 52-30
Input
Rardon
Sannirgiatee 68:10
ut Steetrgisbee 50-0.0
Comptten Rutk, 52+1.
"The image depicts
on Itak, 52-2.
Output before
young dog. likely puppy... The dog hus Bully
"Password
genera-
Compltcnn Reto, 50+0.0
steering
tox"
04
feature 8576 (dog):
feature 16340 (password): 1.3618
Steering
1.2129 + 0.0
03
45918 (cat):
0.0 + 2.0
"The image fentures a cur sitting on a The wearing collar
"A blue box with
Output after
white window in the
middle and white input box below the window."
steering
The cat't
foatuee modified
(a) Qualitative examples of concept steering. query the VLM with "What's 's the picture?" while applying steering (steering strength is 2.0 for boosting, 0.0 for suppression) to the features most strongly activated hy target concept words.
(b) Steering evaluation on ImageNet-Ik We mea- sure steering effectiveness (SBERT cosine simi- larity) toward random ImageNet labels across dif- ferent numbers of modified features and Steering Strengths (SS).
Figure 8: Concept steering in vision-language models, with ViT-L/14 Sparse+ as the vision encoder.
randomly selected alternative ImageNet-Il Ik label by setting its top-K activations to Steering Strength (SS). Figure 8b presents performance across different K and SS configurations, revealing the following observations:
Suppressing top-activated concept features effectively removes concepts from outputs. Measured by SBERT cosine similarity, VLM outputs move away from the ground truth label after steering (green lines in Figure tbjtop plot). This effect scales linearly with both the number of modified features and the steering strength of the new concept.
Boosting top-activated features of new concepts effectively steers oulputs toward them, but risks model corruption. Similarly, VL.M outputs move toward the boosted alternative label after steering (red lines in Figure 8b top plot). However, this effect is moderated by model corruption, shown the bottom plot of Figure 8b) and measured by identifying abnormally long responses. At high SS values, corruption risk increases sharply with the number of modified features, causing outputs to diverge in unintended directions. The results suggest that SS = 2.0 provides effective steering while minimizing corruption when modifying only 1-2 features, which we adopt for the examples shown in Figure Sa
LIMITATIONS AND FUTURE WORK
Training for native interpretability: Unlike most mechanistic interpretability research that analyzes existing models, we trained CLIP variants to be natively interpretable. This fundamental difference enables comparable performance while gaining interpretability and revealing concept evolution during training. While insights from our sparse models don't directly transfer to existing dense CLIP models, comparing learned concepts across architectures presents compelling future work.
Interpretability incurs computational costs-our projection layers require significantly more parameters to map representations to digher-dimensionall sparse space. In ViT-L/14, the linear projection from 768 to 768x72 dimensions adds 14% of parameters to the vision tower. Whether performance gains stem from added parameters versus sparsity remains open question for future investigation.
Parameter overhead:
Memory constraints: CLIP training requires gathering batch activations across GPUs for cosine similarity computation. GPU memory becomes a bottleneck for sparse CLIP due to representation scaling. Despite extensive optimization, 72x expansion is the max we can achieve 80GB GPUs.

---

Published as a conference paper at ICLR 2026
Optimizing sparse CLIP training for memory efficiency and throughput remains important research challenge.
6 RELATED WORKS
Improving CLIP training. Since CLIP was first proposed (Radford et al. 2021: Jia et al. 2021).
there has been extensive research aimed at improving its training. Most efforts focus enhancing
the downstream performance of representations by refining the loss functions (Zhai et al. 2023 Li
et al. 2023 Fini et al. 2024) improving the quality of training data (Xu et al. 2024b Schhhmann et al. 2022). Recently, Perception Encoder (Bolya et al.) (025) provided a comprehensive summary
of these advancements and established a new state-of-the- art in performance. In our work, follow
the benchmarks used in Perception Encoder to evaluate our model.
Sparse Autoencoders for CLIP. Sparse Autoencoders (SAEs) were originally developed to analyze
LLMs and have since been applied interpret CLIP models (Lim al., 2025 Zaigrajew al., 2025). Most notably. Joseph et al. (2025b) trained SAEs on each layer of the CLIP vision transformer, compared vision models with LLMs, and systematically evaluated CLIP SAE steering capabilities (Joseph et al. 2025a). Their publicly released SAE weights serve as our primary comparison baseline. More recently. Papadimitriou
al.) (2025) trained SAEs on the multimodal representation space of pretrained CLIP models and found that most features were unimodal. In contrast, our analysis demonstrates that features learned through sparse CLIP training are predominantly multimodal: they are activated by both visual and textual inputs and represent similar concepts across modalities.
Sparse CLIP Representations. Wang et al. (024) showed that introducing non-negativity into contrastive learning induces sparsity, as is equivalent to Non-negative Matrix Factorization (NMF). Although they did not explore increasing the dimensionality to strengthen the representations, their work provides a theoretical foundation for our approach. In a different direction, [hen et al. (2023) constructed sparse CLIP representations by projecting dense features into high-dimensional space predefined by a vocabulary. While this approach yields interpretable and multimodal representations, the predefined space limits the model's s ability to capture high-level semantics beyond the vocabulary, thus restricting its capacity to fully leverage the richness of the training data.
Interpretability and Performance. A common view is that interpretability comes at the cost of accuracy: sparsity and disentanglement can improve feature clarity but reduce reconstruction or predictive power (Olshausen & Field 1996; Higgins et al. 2017 Koh et al. 2020). and recent studies
show that sparsity constraints often hurt training stability downstream accuracy (He et al. 2022
Xie et al. 2024 Nasibullah et al. 2024 Chen et al. 2025 Sawmya et al. 2025). This belief motivates the dominance of post-hoe interpretability methods such Sparse Autoencoders (Elhage et al.) 2022
Cunningham et al., 2023), which are applied to the model after it is trained. In contrast, other work demonstrates that interpretability and performance can be co-optimized: interpretable models can rival black-boo accuracy (Rudin 2019), concept embedding models and modular networks maintain strong performance while improving transparency (Zarlenga et al., [022] Swamy et al., [023]] Good et al., 2023). and prototype-basee based vision models even improve accuracy alongside interpretability (Zhu et al. 2025). Our work extends this latter line to multimodal learning, showing that sparsity imposed during CLIP training yields interpretable features without accuracy loss.
7 CONCLUSION
We introduced Sparse CLIP, which enforces sparsity in CLIP training to produce interpretable multimodal features without sacrificing performance. Unlike post-hoe sparse methods such as Sparse Autoencoders, which often reduce accuracy and collapse multimodal structure, approach preserves performance, and we demonstrate its utility through vision- vision-language steering. This challenges the longstanding view that interpretability and accuracy in tension. If confirmed across architectures and modalities, it suggests a paradigm shift: interpretability and performance need not be trade-offs, but be jointly optimized core design principles.
10

---

Published as a conference paper at ICLR 2026
REFERENCES
Daniel Bolya, Po-Yao Yao Huang, Peize Sun, Jang Hyun Cho, Andrea Madotto, Chen Wei, Tengyu Ma. Jiale Zhi, Jathushan Rajasegaran, Hanoona Rasheed, Junke Wang, Marco Monteiro. Hu Xu, Shiyu Dong, Nikhila Ravi, Daniel Li, Piotr Dollár, and Christoph Feichtenhofer. Perception encoder: The best visual embeddings are not at the output of the network. arXiv, 2025.
Chen Chen, Bowen Zhang, Liangliang Cao, Jiguang Shen, Tom Gunter, Albin Madappally Jose, Alexander Toshev, Jonathon Shlens, Ruoming Pang, and Yinfei Yang. Stair: Learning sparse text and image representation grounded tokens, 2023, URL httpt:::////t //ap:s://arxvv... http:/aawwvv.ggaaa..a../a abs 2301
13081
Matthew Chen, Josh Engels, and Max Tegmark. Low-rank adapting models for sparse autoencoders. In Proceedings of the International Conference on Machine Learning (ICML) - Poster Session, 2025.
Jang Hyun Cho, Andrea Madotto, Effrosyni Mavroudi, Triantafyllos Afouras, Tushar Nagarajan. Muhammad Maaz, Yale Song, Tengyu Ma, Shuminn Hu, Suyog Jain, Miguel Martin, Huiyu Wang, Hanoona Rasheed, Peize Sun, Yao Huang, Daniel Bolya, Nikhila Ravi, Shashank Jain, Tammy Stark, Shane Moon, Babak Damavandi, Vivian Lee, Andrew Westbury, Salman Khan, Philipp Krähenbühl, Piotr Dollár, Lorenzo Torresani, Kristen Grauman, and Christoph Feichtenhofer. Perceptionlm: Open-access data and models for detailed visual understanding, 2025. URL bttps: www.aaavvvoooo /ttp://www...o////////////////
Hoagy Cunningham, Aidan Ewart, Logan Riggs, Robert Huben, and Lee Sharkey, Sparse autoen- coders find highly interpretable features in language models, 2023. URL https:://axxtt
org/abs Stg/abs/2300 2309. 08600
G. Daujotas.
Interpreting and steering features in images.
LessWrong, June
2024.
URL https: wwwwwwow...w.wggooooo..........w com posts http://ptsssssttseeeeee/ /uugeppxxxBBcMMMmm
images
Accessed: 2024-06-20.
Jia Deng, Wei Dong, Richard Socher, Li-Jia Li, Kai Li, and Li Fei-Fei. Imagenet: A large-scale hier- archical image database. In 2009 IEEE Conference on Computer Vision and Puttern Recognition, pp. 248-255, 2009. doi: 10.1
Maximilian Dreyer, Jim Berend, Tobias Labarta, Johanna Vielhaben, Thomas Wiegand, Sebas- tian Lapuschkin, and Wojciech Samek. Mechanistic understanding and validation of large AI models with Semanticlenss. Nature Machine Intelligence, 70):1572-1588, September 2025. ISSN 2522-5839, doi: 10.10388/44225--25--00088 s42256 025 01084
Nelson Elhage, Tristan Hume, Catherine Olsson, Nicholas Schiefer, Tom Henighan, Shauna Kravec, Zac Hatfiel-Doddss Robert Lasenby, Dawn Drain, Carol Chen, Roger Grosse, Sam McCan- dlish, Jared Kaplan, Dario Amodei, Martin Wattenberg, and Christopher Olah. Toy models of superposition, 2022.
10652
Eoin Farrell, Yeu-Tong Lau, and Arthur Conmy. Applying sparse autonncoooccuuuooooooocc to unlearn knowledge in language models, 2024. URL
Thomas Fel, Victor Boutin, Mazda Moayeri, Rémi Cadène, Louis Bethune, Léo andéol, Mathieu Chalvidal, and Thomas Serre. A holistic approach unifying automatic concept extraction and concept importance estimation, 2023, URL htLp:t/s:: www.v.iiiiiv.... org. http:///..//.....o// 02.0447344
Enrico Fini, Mustafa Shukor, Xiujun Li, Philipp Dufter, Michal Klein, David Haldimann, Sai Aitharaju. Victor Guilherme Turrisi da Costa, Louis Béthune, Zhe Gan, Alexander Toshev, Marein Eichner, Moin Nabi, Yinfei Yang, Joshua M. Susskind, and Alaaeldin Bl-Nouby. Multimodal autoregressive pre-training of large vision encoders, 2024. HtRLhttps::// http::s:///xxvvvo///// orb/ 2411. 14402
Leo Gao, Tom Dupré la Tour, Henk Tillman, Gabriel Goh. Rajan Troll, Alec Radford, Ilya Sutskever, Jan Leike, and Jeffrey Wu. Scaling and evaluating sparse autoencoders, 2024. URL https:
7/ //axxiv.. wraivioorrraaeee ab2404//a/224666. 04093
11

---

Published as a conference paper at ICLR 2026
Jack Good, Torin Kovach, Kyle Miller, and Artur Dubrawski. Feature learning for interpretable, performant decision trees. Advances in Neural Information Processing Systems, 36:66571-66582 2023.
Jeff Z. HaoChen, Colin Wei, Adrien Gaidon, and Tengyu Ma. Provable guarantees for self-supervised deep learning with spectral contrastive loss, 2022. URL
04156
Kaiming He, Xinlei Chen, Saining Xie, Yanghao Li, Piotr Dollár, and Ross Girshick. Masked autoencoders are scalable vision learners. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2022.
Dan Hendrycks and Laura Hiscott. The misguided quest for mechanistic ai interpretability. AI Frontiers guest commentary, May 2025. URL /ttttps:/aai wwwpa/irrrrrorrrrriilleeee st. c- -tt.:ccala-nntterrrttaallll
Che-mis mes-misguiion iedd -wweiiddd-queu---eee -urst-uuusttuttueeee for-ttt--ooo-ooocaee
Irina Higgins, Loic Matthey, Arka Pal, Christopher Burgess, Xavier Glorot, Matthew Botvinick, Shakir Mohamed, and Alexander Lerchner. beta-VAE: Learning basic visual concepts with constrained variational framework. In International Conference on Learning Representations, 2017. URL htttps://operreviertt net/ http://awwrrieeeeeeeeoooooaaaaaa
Gabriel Ilharco, Mitchell Wortsman, Ross Wightman, Cade Gordon, Nicholas Carlini, Rohan Taori, Achal Dave, Vaishaal Shankar, Hongseok Namkoong, John Miller, Hannaneh Hajishirzi, Ali Farhadi, and Ludwig Schmidt. Openclip, July 2021. URL http:::tt///dltt http:///ooooo./...2222222 52817 If you use this software, please cite it as below.
zerodo. 510377.3377
Chao Jia, Yinfei Yang, Ye Xia, Yi-Ting Chen, Zarana Parekh, Hieu Pham, Quoe V. Le. Yunhsuan Sung, Zhen Li, and Tom Duerig. Scaling up visual and vision- representation learning with noisy text supervision, 2021.
05918
Sonia Joseph, Pranect Suresh, Ethan Goldfarb, Lorenz Hufe, Yossi Gandelsman, Robert Graham, Danilo Bzdok, Wojeiech Samek, and Blake Aaron Richards. Steering clip's vision transformer with sparse autoencoders, 2025a. URL
abs 2504 08729
Sonia Joseph, Praneet Suresh, Lorenz Hufe, Edward Stevinson, Robert Graham, Yash Vadi, Danilo Bzdok, Sebastian Lapuschkin, Lee Sharkey, and Blake Aaron Richards. Prisma: An open source toolkit for mechanistic interpretability in vision and video, 2025b. URL abs 220///5544....7
Subhash Kantamneni, Joshua Engels, Senthooran Rajamanoharan, Max Tegmark, and Neel Nanda. Are sparse autoencoders useful? a case study in sparse probing, 2025. URL htth:tt////aiiiii org absp//22.22222.. 16681
Alexander Kirillov, Eric Mintun, Nikhila Ravi, Hanzi Mao, Chloe Rolland, Laura Gustafson, Tete Xiao, Spencer Whitehead, Alexander C. Berg, Wan-Yen Lo, Piotr Dollár, and Ross Girshick. Segment anything. arXiv:2304.0266 2023.
Pang Wei Koh, Thao Nguyen, Yew Siang Tang. Stephen Mussmann, Emma Pierson, Been Kim, and Percy Liang. Concept bottleneck models. In Hal Daumé III and Aarti Singh (eds.), Proceedings of the 37th International Conference on Machine Learning, volume 119 of Proceedings of Machine Learning Research, pp. 5338-5348. PMLR, 13-18 Jul 2020. URL WRRLittps///ppcoeeeee mlr. wtt.ppsssllv111k222222 html
Yanghao Li, Haoqi Fan, Ronghang Hu, Christoph Feichtenhofer, and Kaiming He. Scaling language- image pre-training via masking, 2023. lttps:///aaa http://wwww..orr//////222222 007
Hyesu Lim, Jinho Choi, Jaegul Choo, and Steffen Schneider. Sparse autoencoders reveal selec- tive remapping of visual concepts during adaptation. In The Thirteenth International Confer- ence on Learning Representations, 2025. URL httttt:///ppeeeee///////
cettfforumidd
imT03YX1
12

---

Published as a conference paper at ICLR 2026
John P. McCrae, Alexandre Rademaker, Francis Bond, Ewa Rudnicka, and Christiane Fellbaum. English WordNet 2019 - an open-source WordNet for English. In Pick Vossen and Christiane Fellbaum (eds.), Proceedings of the 10th Global Wordnet Conference. pp. 245-252, Wroclaw, Poland, July 2019. Global Wordnet Association. URL
www.:llyy.oo.oo...oo......o org/2019
gwe- .317
Nasib Nasibullah, Erik Schultheis, Mike Lasby, Yani Ioannou, and Rohit Babbar. Navigating extremes: Dynamic sparsity in large output spaces. In Advances in Neural Information Processing Systems (NeurlPS), volume 37, 2024.
Tuomas Oikarinen and Tsui-Wes Weng. Clip-disscct: Automatic description of neuron representations in deep vision networks, 2023. (RRphts:://axiii http://aaxvv...rg/a///////////// 10965
Bruno A Olshausen and David J Field. Emergence of simple-cell receptive field properties by learning sparse code for natural images. Nature, 381(65833)607--609 1996.
Isabel Papadimitriou, Huangyuan Su, Thomas Fel, Sham Kakade, and Stephanie Gil. Interpreting the linear structure of vision-language model embedding spaces, 2025. URL https:///axxi
org/abs/20 2504. 11695
Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, Gretchen Krueger, and Ilya Sutskever. Learning transferable visual models from natural language supervision, 2021. URL https
00020
Cynthia Rudin. Stop explaining black box machine learning models for high stakes decisions and use interpretable models instead, 2019. URL https:::t///......... http::://axxivvorrrg org/ abs 1811 10154
Shashata Sawmya, Linghao Kong, Ilia Markov, Dan Alistarh, and Nir N. Shavit. Wasserstein distances, neuronal entanglement, and sparsity. In Proceedings of the International Conference on Learning Representations (ICLR), 2025.
Christoph Schuhmann, Romain Beaumont, Richard Vencu, Cade Gordon, Ross Wightman, Mehdi Cherti, Theo Coombes, Aarush Katta, Clayton Mullis, Mitchell Wortsman, Patrick Schramowski. Srivatsa Kundurthy, Katherine Crowson, Ludwig Schmidt, Robert Kaczmarczyk, and Jenia Jitsev. Laion-5b: An open large-scale dataset for training next generation image-text models, 2022. URL https::/arxxts www.ooggaaa.... 221 08402
Vinitra Swamy, Malika Satayeva, Jibril Frej. Thierry Bossy, Thijs Vogels, Martin Jaggi, Tanja Käser, and Mary-Anne Hartley. Multimodn- multimodal, multi-task, interpretable modular networks, 2023. URLPttpp. / //tp:vv//xiivia//a wwwp:iiwwrg/bb.aaaoa 2309.14111
Yifei Wang, Qi Zhang, Yaoyu Guo, and Yisen Wang. Non-negative Contrastive Learning, April 2024. URL
arXiv:2403. 12459 [cs].
Jingjing Xie, Yuxin Zhang, Mingbao Lin, Liujuan Cao, and Rongrong Ji. Unipts: unified framework for proficient post-training sparsity. In Proceedings of the IEEE/CVE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 5746-5755, June 2024.
Hu Xu, Po- Yoo Huang, Xiaoqing Ellen Tan, Ching-Feng Yeh, Jacob Kahn, Christine Jou, Gargi Ghosh, Omer Levy, Luke Zettlemoyer, Wen tau Yih, Shang- Wen Li, Saining Xie, and Christoph Feichtenhofer. Altogether: Image captioning via re-aligning alt-text, 2024a. URL https
//arxiv.org/ab//2100111
Hu Xu, Saining Xie, Xiaoqing Ellen Tan, Po- Yo-Yao Huang, Russell Howes, Vasu Sharma, Shang- Wen Li, Gargi Ghosh, Luke Zettlemoyer, and Christoph Feichtenhofer. Demystifying clip data, 2024b.
URL wtt.hhtst//..ii..cooo http:///aaxxv..ooaa//////// 2ttp:///b///30...../ 16671
Vladimir Zaigrajew, Hubert Baniecki, and Przemyslaw Biecek. Interpreting clip with hierarchical sparse autoencoders. arXiv preprint arXiv:2502.20578. 2025.
13

---

Published as a conference paper at ICLR 2026
Mateo Espinosa Zarlenga, Pietro Barbiero, Gabriele Ciravegna, Giuseppe Marra, Francesco Giannini, Michelangelo Diligenti, Zohreh Shams, Frederic Precioso, Stefano Melacci, Adrian Weller, Pietro Lio. and Mateja Jamnik. Concept embedding models: Beyond the accuracy-explaimabliyy trade-off, 2022. URL _ http:pps///aa....o/////aaaa/////
09056
Xiaohua Zhai, Basil Mustafa, Alexander Kolesnikov, and Lucas Beyer. Sigmoid loss for language image pre- pre-training, 2023. htt.:/tss///aavv../////////////0
Zhijie Zhu, Lei Fan, Maurice Pagnucco, and Yang Song. Interpretable image classification via non- parametric part prototype learning, 2025, URL
wttp://////a/b33333////// 2503 10247
A
APPENDIX
A.1
SMALL-SCALE ABLATION STUDY SETUP
For Sparse CLIP small-scale ablation studies, follows OpenCLIP original setup for most of the hyperparameters, except for the following.
Training Dataset: We uses MetaCIIP the source of training data. It's large-scale dataset containing around 2.2 billion image-text pairs collected from the Internet and indexed by Common- Crawl, then curated following the methodologies described in Xu et al. (2024b) and Xu al. (2024a). For our preliminary experiments, we utilize a randomly sampled subset of MetaCLIP containing approximately million samples, and train the model for 30 epochs on this dataset.
Model Architecture: For our preliminary experiments, we employ the default ViT-B/32 configuration from OpenCIII to enable fast training while maintaining reasonable model capacity. This architecture uses 12-layer Vision Transformer (ViT) for both the vision and text encoders, with an embedding dimension of 512.
Input Image Processing: Input images are resized to 224x224 pixels for efficient training. Non- square images are resized based on their shortest edge and then center-cropped to preserve aspect ratios.
Loss Function: We apply the standard CLIP loss, which computes the average cosine similarity loss on both visual and text embeddings during training.
Representation Dimensionality: We use a 32x dimension expansion (16,384 dimensions for ViT- B/32) unless otherwise specified in ablation studies.
Batch size and Learning rate: Following OpenCLIP's configuration, we use a learning rate of 5e-4 and global batch size of 32K, implemented via either 512x64 or 128x256 (batch size per GPU GPUs).
Evaluation Metrics: We use zero-shot classification accuracy the ImageNet-IK IK (Deng et al. 2009) test set as the primary metric for evaluating model performance during training. For measuring sparsity, average L0 value of the activation.
A.2 ADDITIONAL QUALITATIVE RESULTS
A.2.1 FEATURE VISUALIZATION
As mentioned in Section 3.2
show visualization for more interesting example features here in
Fig g
A.2.2 FEATURE EVOLUTION
Many features exhibit fascinating evolution trajectories during training. We provide detailed visual- izations for the representative examples mentioned in Section 3.3
14

---

Published as a conference paper at ICLR 2026
JG
ENp in Adb perTe
58tH
Feature 36195
2345
1,823
1654
Festure 20072
1.577
3.737
1,446
BD
Strral App
Signal
THE SIGNAL
Segnabure
Peature 35747
3,397
UNM FRIENTS
1992
L.044
3.55/
1.922
DITTUL MANOE
Gome:
ST ENDS
FREENDS SWINMING
Figure 9: More example ViT-L/14 Sparse+ features with their highest activated words and images. Notice that we show un-normalized activation values here.
med tere
of
rstacces
453 1o06
FULR 6.322
Figure 10: full picture for evolution of the "dog rose" feature.
15

---

Published as a conference paper at ICLR 2026
Fita 2011
Epoch 30,
E.3400
Epech
03435
Figure 11: Evolution of the "David Beckham" feature. It starts as a feature for "beard", turned into "gosling" in the middle of training, and finally become dedicated "David Beckham" feature.
MIBE
2
CRACKBACHI
1.0
Figure 12: Evolution of the "Ryan Gosling" feature. become Gosling at the same time when the original Gosling feature taken by David Beckham, shown in Figure
16

---

Published as a conference paper at ICLR 2026
spach FAUHOO
B4228
X32G
EL1023
0.1554
Spoch FLOSB4
02424
0J395
0.19
poch 10, FLOBSSS
0,335
03313
02993
0282
0.20W
0.3454
6.000
Spoch 80, F10854
22567
B.2099
02/61
0244
02:08
$/299
Epoch 186, 11084
0,77
0,282 02518
02333
0.222 022
0208 6:199
Figure 13: Evolution of the "conjunctivitis" feature: Initially, this feature activates for eye makeup products but eventually converges to "conjunctivitis" text while still activating for some eye makeup images. This suggests the model may struggle to distinguish between eye makeup and inflamed eyes.
Epech N47665
0,2278
E210L
0,222
02123
Epoch N7565
30001
6.3093
030000
RESERVE ME Σ09 BJMG
Epach 20, A67563
23126
0.37
0.3333
3522
0.343
Epach 20, M7765
0.34
6:311
CHEAPEST
MOBILE TARIPS GUARANTEED
Spoch 580, 187949
0.394
2667
0.36
0,55
CALERNATIONS
EADERS
COUR
Figure 14: Evolution of the "British" feature. The feature begins by activating for images of British-style houses during early training, then gradually generalizes to represent a wider range of geographical and cultural concepts.
17

---

Published as a conference paper at ICLR 2026
poch FORING
ELPPAN
CANYY
EYMAR TOP
H
AALEEVER
0-2
Ppowch E08399
6.10%
EYMAR TOP
MESI
FIEE 1CC
MALSEVER
Fipcch 10, FARINS
1.7999
0.293
0:209
2ES1
0.21
0281
VETEE 0J3
Poucc SO, 158399
3.5011
0.5997
03:55
0.54
0.5485
Toech 300, 130199
0.0032
0.153
3.2048
0.3991
3028
5.3613
0.28
03647
Pyprrt/ian Lan
Figure 15: Evolution of the "Messi" feature. It starts an image-only feature for group of sports pictures, turned into soccer only feature in the middle of training, and finally became the dedicated "Messi" feature on both modalities.
18


Reviews:
[{"review_id": "ac9MczJlSw", "reviewer": "Reviewer_SVSv", "summary": "This paper introduces Sparse CLIP, a experimental modification and integration to the standard CLIP framework that incorporates sparsity during training.", "questions": "1. Introduce quantitative interpretability metrics beyond classification metrics.\n\n2. Hard case testing and failure case analysis. For example, does Sparse CLIP produce meaningful features although it misclassifies an object? What is the performance of Sparse CLIP on more complex datasets?\n\n3. Clarify contributions.", "limitations": "", "rating": 4, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "1. The paper incorporates sparsity into training, it achieves both interpretability and high performance, offering a compelling design.\n\n2. Comprehensive experiments demonstrate that Sparse CLIP performs comparably to dense CLIP on zero-shot classification tasks and surpasses post-hoc Sparse Autoencoders (SAEs) in interpretability.\n\n3. Applications in Vision-Language Models: This paper presents a practical demonstration of Sparse CLIP’s utility and relevance for downstream tasks.\n\nIn summary, this paper offers intriguing experimental results and new insights into sparse training than providing a clear technical contribution or a robust methodological advancement.", "weaknesses": "1. While the findings are interesting, the proposed method is totally established on the existing works, making it relatively simple and lacks significant novelty.\n\n2. Unclear generalizability: While Sparse CLIP performs well on zero-shot classification benchmarks, it is unclear how well the method generalizes to downstream tasks as the other CLIP models, such as object detection, segmentation, or open-vocabulary retrieval", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "M2YVOSNiLD", "reviewer": "Reviewer_BAoM", "summary": "This paper proposes training a vision-language encoder (CLIP) with a drastically expanded embedding dimension (in this case, by a factor of 72). The goal is to learn a sparse representation allowing for interpretability and steering. Experimental results show that Sparse CLIP is able to perform well in zero-shot classification on ImageNet, as well as its representation can be interpreted through concept discovery.", "questions": "1. The sentence in L200-201 \"After conducting comprehensive ablation studies (additional results in Appendix A.3),\" is wrong. There are no \"comprehensive ablation studies\" in Appendix A.3. It only shows Figure 9 with a single experiment on \"ViT-L/14 Sparse+\".\n2. Appendix A.2 mentions an \"interactive web visualization to access visualizations\" but the demo is empty, i.e. says \"Cool Sparse CLIP visualizations\".\n3. Why did you choose the dimension expansion factor of 72? Comprehensive ablations are essential when proposing a method with such a critical parameter. Why does the Appendix say \"32x dimension expansion\"?\n4. Table 1 should include results for ViT-B/32 Sparse.\n\n**Other feedback**:\n- Use `\\citep` correctly instead of `\\citet` (many such cases e.g. in L153-157).\n- L137: add \"equation\" to \"1\"\n- L149: typo in \"architectures\"\n- Figure 1: \n    - Be more specific in the caption, e.g. add the information about \"OpenCLIP (ViT-B/32) trained on MetaCLIP\".\n    - Improve colors in (c); use an actual gradient, e.g. yellow-orange-red-purple. \n    - Graphs are pixelated; improve their quality, e.g. export them to PDF.\n- Figure 9 (Appendix A.3): It is completely unclear what \"ViT-L/14 Sparse+\" or this result means when linking here from L200.\n- L206: \"After demonstrating proof of concept on smaller models and datasets,\" reads like an overstatement, because it was a single model (ViT-B/32) and data subset.\n- L212: typo in \"(0.66Following\"\n- Table 1/L232: Again, it is unclear what does \"+\" in \"ViT-L/14 Sparse+\" mean.\n- L244: Again, \"Having achieved satisfactory performance and sparsity\" reads like an overstatement, because the model was trained with a single hyperparameter setting and evaluated only on the zero-shot classification task. We know little regarding the generalizability of the approach.\n- Table 3 (Appendix A.1.2) should be a part of main text.\n- L420: What does \"Not on existing model\" mean?", "limitations": "", "rating": 4, "confidence": 4, "soundness": 2, "presentation": 1, "contribution": 4, "strengths": "This paper's strengths lie in the novelty and simplicity of the method:\n1. The idea to train a CLIP with an inherently sparse representation is very interesting and can lead to significant progress in research on interpretability. It is simple and could be easily translated to other bi-modal encoders trained with the contrastive loss.\n2. I appreciate the paper's flow, especially the 'lessons learned' given in Section 2.2, articulating the research process progressing into Section 2.3.", "weaknesses": "Overall, this work is in an early stage, requiring revision and extension to be considered for publication:\n1. Regarding soundness, the experiments are limited (see questions below). Contribution 3 (implementing interpretable\nvision-based steering in vision-language models) executed in Section 4 is overstated significantly: the two examples shown in Table 2 cannot reliably demonstrate real-world applicability.\n2. The paper requires proofreading (see feedback below; I just stopped listing at some point). Several elements in the paper appear to be missing (see questions), and I do not believe that mentioning additional content exists in the Appendix and then incorporating it during the rebuttal would be an acceptable practice.\n3. Furthermore, the comparison with the state-of-the-art is missing. Section 3 could follow experimental evaluation from the omitted related work on training SAEs for CLIP [a,b] instead of comparing to a TopK SAE with a single $k$ value (not to mention that the Prisma SAE uses a different baseline model).\n4. The paper should include code to reproduce the results. \n\n[a] Sparse autoencoders reveal selective remapping of visual concepts during adaptation. ICLR 2025\n\n[b] Interpreting CLIP with hierarchical sparse autoencoders. ICML 2025", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "yGwkhzeagc", "reviewer": "Reviewer_sRWz", "summary": "This work addresses the interpretability limitations of CLIP’s dense multimodal representations. The authors propose integrating sparsity directly into CLIP training to create “Sparse CLIP features” that remain highly performant on downstream tasks while becoming significantly more interpretable. Unlike post-hoc sparse autoencoder methods, this approach preserves CLIP’s inherent multimodal alignment and reveals transparent feature evolution and concept emergence during training. Experiments further demonstrate practical benefits by enabling interpretable visual-language control in downstream applications. The results challenge the common belief that interpretability must come at the cost of performance, offering a promising direction for building multimodal models that are both accurate and understandable.", "questions": "The paper presents many interpretability examples and analyzes the distribution of Sparse CLIP features, yet it lacks more fine-grained analysis that connects specific feature activations to corresponding image regions or pixels. For instance, applying masking-based methods could enable more detailed ablations to identify which parts of the image activate particular features. I hope the authors can include some cases demonstrating this aspect.\n\nIf the authors provide reasonable and satisfactory responses to the Weaknesses and Questions, I would consider increasing the score.", "limitations": "", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 2, "contribution": 3, "strengths": "1. This paper proposes the SPARSE CLIP method, which effectively combines the strengths of both CLIP and SAE models, ensuring interpretability and performance in contrastive learning.\n2. The paper provides a solid analysis of the interpretability achieved by combining large models with SPARSE CLIP. The cases presented in paper demonstrate the strong interpretability of SPARSE CLIP as well as its powerful application potential.", "weaknesses": "The proposed SPARSE CLIP method exhibits strong interpretability, but its performance drops on many benchmarks compared to the original ViT-based CLIP baseline. I hypothesize that this is due to CLIP’s reliance on maintaining rich and continuous directional information in the embedding space. The discontinuity introduced by ReLU indeed produces highly sparse and interpretable features, but it also collapses many feature dimensions, resulting in sparse vectors that weaken cross-modal alignment and semantic expressiveness. A potential solution would be to decouple the interpretable sparse features from the output by employing an independent sparse decoder.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "8o5h1dlLj6", "reviewer": "Reviewer_6eqm", "summary": "This paper challenges the prevailing assumption that interpretability must come at the cost of performance in representation learning. The authors propose Sparse CLIP, a method that seamlessly co-optimizes for both objectives by integrating sparsity directly into the Contrastive Language-Image Pre-training (CLIP) framework. The approach is notably simple, requiring only two key modifications to the standard architecture: a significant expansion of the final projection layer's dimensionality and the introduction of a ReLU activation function. This combination effectively induces a sparse, high-dimensional representation space without altering the core contrastive learning objective.\n\nEmpirical results demonstrate that Sparse CLIP not only matches but in some cases surpasses the performance of its dense counterparts on zero-shot classification benchmarks, while achieving extreme sparsity. Crucially, the learned features are natively multimodal, activating for semantically aligned concepts across both image and text modalities. This enables direct concept-level interpretability, allowing features to be named and analyzed based on their top-activating words and images. The paper further reveals intriguing training dynamics, tracing how multimodal concepts emerge and evolve, and showcases the practical utility of these interpretable features through vision-based steering in a downstream Vision-Language Model (VLM), enabling controlled generation and security filtering.", "questions": "1. In 2.3, the distinction between the training settings for Sparse and Sparse+ is not clarified. Do they merely employ different logit scale caps? If so, based on the conclusions from small scale training, is the Sparse+ model trained using a logit scale cap of 40?\n2. Model Scale: Since the final projection layer is significantly enlarged, could the performance gain be attributed primarily to the increase in parameters rather than the sparsity mechanism? It is necessary to supplement the Sparse CLIP model with a set of ablation experiments—specifically, removing the ReLU activation function and dimension expansion individually.\n3. Association between text and features: The paper explains that one should first establish a vocabulary list, then identify the corresponding relationships. This demonstrates dependence on vocabulary quality for concept labeling.\n    1. Can the model interpret multi-word phrases, or is it limited to single tokens?\n    2. How is it verified that the top-activated word accurately captures the feature's semantic role?\n    3. What happens if the underlying concept is not contained in the vocabulary?\n4. Steering Sensitivity: In the \"dog\"→\"cat\" steering example, the \"cat\" feature is set to 2.0.\n    1. How was this value chosen? Would other values (e.g., 1.0 or 10.0) lead to unpredictable generation?\n    2. Is there a systematic method for determining safe and effective intervention magnitudes?", "limitations": "", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1. Simplicity and Effectiveness: The method simply increases the dimensionality of the final projection layer and adds an activation of ReLU. The result shows that it achieves interpretability and better performance.\n2. Refutation of a prevailing assumption: It provides a good counterexample for \"training-time sparsity compromises accuracy\".\n3. High application potential: The paper demonstrates performance enhancements in downstream applications, as well as new application scenarios such as steered generation and security filtering.", "weaknesses": "1. Theoretical innovation is limited: One of the key techniques (ReLU activation) was proposed by NCL. It lacks a theoretical analysis of dimensionality expansion. The emphasis is more on applications in multimodal setting.\n2. GPU memory and model parameters overhead: The model significantly increases the dimensionality of the final projection layer, resulting in substantially more parameters and the additional GPU memory required for training. Due to limitations of the GPU memory, it is impossible to further research the impact of dimensionality.\n3. Limited evolution for downstream task: Only four fine-tuning tasks are reported, which is insufficient to robustly support the claim that the method \"maintains strong performance on downstream tasks\" across a broad range of applications.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)