MICLIP: Learning to Interpret Representation in Vision Models
Authors: Yingdong Shi
Zhiyu Yang
Changming Li
Jingyi Yu
Kan Ren
Venue: ICLR 2026 Poster (2026)
Keywords: mechanistic interpretability
contrastive learning
sparse autoencoder
Primary area: interpretability and explainable AI
TL;DR: A method uses contrastive learning to provide causal-aware interpretability, offering deep insight and control over vision model mechanisms.

Abstract: Vision models have demonstrated remarkable capabilities, yet their decision-making processes remain largely opaque. Mechanistic interpretability (MI) offers a promising avenue to decode these internal workings.  However, existing interpretation methods suffer from two key limitations. First, they rely on the flawed activation-magnitude assumption, assuming that the importance of a neuron is directly reflected by the magnitude of its activation, which ignores more nuanced causal roles. Second, they are predominantly input-centric, failing to capture the causal mechanisms that drive a model's output. These shortcomings lead to inaccurate and unreliable internal representation interpretations, especially in cases of incorrect predictions. We propose MICLIP (Mechanism-Interpretability via Contrastive Learning), a novel framework that extends CLIP’s contrastive learning to align internal mechanisms of vision models with general semantic concepts, enabling interpretable and controllable representations. Our approach circumvents previous limitations by performing multimodal alignment between a model's internal representations and both its input concepts and output semantics via contrastive learning. We demonstrate that MICLIP is a general framework applicable to diverse representation unit types, including individual neurons and sparse autoencoder (SAE) features.  By enabling precise, causal-aware interpretation, MICLIP not only reveals the semantic properties of a model's internals but also paves the way for effective and targeted manipulation of model behaviors.

Published as a conference paper at ICLR 2026
MICLIP: LEARNING TO INTERPRET REPRESENTATION IN VISION MODELS
Yingdong Shi; Zhiyu Yang; Changming Li, Jingyi Yu, Kan Ren ShanghaiTech University (shi (dh2y3 yangzhy22022, renkan] rennkanhssannghaiteechccc
ABSTRACT
Vision models have demonstrated remarkable capabilities, yet their decision- making processes remain largely opaque. Mechanistic interpretability (MI) of- fers a promising avenue to decode these internal workings. However, existing interpretation methods suffer from two key limitations. First, they rely on the flawed accivation-magniiuue assumption, assuming that the importance of neu- ron directly reflected by the magnitude of its activation, which ignores more nuanced causal roles. Second, they are predominantly input-centric, failing capture the causal mechanisms that drive model's output. These shortcom- ings lead to inaccurate and unreliable internal representation interpretations, pecially in cases of incorrect predictions. We propose MICLIP (Mechanism- Interpretability via Contrastive Learning), novel framework that extends CLIP's contrastive learning to align internal mechanisms of vision models with general semantic concepts, enabling interpretable and controllable representations. Our approach circumvents previous limitations by performing multimodal alignment between model's internal representations and both its imput concepts and out- put semantics via contrastive learning. We demonstrate that MICLIP is gen- eral framework applicable to diverse representation unit types, including indi- vidual neurons and sparse autoencoder (SAE) features. By enabling precise, causal-aware interpretation, MICLIP not only reveals the semantic properties of model's internals but also paves the way for effective and targeted ma- nipulation of model behaviors. Our project homepage is available at https: I/Eoundation-mmdde www.errchrreeeeecccmmo qithub Ballli..//MMM 1o/MICLIP
INTRODUCTION
Mechanistic interpretabiliry (MI) (Zeiler & Fergus 2014 Oikarinen & Weng 2023) on vision mod- els (Dosovitskiy al.) 2021 Rombach al. (022) offers a promising avenue for making models more transparent and controllable. MI aims to uncover models' internal mechanisms, e.g., hidden representations and computational circuits, and relate them both model behavior and human- understandable concepts. For instance, Waan et al. (2025) shows that tracing the successive trans- formation of inputs into outputs can reveal how object concepts are processed within vision models. Furthermore, once such mechanisms are identified, they can be systematically manipulated to steer models toward desired behaviors (Li et al. 2024 Ferrando et al.] 2025) Shi et al., 2025).
Researchers have proposed range of interpretability methodologies decode the internal repre- sentations of vision models into human- unt--unannrrtbbleee semantics (Olah et al.) (017)). Early efforts largely relied manual annotation and feature visualization (Zeiler & Fergus 2014 Selvaaal
al. 2017). More recent work instead analyzes units within the hidden representations, such as individual neurons, and associates them with semantic concepts by measuring the co-occurrence between high neuron activations and the presence of corresponding concepts in input images (Bau et al. 2017
Oikarinen Weng 2023 Zhang et
2024 Bai et 2025)
Despite these advances, existing approaches face notable limitations. First, most prior work relies on the activation maririuud assumption: for any representation unit, larger activation value inter-
Equual contribution.
"ooresponneecc Kan Ren.

---

Published as a conference paper at ICLR 2026
preted indicating stronger presence of the unit's associated concept in the model's information- processing pipeline. However, a neuron's contribution to model behavior is often more complex. An increase in activation value does not necessarily imply the occurrence of the corresponding cept during inference. Conversely, even negative activations can positively influence the model's prediction of certain concepts. Second, existing methods predominantly input-centric, focusing on aligning internal representations with concepts present in the input. This paradigm introduces several issues, it is not grounded in the causal mechanisms that actually drive model behavior (Gur-Arieh et al. 2025). Such limitations are particularly evident when the model produces incor- rect predictions. Similarly, input-centric methods (Bau al., 2017] Zhang et al.) [0244] Bai et al.] 2025) fail in cases of incorrect predictions, as they do not capture the intrinsic causal mechanisms underlying the model's decision-making.
In this paper, we introduce MICLIP, an MI framework for vision models from novel functionality perspective on the model's rep- resentation unite" Rather than relying on the conventional activation-magee magnniude assumption, MICLIP represents a target unit, such as an individual neuron a sparse feature (Huben eet al.] 2024), as a semantic vector embedded within human-uumenete understaadddseeeeleeeeeee space, such as the CLIP (Radford et al. (202I) semantic space. By directly measuring the semantic relatedness of internal units to specific concepts, our ap- proach bypasses the limitations of activation- magnitude-basee methods.
Representation Correlation
Image
Model
XN
Actvoston Value
Label: Dog
Input-centric
Large!
Output
Correlation
Neuron
Figure 1: Traditional input-centric methods iden- tify neurons by correlating their activation magni- tudes with the given input label (e.g., "Dog"). In contrast, our framework instead establishes corre- lations in the representation space and takes both the input and the output into consideration.
To achieve this, MICLIP performs multimodal alignment between hidden representation units and concepts derived from both inputs and out- puts of the model. In particular, contrastive learning (Radford al. 2021) is employed to ground feature functionality with respect to input concepts and output semantics. This dual grounding inte- grates input- and output-centric perspectives, thereby revealing the causal trajectory of information processing: from input, through internal units, to model outputs as shown in Figure [ In doing so, MICLIP not only provides more faithful interpretability of the mechanisms underlying model behavior, but also enables direct model steering by manipulating concept-aligned internal units.
Our work advances MI study in vision models with key contributions listed follows: (i) To the best of our knowledge, this is the first study on learning semantic representations of the model internal features and aligning them with numan-unnderstaddale semantic spaces. This multimodal grounding provides a unified and generalizable interpretability framework across diverse vision architectures. (ii) Unlike prior input-centric explanation or attribution methods. MICLIP incorporates both input- and output-groundh
semantics, aligning the entire reasoning trajectory of the model with human concepts. (iii) MICLIP applies broadly to different forms of internal representation units, including individual neurons and SAE features (Huben et al., 2024 Gao et al., 2025). offering versatile tool for understanding and steering vision models.
BACKGROUND AND RELATED WORK
Existing approaches that aim to associate textual concepts with internal representation units in vision models remain largely input-centric and can be grouped into two main trends.
Activation- The first line of work selects inputs that strongly activate a neuron or crops out the highly activating region (Kalibhat al. (023) and then interprets the recurring patterns. This can be done by correlating activations with annotated concepts (Bau et al. 2017). or by leverag- ing pretrained models to automatically generate textual descriptions for the highly activating in-
puts (Hernandez et al. 2021 Kalibhat et al. 2023 Zhang et al. 2024 Bai et al. 2025). Network
'Representation units include neurons (dimension of the representation) and features (direction in the acti- vation space (Huhen et al. 2024) that he learned by sparse autoencoders (Gao et al. 2025)
2

---

Published as a conference paper at ICLR 2026
Dissection (Bau et al. 2017) quantifies the correlation between hidden units and concepts by com- puting the intersection-over-unioo (loU) between thresholded activation maps and pixel-level anno- tations in the Broden dataset. Describe-and-Dissee
(DnD) (Oikarinen & Weng 2023) follows the same activation-selection paradigm, identifying top activating images for each neuron and then as- signing textual concepts by matching them with vision-language embeddings. Similarly, automated framework V-Interp (Zhang et al. (2024) relies on highly activating samples as prompts to large mul- timodal models, which then produce free-form textual explanations of neurons and features. Despite their advances, these methods all rely on the activation-mage maggnitde assumption, presuming that larger activations correspond a stronger presence of a concept, which does not necessarily hold and fails to capture unit's causal influence on the model's 's output.
Representation-based. The second trend constructs representations for both model internals and semantic concepts and aligns them to caption (Oikarinen & Weng 2023 Balasubramanian et al. 2024). Balasubramanian et al. (2024) aims at explaining the roles of ViT components (e.g. attention head) by decomposing the contribution vector in the final layer's activation, then learns a set of linear maps for each component to align them in CLIP's embedding space, enabling text description. However, this method is restricted to VIT submodules and does not generalize more fin-grrinnd units such as neurons or features at arbitrary positions in diverse vision models.
CLIP-Dissecs (Oikarinen & Weng 2023) prepares a probing dataset D and concept set C. For each neuron k, it constructs an activation vector Act(k; D) IPP) by recording its responses across all samples in 'D as the neuron representation. For each concept e E C, it builds a similarity profile sim D) using CLIP embeddings of text and images as the concept representation. The most cor- related concept for neuron l is then identified as Equation I where F(-) is a handcrafted similarity.
arg max F (Aclikk sim((, D)). CEC
Despite its strong performance, it relies on heuristically constructed neuron representations, which are still grounded in the activation-mage magnntudd assumption. In contrast, MICLIP learns representa- tions for model internals directly through contrastive training, avoiding heuristic designs and en- abling more principled and flexible representation-coneept alignment across diverse vision models.
Limitations of input-centric interpretability. A majority of existing works on explaining model internals with concepts remains input-centric: they infer unit's meaning from correlations between its high activations and input-side patterns or captions alone (Bau et al. 2017: Hernandez et al.) 2 Zhang et al. 2024 Bai et al. 2025 Oikarinen & Weng 2023). This paradigm implicitly assumes the activation- magattioo--mannuudeeeee assumption yet provides no guarantee that the unit causally steers the model's predictions in that concept's 's direction. Recent work formalizes these concerns from an output-centric perspective, emphasizing that choosing the right causal mediator is central to faithful explanations (Mueller al.) (2024), and demonstrates in language models that descriptions grounded to both input and output yield more behaviorally faithful characterizations than input correlations alone (Gur-Arieh et al. 2025). Parallel evidence vision (Gandelsman
al., 2025 proposes CLIP-specific, output-centric method that interprets neurons by tracing their second-orde effects on the model's output embeddings, thereby revealing each unit's causal semantic influence rather than just its activation correlations. Inspired by this discussion, MICLIP serves as universal framework that aligns internal representations with uuman-understandable concepts retrieved from both input image semantics and the model's output decision.
3 METHODOLOGY
In this section, we introduce our proposed method MICLIP in detail, including how it adopts the contrastive learning paradigm to create a learned representation for model internals (Section (.1.. how it utilizes the representation to connect model internals with human-understandabbl semantics, enabling precise and interpretable descriptions of model internals, as well as accurate identification of concept-relevant components in the model (Section B.2). In Section 3.3 will discuss how MICLIP supports fine-grained model steering through unit-level interventions based on the identified representation units. The overall framework is demonstrated in Figure
3

---

Published as a conference paper at ICLR 2026
CLIP Embedding Space
oulput
Sparse Autoencoder
Eg
Encoder
Decoder
Ep
-+
E
Feed Forward
Great Pyrenoos
Label
Image
(b) Training
Attention
(e) Feature disentanglement
Localization
concept
sim((,,)
feature
GLIP -..
Res
Suppreee
Description
(c) Inference
(a) Target models
(d) Model steering
Training
Inference
Neuron Embedding
$ Concept Embedding
O Masked Feature
Selected Feature
Figure 2: Framework of MICLIP on (a) the target models (frozen), (b) contrastive learning of the
shared embedding space (Section B.T), J (c) mechanistic feature localization and description (Sec-
tion b.2). (d) model steering (Section 3.3). (e) feature disentanglement with k-SAE (Optional).
3.1
MECHANISM-CONCEH ALIGNMENT VIA CONTRASTIVE LEARNING
The CLIP framework (Radford et al. 2021 is pretrained to associate visual and textual modali- ties within a unified embedding space. Inspired by this, we extend CLIP's contrastive formulation to directly learn mappings from the activation space into CLIP's embedding space. This learned aligument avoids heuristic design and enables an interpretation of representation units in terms of human-undersstandabbe concepts, grounded on both inputs and outputs.
Given a labeled image dataset D = { 4) +)) with input images x; E X, we forward them through the target vision model to retrieve activations a I from the residual stream of the target layer and the corresponding predicted labels è E C. This yields two paired sets: acti- vations A {a, J.V and predicted concepts J.N Then, the mechanism-conceepl alignment process within MICLIP is formulated as CLIP-based contrastive loss, specifically the symmetric InfoNCE (He et al.] (2020) loss, including alignment between neurons (or features) and concepts, as well as between neurons and input images. More details are shown in Appendix B.I
Calignment
nott (En (A: 0,), E. ((2)) + cin COLIP (A: 0a). Ex)))
(2)
neurea-coneept.om
neuron-image loss
Here, the projection functions are implemented as CaiP-based encoders, which include trainable neuron encoder (- Om) parameterized by 0, and frozen encoders from a previously trained CLIP model: concept encoder (-) and image encoder Eç(-). Specifically, F,, maps the original neuron representation a E R" to a neuron embedding n = En (a) Rd Eg maps the concept e C concept embedding
(c) Rdd, and E, maps the image E to an image embedding (z) E Rd. Here, the neuron encoder (-;00, connects the internal mechanism of the target model the comprehensive semantic space CLIP, which enables mechanistic feature localization
and description, as discussed in Section 3.2
Discussion. Compared to prior works (Kalibhat al., 2023) Hernandez et al. 2021 Oikarinen & Weng 2023 Balasubramanian et al.] (024)) discussed in Section 2 MICL.P introduces several key advantages through its contrastive learning paradigm. For instance, CLIP-Dissect constructs neu- ron-concept alignments heuristically through probing dataset correlations, whereas approach replaces such heuristics with a learning-baat mapping into the semantic space. On the other hand, Balasubramanian al. (2024) emphasizes module-level decomposition of ViTs rather than fine-grained internal units. By contrast, MICLIP provides general, principled, and learning- based alignment framework for diverse internal representations.
3.2
MECHANISM LOCALIZATION AND DESCRIPTION
Once trained, MICLIP enables both concept-to-nn cancept-to-mechanee mechanism localization and mechanism-to-ccome to-ccnneettt description. This is made possible by encoding both the target model's representation and concept
4

---

Published as a conference paper at ICLR 2026
spaces into shared embedding space, where symmetric identification becomes feasible. This sec- tion describes how operations within MICLIP'S unified embedding space support these two tasks.
Our core idea is characterize the relationship between representation unit of the target model and a human-ume saman-understandable concept by comparing the relevance score sim(..) between their embed- dings in the unified semantic space of MICLIP. Here, the representation unit can be either neurons or features learned from SAE, making the framework more generalizable.
To obtain the embedding for representation unit, adopt the encoder E.,() to project it into MICLIP'S embedding space. If the representation unit u is a specific neuron with activation value a, E R. it directly encoded as u = En (a. ee))) if represents dimension i, where e(x) E IR" is the standard basis vector with 1 at position i. If is SAE feature f; R from the encoder dictionary, the embedding is obtained by u u= = (fa). Meanwhile, the embedding of concept e is c = E.(c))- The relevance score sim(u, between mechanism and concept e then defined the cosine similarity sim(u,.) e)
u.c of their embeddings u and c. More information about SAE is mentioned in Appendix C.1 Notably, the linear design of E,, in Section 4.1 ensures consistent
Iuu [|c]
localization of individual neurons with the training process on full activation vectors. A detailed theoretical derivation of this consistency is provided in Appendix B.3
With this similarity measure, we can find the most relevant representation units or concepts given their counterparts.
Concept-to- MMMechanii Localization. Given a human-interpretable semantic concept e E C, this task aims to identify the mechanisms from set of all representation units U (e.g.. g. all neurons or all interpretable features) that closely align with c. We identify the top representation units related to concept and record their indices according their similarity score in the set Lg:
L SelectTop-r ((sim(ws u,E))
where SelectTop-T selects the indices of the top T elements. This process identifies the most influ- ential mechanisms within the model that are responsible for a given concept.
Mechanism-to-Coneee Description. Given specific representation unit this task aims to find the concepts from set that best describe the mechanism. We identify the top t concepts related to representation unit and record the concepts according their relevance score in the set D,;
D L SelectTop-T ((sim(u, sy)te,ec).
3.3 MODEL CONTROL WITH MECHANISM INTERVENTION
To steer the target model, we intervene on the representation units identified in Section 3.2 For given concept we collect its corresponding units indexed by L. (neurons or SAE features), and adjust their activations suppress or amplify the concept's 's influence on the model.
First, we apply an intervention to the target units. Then, these modified mechanisms are decoded into the original neuron space for subsequent operations. For each representation units u (u = 0g e( = fi) indexed within Lg, either apply a sealar multiplication or add an additive bias
ú, = Bu, (Scaling) or u, = u, + 3 (Adding), Vi E Le, B E R.
(5)
By applying distinct parameter 3, we can suppress amplify the target feature to adjust the model.
In Section we present intervention experiments that empirically demonstrate how targeted manipulations enable fine-grained model control, thereby validating the precision of concept-to- mechanism localization in MICLIP.
EXPERIMENT
4.1
EXPERIMENTAL SETTING
Target models for interpretation. We evaluate different methods on (i) image classification models
trained on ImageNet- 1k dataset (Deng et al. 2009) including, ResNet-50 (He et al. 2016) and ViT- B-16 (Dosovitskiy et al. 202I) (ii) pretrained multimodal models, specifically CLIP/VIT-B-16, for which we use its zero-shot classification ability test the intervention in Section 4.3
5

---

Published as a conference paper at ICLR 2026
Baselines. We compare MICLIP against several baselines that are categorized into neuron-based and feature-based methods. Different baselines are selected for each experiment based on their applicability to the specific setting. We include Network Dissection (Bau et al. 2017). CLIP- Dissect (Oikarinen & Weng 2023). V-Interp (Zhang et al. 2024) that relys on activation-magnitube assumption identity concepts. Additionally, construct method named Act- vel-Va, which identifies concepts according to the neuron activation values. More details are provided in Ap- pendix D.1 For each of our baselines, we use 100,000 images randomly sampled from the ImageN et- k (Russakovsky et al. 2015) training set for computation or training.
Implementation details. We train MICLIP on subset of 100,000 images sampled from the ImageNet-Ik training set. We use the frozen pretrained text encoder E, and vision encoder E, in CLIP/VVT-B-11. and we train a neuron encoder E,, with a linear projection layer to map neuron ac- tivations to the embedding space. We use k-SAE (Gao et al. 2025) for feature disentangling, which trained on the residual stream of model. We validate our intervention approach on ImageNet- 1k and further evaluate its generalization capabilities on an unseen dataset DTD (Cimpoi et al., 2014), which is a texture-based image classification dataset consisting of 56 texture categories.
4.2 EVALUATION ON MECHANISTIC INTERPRETATION AND LOCALIZATION
Quantitative Results on Mechanism Description. We adopt the CLIP-Dissect eval- uation framework, which benchmarks description accuracy the final classification layer. In this setting, each neuron's ground-truth function is simply its corresponding class name (e.g., "sea lion"), allowing for direct and objective evaluation. This setup enables an objective and scalable evaluation based on the neuron's 's actual function, avoiding the limitations and subjectivity of human assess- ments based on a few top- activatii images.
Analysis 1:
We evaluate performance with three metrics. For the open- vocaauuayyy experiments. we follow CLIP- Dissect and use the same open-ended concept sets of varying sizes (Common-3k, Common- 10k, and Common-20k), which consist of the 3,000, 10,000, and 20,000 most common English words, respec- tively. For closed-set concept sets (1,000 ImageNet-IK Ik classes), we additionally report Accuracy (Acc.), the percentage of neurons whose top-ranked description exactly matches the ground-truth class. For general tasks, we use CLIP Score and Mpnet Score, which measure the cosine simi- larity between the generated description and the ground-truth class name using CLIP/ViT-B B-VVT-B--LL 16 and Mpnet-base base-v2 encoders, respectively. Finally, to ensure the reliability our improvements, we perform one-tailed paired t-test across three random seeds verify the statistical significance of our model against the baselines.
Table I: Evaluation on neuron description. We compare with baseline methods on interpreted con- cepts for the neurons the last layer in different vision models. t: higher is better. (* indicates statistical significance, e.g., p-value < 0.05).
ResNet-50
ViT-B/16
Dataset
Method
Acc.t
CLIPt
Mpnett
Acc.t
CLIP
Mpnetf
CLIP-dissect
0.7456
0.4161
0.7182
0.2718
Common-3k
MICLIP (Ours)
0.7624
0.4334
0.7618
0.4310
CLIP-dissect
0.7656
0.4696
0.7342
0.3637
Common- 10k
MICLIP (Ours)
0.7885
0.5029
0.7786
0.4748*
CLIP-dissed
0.7900
0.5257
0.7563
0.4376
Common-20k
MICLIP (Ours)
0.8145*
0.5812
0.8138*
0.5783
Act- Values
0.9940
0.9995
0.9983
0.9940
0,9989
0.9975
ImageNet- 1k
0.9560
0,9902
0.9746
0.9500
0.9881
0.9631
MICLIP (Ours)
1.0000
1.0000
1.0000
1.0000
1.0000
1.0000
Finding 1: MICLIP gives more precise interpretations given specific representation units. In
Table our MICLIP can achieve the highest Accuracy, CLIP score and Mpnet Score among base- lines, illustrating precise localization of identified features. Furthermore, MICLIP outperforms CLIP-Dissec even when evaluated on concept set rather than ImageNet- 1k,, which MICLIP is trained This also suggests that MICL.PP generalizes broader scope of concepts.

---

Published as a conference paper at ICLR 2026
Table 2: Accuracy deviations of enhancement and removal interventions on neurons and features. Best performing methods are highlighted in bold. Values that contradict the expected outcome (e.g., enhancement leading to a decrease in accuracy) are marked in red.
Intervention on neurons
Eahancement Aoe (%) (1)
Removal Acc (%) ())
Method
5osNet-50
ViT-B/16
CLIP
ResNet-50
VIT-B/16
CLIP
Act-Values
2.27 (#: 0.03)
-0.19 0.01)
-8.05 (2: (..00))
-8.98 (# 0.08)
-1.43 (2 0.01)
-23,30 (#: 0.00)
Network Dissection
0.78 (# 0.02)
0.35 (#: 0.01)
-2.95 0.15)
-0.37 (# 0.03)
-0.88 (:: 0.05)
CLIP-dissect
3.05 0.18)
(219 0.03)
-0.04 0.03)
-12.31 (= 0.67)
-0.04 0.00)
-1.16 (1: 0.14)
V-Interp
1.71 (+ 0.22)
.0.04 0.02)
-0.29 (+ 0.10)
-8.04 (+ 0.71)
-0.04 0.00)
-0.14 0.05)
MICLIP (Ours)
5.32 (+: 0.03)
0.18 0.01)
1.10 0.05)
-17.24 (= 0.05)
-0.04 (2: 0.02)
-1.50 (4: 0.08)
(b) Intervention on SAE features
Enhancement Aox (1)
Removal AAcc (L)
Method
ResNet-50
ViT-B/16
CLIP
ResNet-50
VIT-B/16
CLIP
Act-Values
4.34 0.00)
3.68 (= 0.02)
0.43 (# 0.08)
-11.98 (#: 0.00)
-22.77 0.11)
-15.94 0.06)
Network Dissection
0.02 (+ 0.04)
(- 0.05)
0.50 (+ 0.06)
-0.08 (+ 0.05)
-4.03 (+ 0.13)
-1.99 (+-05)
CLIP-dissect
2.27 (+ 0.09)
5.04 0.05)
4.85 (4: 0.03)
-7.30 (+: 0.03)
-27.78 (+:: (::00.)
-11.05 0.12)
V-Interp
0.91 (# 0.02)
(-90 0.01)
1.33 (+ 0.00)
-2888 (# 0.09)
-7.55 (= 0.06)
-2.83 (+ 0.00)
MICLIP (Ours)
3.89 (+ 0.03)
5.57 0.02)
588 (+ 0.03)
-10.99 (+ 0.02)
-32.04 0.200)
-17.70 0.02)
4.3
INTERVENTION FOR MODEL STHERING
Analysis 2: Verifying Mechanism Localization via Intervention on Discriminative Models. Ef- fective mechanistic interpretability should enable meaningful interventions, either enhancing or sup- pressing the influence of concept-related mechanisms, thereby improving degrading classification performance on the target concept. We verify our localization by intervening on the top-5 neurons features (i.e., т 5 in Equation 3 for each ImageNet- Ik concept. We measure the change in classification accuracy A Acc after applying either enhancement (x2 scaling) and removal ( (x0 scal- ing) after interventions on the activations of the following layers: the 10th-layer for VÝT-B/16 and CLIP/ViT-B-16 16, and the stages...ayeers.sshoo I.shrreuu layer for ResNet-50.
Our method, MICLIP, enables precise local- ization of the mechanisms that govern model classifica- tion. For reference, the original classification accuracies of the models are 80. 14% for ResNet-50, 10.32% for ViT- B/16, and 61.12% for CLIP. As detailed Table [ our MICLIP consistently enables predictable and stable de- viation in classification accuracy. In contrast, baselines like Aat-Va exhibit an inconsistent response. Although they may show strong performance degradation upon removal, the same set of neurons often fails to enhance the model'ss 's performance. The ability of MICLIP to both enhance and suppress model performance using the same of local- ized representation units provides strong evidence that we have successfully identified the true, functionally relevant neurons and features.
Finding 2:
Table 3: Accuracy deviations from enhancement and removal interven- tions on neurons and features for un- seen concepts. The best-performing methods are highlighted in bold.
(a) Intervention on neurons
Enfuncement
Removal
Method
Ace (1)
Ace (9) (1)
CLiP-dissect
0.00 (+ 0.15)
-0.65 (4 0.28)
V-Interp MICLIP (Ours)
-0.39 (= 0.24)
0.06 (+: 0.37)
0.38 (+ 0.11)
4.91 (+ 0.20)
(b) Intervention on SAE features
Enhancement
Removal
Method
Analysis 3: Verifying the Generalization of Mechanism Localization to Unseen Concepts via Intervention. To evaluate generalization to unseen concepts, we repeat the intervention experiment in Analysis 2 on the DTD texture dataset, measuring the impact on CLIP's zero-shot classification accuracy. The original zero-shot classification accuracy of the CLIP model on this dataset is 44.80%.
Acc (T)
A (%)
CLIP-dissect
1.84 (1: 0,15)
-5.04 (2: 0.08)
V V-Interp
0.03 (1: 0.01)
.0.04 (1. 0.04)
MICLIP (Ours)
2.00 (+: 0.10)
-4,98 (#: 0.27)
The mechanisms identified by MICLIP are semantically grounded and general- izable to unseen concepts. As shown in Table 3 our MICLIP enables effective interventions on the CLIP/ViT-B-I model, consistently enhancing or suppressing its zero-shot classification accu- racy for unseen concepts. This is demonstrated by the predictable and stable changes observed in the model's performance metrics when we apply interventions to the localized neurons or features. Although MICLIP was trained solely on the ImageNet-Ik l dataset, its effectiveness in zero-shot
Finding 3:
7

---

Published as a conference paper at ICLR 2026
setting highlights its strong generalization capabilities. These results confirm that approach suc- cessfully localizes the key representation units governing model behavior, even for concepts not present in its original training data.
4.4 SEMANTIC ANALYSIS ON MICLIP
Analysis 4: Unit Semantic Geometry in the Aligned Embedding Space. this part, we analyze the semantic geometry of representation units localized by MICLIP. This analysis aims determine whether the localized SAE features exhibit semantic coherence as concepts. Similarly, we select concepts in ImageNet-Ike Ik, and then into four categorics ( ("mammal", "non-men mammall, "tool" and "vehicle"). Following |ubenn et al. (024), extract localized features from Wae. dec. iee., the learned dictionary. Then, visualize 21) t-SNE (van der Maaten & Hinton 2008) of the embeddings of these features in Figure 3 We provide more details in Appendix D.2
WordNet (Miller 1995) to categorize these concepts
Finding 4: MICLIP learns a semantically coher- ent feature space. Figure[Jshows that SAE features associated with related concepts form coherent clus- ters in MICLIP'S aligned embedding space.
Scatters of Features in Aligned Space
15
mammal
10
non mammal
t-SNE Dim-2
Analysis 5: Visualization Verification of Local- ized Features via Attention Maps. In this analysis, we investigate the spatial grounding of the learned features by examining their activations within the model's attention maps. We adopt the visualization method from DINO (Caron al., 202T) explore whether the SAE features have localized corre- spond to a specific visual concept. Specifically, we leverage the self-attention map from 8th layer of the CLIP/ViT-B- model, between the CLS token and all other image patch tokens. By preserving the out- put of specific localized features of image tokens, we generate a saliency map by computing the attention weights between the CLS token and image tokens. This map precisely highlights the gions of the input image that the features are attending to, providing qualitative verification of the feature's intended visual semantics. The attention map shown Figure 4 visualizes the spatial grounding of top-5 localized features for the first class "kit fox" in the ImageNet- 1k metadata class order. The visualizations are specifically for the first six images of "kit fox" from the ImageNet- 1k validation set. Additional results for other classes and images can be found in the Appendix E.
-10
20
40
L-SNE Dim-1
Figure 3: Features belonging the same mantic category demonstrate compact clus- tering in the 2D embedding space.
Class: kit fox
Origina
CLIP Dissect
Ours (MiCrr)
Orignall
CLIP Dissect
Ours (MiCCi)
Original
CLIP Dissect
Ours (MrCuar)
Normalized Attention
Original
(MICUP)
Original
(MrCLIP)
Orgina
Our (McCuz)
Figure 4: Visualizing the spatial grounding of top-5 localized SAE features of class "kit fox".. The attention map highlights the precise location of "kit fox" ImageNet-Il Ik (Russakovsky al. 2015). confirming the feature's 's effective localization.
Finding 5: MICLIP effectively localizes features that semantically correspond to specific visual concepts. The attention map in Figure4hhigh highhights that our identified feature consistently activates around the of the "kit fox* key visual identifier for this class.
8

---

Published as a conference paper at ICLR 2026
Table 4: Ablation Study: Accuracy deviations of enhancement and removal interventions on seen and unseen concepts. Best performing methods are in bold, second best underlined, contradictory values (e.g., enhancement leading to an accuracy decrease) in red.
(a) Interventions on ImageNet-Ik
(b) Interventions on DTD
Mothed
ResNet 50
Buluneemntt (M
B/1
CLIP
ResNet
VT B/16
A.eer
CLIP
Enhunvement A/Aox (58) (7)
(1)
Newrons
Input-Only
3.1 61 0.00)
62 0.07)
0.31 (: 1,00
-1263 (1: 0.1
0.18
-.09 (1 2.50)
Inpat-Cuily Oulput-Dily
0.69 (=: 0.36)
-5.18 (+: 1.621
Ouipn-Dely
5.7
0.44
-17.8
0.00T
-1.17
-2 60 c 2:50)
-0.53 (1: 0.30)
MICLIP
(2 000)
0. 0,01)
1.10 0.05)
17.24
0.04 0.92)
155001
MICLIP
0.38 (a 0.11)
00.91 (4: 0.20)
Fostwres
SAE Feanures
Iapus Ouly
0.08)
0.50)
5.49
3.11 0.05)
-2.29 5,97)
-15.25
0.14)
Iapur-Duly Dutpur-Dely
2,84 0.16)
4685 (+ 0.161
Outpur-Daly
4,8 (- 0.01)
0.83)
-11.77 (- 0.01)
-32.79 0.04)
-16.59
0.08)
0.01) 2 0.10)
-2.96
MICLIP
3.89 L1 0.00)
5.57 (1 0.02)
5.88 (L. 0.03)
10.99 (1 0.02)
32.04 (1) 0.20)
.17.70 (1L 0.02)
MiCLaP
-.98 (+ 0.27)
4.5 ABLATION STUDY
Analysis 6: Ablation Study on Input- and Output-Grounded Alignment. To dissect the contri- butions of our two alignment components, we perform ablation study with three variants: MICLIP (Inpu--Onl),, MICLIP cotput-Only)) and the full MICLIP. We replicate the intervention experi- ments on seen concepts from ImageNet- Ik (Analysis 2) and unseen texture concepts from DTD (Analysis 3), evaluating effectiveness by the accuracy change A Ace after enhancement (x2) and removal (x0) of the top-5 localized mechanisms.
Finding 6: The combination of input- and output-grounded aligmment is crucial for robust and generalizable mechanistic interpretability. The results, presented in Table[4] reveal a clear synergy between the two alignment strategies. The Output-Only performs strongly on seen ImageNet- Ik concepts but fails to generalize the unseen DTD dataset, as evidenced by its poor performance and contradictory results (e.g., enhancement causing an accuracy drop). Conversely, the Input- Only shows better generalization to DTD but is less effective on ImageNet-1k, The full MICLIP achieves strong performance on seen concepts while maintaining robust generalization to novel ones, confirming that integrating both alignment strategies is essential
4.6 UNDERSTANDING FLAWED VISUAL REASONING IN MODEL PREDICTIONS
We leverage MICLIP to diagnose failures in visual reasoning in the CLIP/ViT-B-16 model by trac- ing the semantic trajectory of an image's internal representations across layers. This achieved by projecting layer-specific activations into a seman- tically aligned embedding space via the neuron en- coder F.,. We then compute cosine similarities to text embeddings of the ground-truth (GT) and misclassified labels in the incorrect case using the CLIP text encoder Eg, For the incorrect case, the predicted label is the misclassification which is also used for comparison in the correct case. As illustrated in Figure 5 presenting an example with a GT of "sea anemone" and a misclassifica- tion of "feather boa". Correct predictions maintain higher GT similarity across all residual layers. Erroncous predictions show a clear point where predicted-label similarity surpasses GT, pointing out where the model's 's view shifts. With this diag- nose method, MICLIP is able trace the semantic trajectory of the reasoning process, and localize the layer where the representation shifts from the correct concept to the incorrect one.
Similarity between Layer-wise Representations and Concepts: "sea anemone (GT)" and "feather boa (Misclassified)
Original Image
Correct
Similertt
Rido
Layar
Incorrect
Similariy
623
Mis
620
Layer
Figure 5: The plot shows cosine similarities between layer-wise representation embeddings and text embeddings of the ground-truth (GT; blue) and misclassified label (Mis.; orange). ① marks the layer where GT dominates most, while @ marks where Mis. overtakes GT, re- vealing the failure point.
4.7 VISUALIZING
FEATURE PRIMITIVES
Analysis 7: Visualization Verification of Low-Level Features. In this analysis, extend

---

Published as a conference paper at ICLR 2026
investigation to ascertain whether our learned features also correspond to low-level visual primitives, such as colors, shapes and textures. To explore the spatial grounding of these elementary features, we employ the same attention map visualization technique detailed in Analysis 5, specifically utilizzig the attention maps from relatively low-level layers (i.e., the 3rd) layer of the CLIP/VVTTB-16 model.
While the ImageNet-1k dataset lacks explicit labels for such primitives, we identified representative samples by selecting images that exhibited high activations for certain localized features discov- ered by MICLIP. For instance, we found features that systematically activate images containing prominent green regions, and indeed, their rop-activating images from the dataset consistently share this specific visual characteristic. The attention maps shown in Figure visualize the spatial ground- ing for this feature, confirming its specialization for the color "green". ". This provides a qualitative verification that our method can effectively identify and localize not only high-level semantic con- cepts but also fundamental visual building blocks. Additional results for other low-level features, such as shape-detecting feature for grids, can be found in Appendix
Concept: green
Original
CLIP -iseett
Originll
CLIP-
Qural (UCCUP)
Original
CLiP-Diswet
Oun(MMCUu))
Normalia
Original
CLIP-Dioect
Ourx(MiChur
Origial
CLIP-DiAnct
Ouns MMICUP)
Origial
CLiP-Dieect
Oune MiCue)
Figure 6: Visualizing the spatial grounding of a localized SAE feature for the color "green". The attention maps highlight that this feature consistently activates on green regions across different images from ImageNet- 1k, confirming its effective localization of this low-level visual property.
Finding 7: MICLIP effectively localizes low-level features that correspond to fundamental visual primitives. As demonstrated in Figure the attention maps for the identified "green" feature consistently and precisely highlight green- sreenccllored regions, regardless of the object's semantics. This confirms that our method can ground foundational visual concepts, like specific colors, to their corresponding spatial locations within an image.
CONCLUSION
We propose MICLIP, representationn-asse automated mechanistic interpretability framework that bridges the gap between the internal mechanisms of vision models and human-understandable con- cepts, through aligning them in shared semantic embedding space. MICLIP eliminates the reliance on the activation- -agannttn- assumption and jointly leverages semantic signals from both the inputs and outputs of vision models, thereby capturing model's 's reasoning process. By leveraging contrastive learning to align internal features with seman- tic concepts, MICLIP provides fine-grained and precise mechanistic interpretation across diverse vision models. Coupled with external modules like k-SAE, it establishes a universal framework supporting both neuron- and feature-level analysis. Extensive qualitative and quantitative experi- ments confirm its effectiveness in interpreting, localizing, and steering internal features, while also demonstrating generalization to unseen concepts, applicability across diverse architectures, and util- ity in analyzing flawed model behaviors.
comprehensive and faithful view of the
Despite these advancements, we acknowledge certain limitations. First, MICLIP'S interpretability fidelity is inherently bounded by the semantic coverage and potential biases of the pre-trained CLIP embedding space. However, our empirical observations indicate that CLIP's classification biases do not necessarily hamper MICLIP'S intervention capabilities; valid features can still be identified and steered when CLIP performs poorly. Integrating orthogonal advancements to mitigate these biases could further enhance fidelity. Second, current validation is restricted discriminative vision models. Future work could extend this approach to generative models, broadening the applicability and impact of mechanistic interpretability in real-world AI systems.
10

---

Published as a conference paper at ICLR 2026
6 ACKNOWLEDGEMENT
The research received partial support from National Natural Science Foundation of China (Grant No. 62406193). The authors also gratefully acknowledge further assistance provided by Shanghai Frontiers Science Center of Human-centered Artificial Intelligence, MoE Key Lab of Intelligent Perception and Human-Machine Collaboration, and HPC Platform of ShanghaiTech University.
7 ETHICS STATEMENT
This research complies with the ICLR ethical guidelines, upholding transparency, reproducibilityy, and responsible use AI. Our work aims to make vision models more transparent and interpretable, contributing to society by enabling safer and more responsible Al systems. We believe that increas- ing model transparency benefits a broad range of stakeholders and helps mitigate potential misuse. This research does not involve human subjects, personal data, sensitive information, and thus poses no risks to privacy, health, or safety.
We have faithfully reported our findings without fabrication falsification. All datasets, baselines, and related works are properly cited, and our methods are designed to be transparent and repro- ducible. We have discussed the limitations of our work and possible future extensions.
We appreciate prior contributions in mechanistic interpretability and aim for our work to further promote fairness, accountability, and responsible AI deployment.
REPRODUCIBILITY STATEMENT
To ensure the reproducibility of our work, this paper provides comprehensive details regarding our experimental setup, models, and evaluation protocols. Specifically:
Models and Datasets: The target models (ResNet-50, ViT-B-16, and CLIP/ViT-B-16) and datasets (ImageNet-Ik and DTD) used for our experiments are explicitly stated in Section 4.1
Implementation Details: Key details for implementing our method, MICLIP, are provided in Section[4.1 This includes the training data size (100,000 images from ImageNet-Ikk) the use of frozen CLIP encoders, the architecture of the neuron encoder (F,,), and the choice of k-SAE for feature disentangling. Further details on baseline methods are available in Appendix D.I
Experimental Procedures: Each analysis is described with its specific protocol. For in- stance, the quantitative evaluation framework is detailed Section [4.2 (Analysis 1). The intervention strategy, including the specific layers targeted and scaling factors, is described in Section 4.3 (Analysis 2). All other analyses are similarly detailed in their respective sections.
REFERENCES
Nicholas Bai, Rahul Ajay Iyer. Tuomas Oikarinen, Akshay R. Kulkarni. and Tsui-Wei Weng. In- terpreting neurons in deep vision networks with language models. Transactions Machine Learning Research, 2025. ISSN 2835-8856. URL botttss://pperreeee
net forum?
[t=xIdXvvEIa
Sriram Balasubramanian, Samyadeep Basu, and Soheil Feizi. Decomposing and interpreting image representations via text in vits beyond CLIP. The Thirty-eighth Annual Conference Neu- ral Information Processing Systems, 2024. htLhttts://// http:////ppeeeeeeeeeeeeeeeeeeeee http://un/e//imitt..
Vhh 7ONt FvV
David Bau, Bolei Zhou, Aditya Khosla, Aude Oliva, and Antonio Torralba. Network dissection: Quantifying interpretability of deep visual representations. Proceedings the IEEE conference computer vision and pattern recognition, pp. -6549, 2017.
11

---

Published as a conference paper at ICLR 2026
Steven Bills, Nick Cammarata, Dan Mossing, Henk Tillman, Leo Gao, Gabriel Goh, Ilya Sutskever, Jan Leike, Jeff Wu. and William Saunders. Language models can explain neurons in language models. hhtpps: http:////ppnapppp.....o....oo neetron--xplaieeeee
www.www..dww.....w.mww.......... .nt//
nndx.. html, 2023.
Mathilde Caron, Hugo Touvron, Ishan Misra, Hervé Jégou, Julien Mairal, Piotr Bojanowski, and Armand Joulin. Emerging properties in self-supervised vision transformers. In Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), pp. 9650-9660, October 2021.
M. Cimpoi, S. Maji, I. Kokkinos, S. M.hamed,, and A. Vedaldi. Describing textures in the wild. In Proceedings of the IEEE Conf. on Computer Vision and Pattern Recognition (CVPR), 2014.
Jia Deng, Wei Dong, Richard Socher, Li-Jia Li, Kai Li, and Li Fei-Fei. Imagenet: A large-scale hi- erarchical image database. In 2009 IEEE conference on computer vision and pattern recognition, pp. 248-255. Icee, 2009.
Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xiaohua Zhai, Thomas Unterthiner, Mostafa Dehghani, Matthias Minderer, Georg Heigold, Sylvain Gelly, Jakob Uszko- reit, and Neil Houlsby. An image is worth 16x16 words: Transformers for image recogni- tion at scale. In International Conference on Learning Representations, 2021. URL https: //openrevis
www.//ftrooeee d=Yi cbF dNTTY
Javier Ferrando, Oscar Balcells Obeso, Senthooran Rajamanoharan, and Neel Nanda. Do i know this entity? knowledge awareness and hallucinations in language models. In The Thirteenth International Conference Learning Representations, 2025. htttttt:s///peeeeee
net: heet/Eorrum:iddNNKPP11131
Yossi Gandelsman, Alexei A. Efros, and Jacob Steinhardt. Interpreting the second-order effects of neurons in clip, 2025. URL
Leo Gao, Tom Dupre la Tour, Henk Tillman, Gabriel Goh, Rajan Troll, Alec Radford, Ilya Sutskever, Jan Leike, and Jeffrey Wu. Scaling and evaluating sparse autoencoders. In The Thirleenth Inter- national Conference on Learning Representations, 2025.
.nt//
Forum?id-tcs24 9ZNKD
Yoav Gur-Arieh, Roy Mayan, Chen Agassy, Atticus Geiger, and Mor Geva. Enhancing automated interpretability with cutput-centric feature descriptions. In Wanxiang Che, Joyce Nabende, Eka- terina Shutova, and Mohammad Taher Pilehvar (eds.), Proceedings of the 63rd Annual Meet- ing of the Association for Computational Linguistics (Volume 1: Long Papers), pp- 5757-5778, Vienna, Austria, July 2025. Association for Computational Linguistics. ISBN 979-8-89176- 251-0. doi: 10.18653/v1/200 aen55://onn....lloooon........ acl- long. 2887
URL htttp:://aclanttiitt
Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recog- nition. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp- 770-778, 2016.
Kaiming He, Haoqi Fan, Yuxin Wu, Saining Xie, and Ross Girshick. Momentum contrast for unsupervised visual representation learning. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), June 2020.
Evan Hermandez, Sarah Schwettmann, David Bau, Teona Bagashvili, Antonio Torralba, and Jacob Andreas. Natural language descriptions of deep visual features. In International Conference on Learning Representations, 2021.
Robert Huben, Hoagy Cunningham, Logan Riggs Smith, Aidan Ewart, and Lee Sharkey. Sparse autoencoders find highly interpretable features in language models. In The Twelfth International Conference on Learning Represenationss 2024. peatttss://ppeneeveee
net forum?
id Edw
Neha Kalibhat, Shweta Bhardwaj, Bayan Bruss. Hamed Firooz, Maziar Sanjabi, and Soheil Feizi. Identifying interpretable subspaces in image representations. In Proceedings of the 40th Interna- fional Conference on Machine Learning, ICML'23. JMLR.org, 2023.
12

---

Published as a conference paper at ICLR 2026
Hang Li. Chengzhi Shen, Philip Tor, Volker Tresp, and Jindong Gu. Self-discovering inter- pretable diffusion latent directions for responsible text-to-image generation. In Proceedings of the IEEE/CVF Conference Computer Vision and Pattern Recognition, pp. 12006-12016, 2024.
Alireza Makhzani and Brendan Frey. K-sparse autoencoders. International Conference on Learn- ing Representations, 2014.
George A. Miller. Wordnet: lexical database for english. Communications of the ACM, 38(11); 39-41, 1995.
Jesse Mu and Jacob Andreas. Compositional explanations of neurons. In Advances Neu- ral Information Processing Systems, volume 33, pp. 17153-17163. Curran Associates, Inc., wwwess/pppee///22///
2020. URL
htw.eeedddinnnnnnininnee neuri cc http:///ea..aaoeeeo/oo/ee/eeeeee
Fi File/c7495ff 188.:/55555.33334446666777777773
Aaron Mueller, Jannik Brinkmann, Millicent Li, Samuel Marks, Koyena Pal, Nikhil Prakash, Can Rager, Aruna Sankaranarayanan, Arnb Sen Sharma, Jiuding Sun, Eric Todd, David Bau, and Yonatan Belinkov. The quest for the right mediator: A history, survey, and theoretical grounding of causal interpretability. 2024. URI wtt.thttt////xxii wttp:/iwwwrr/////////// 01..0.801111 01416
Tuomas Oikarinen and Tsui-Wet Weng. CLIP-dissec:: Automatic description of neuron representa- tions in deep vision networks. In The Eleventh International Conference on Learning Represen- tations, 2023. URL htt.tttts////peeeee///e// net/ wtt::/orrmmmmiiiiiiiiie
Chris Olah, Alexander Mordvintsev, and Ludwig Schubert. Feature visualization. Distill, 2017. doi: 10.23915/disttll.0000 https://distill.pub/22 httpp////////aaurrevisis//////// 7/feaauree
Gonçalo Paulo, Alex Mallen, Caden Juang, and Nora Belrose. Automatically interpreting millions of features in large language models. arXiv preprint urXiv:2410.13923 2024.
Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et al. Learning transferable visual models from natural language supervision. In International conference on machine learning, pp- 8748-8763. PMLR, 2021.
Tianhe Ren, Shilong Liu, Ailing Zeng, Jing Lin, Kunchang Li, He Cao, Jiayu Chen, Xinyu Huang, Yukang Chen, Feng Yan, Zhaoyang Zeng. Hao Zhang, Feng Li, Jie Yang. Hongyang Li, Qing Jiang, and Lei Zhang. Grounded sam: Assembling open-world models for diverse visual tasks, 2024. URL /tttsttps: /tctts://arxiv. org wabs/2200 14159
Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Bjöm Ommer. High- resolution image synthesis with latent diffusion models. In Proceedings of the IEEE/CVF confer- ence on computer vision and pattern recognition, pp. 10684-10695, 2022.
Olga Russakovsky, Jia Deng, Hao Su, Jonathan Krause, Sanjeev Satheesh, Sean Ma, Zhiheng Huang, Andrej Karpathy, Aditya Khosla, Michael Bernstein, Alexander C. Berg, and Li Fei-Fei. Imagenet large scale visual recognition challenge. International Journal of Computer Vision 115 13)2211-252, 2015. doi: 10. (0.10077/11223--15--0888 611263-015-0816-3
URL httpt::tt//cct http://c...or/1111001111
Ramprasaath R Selvaraju, Michael Cogswell, Abhishek Das, Ramakrishna Vedantam, Devi Parikh, and Dhruv Batra. Grad-cam: Visual explanations from deep networks via gradient-based local- ization. In Proceedings of the IEEE international conference on computer vision, pp. 618-626, 2017.
Yingdong Shi, Changming Li, Yifan Wang, Yongxiang Zhao, Anqi Pang, Sibei Yang. Jingyi Yu, and Kan Ren. Dissecting and mitigating diffusion bias via mechanistic interpretability.. In I Proceedings of the Computer Vision and Pattern Recognition Conference, pp. 8192-8202, 2025.
Laurens der Maaten and Geoffrey Hinton. Visualizing data using t-sne. Journal of Ma- chine Learning Research, 986):2577 2008. URL http::/// /// jmir. prt/papers/v9/
vaade w html
13

---

Published as a conference paper at ICLR 2026
Yifan Wang, Yifei Liu, Yingdong Shi, Changming Li, Anqi Pang, Sibei Yang. Jingyi Yu, and Kan Ren. Discovering influential neuron path in vision transformers. In The Thirteenth International Conference Learning Representations, 2025. Jdd=W00yJbr:5.h
Matthew D Zeiler and Rob Fergus. Visualizing and understanding convolutional networks. In Computer Vision-ECCV 2014: 13th European Conference, Zurich, Switzerland. September 6-12, 2014, Proceedings, Part I 13, pp. 318-833, Springer, 2014.
Kaichen Zhang, Yifei Shen, Bo Li, and Ziwei Liu. Large multi-modal models can interpret features in large multi-modal models. arXiv preprint 1rX1::2411.1498,, 2024.
14

---

Published as a conference paper at ICLR 2026
A USAGE OF LARGE LANGUAGE MODELS (LLMS)
In this work, LLMs are used in some automated interpretability baselines (Zhang et al. 2024 to generate summarize textual explanations. Our proposed method does not rely on LLMs, and their use restricted to fair comparison with prior approaches.
During paper writing, LLMs are only used to polish the language and of words. We promise they are not used to generate ideas, methodology design, and experimental analysis.
B MICLIP ALGORITHMS
In this section, elaborate the methodology of our MICLIP, introduced in Section 3.1 and Section 3.2 For clarity, we present two algorithms that outline the key steps the approach in detail.
B.1 CONTRASTIVE LEARNING FOR ALIGNMENT
We introduce the detailed steps of our contrastive learning (Section 3.1 in Algorithm train the neuron encoder E,, (+; (,)).
We only
Algorithm Mechanism--Cnneep Alignment using CLIP-based Encoders
Input: Input image set X' = {>x J.V. activation set A = {a.) i= predicted output set (2) Pretrained CLIP visual encoder Ej and text encoder (-).
Batch size B, step size 7).
Output: Trained parameter 0n of neuron encoder Ea(;; 6n).
1: 0 < (nido
D. IIitiiaizaaion.
2: for /= 1,... .T do
Batch training, terminate until On converges.
3:
{(z,, a,, [.) + SampleBatch(D,
4:
for i == 1=,....,1 ... do
> Compute embeddings. SSectinn 3.1
5:
n, < F,, (a;;
6:
C; < (Gi)
7:
X; +- Ej(x;)
8:
for (i.j) E (I.... BJ² do
Compute scores.
9:
Si + niies
||n:I1-|e;
S In: ||;1|
11:
CLP ΣB log
1
2j=1 log
5
Neuron-concept loss
n
Σ d c3
2B EB log
Σ log
B
Neuron-image loss
E
2B
Σ
Calignment
Lout + L
Eqqation
14:
0 0- 1 + 1) Vo, Calignment
b Update, GD as an example.
0n x 0
n
16: Return: On.
B.2 LOCALIZATION AND DESCRIPTION OF REPRESENTATION UNITS
We also describe the detailed steps of our feature localization and descriptions (Section 3.2 in Algorithm [ For interpretation, we can get L, (top feature indices for concept c) or D.., (top concepts for unit uz).
15

---

Published as a conference paper at ICLR 2026
Algorithm 2 Representation-Baaed Mechanism-Concepp Identification
Input:
Feature vector f IR. for input x (from k-SAE, optional), Concept set C.
Dictionary Wec Raxm bias Dpre R Pre-trained encoders En, Eg.
dyyerr-arameee
One-hot vectors el [0,
OJT E 1R
i= 1, ,
i-th position
(Optional) Specific concept e E C (for localization) or feature index (for description).
(Optional) Feature vector f 12 for imput x (from k-SAE). Output: Le (top unit indices for concept c) or D,; (top T concepts for unit u.).
1: function CONCEPT-TO-MECHANT
LOCALIZATION(C)
D Given concept c.
2:
c (c(
3:
for i = 1 to do
4:
if + None then
Localize SAE feature
5:
a t Woco(f @ e + bpc
Equation TO b Localize neuron
6:
else
7:
a, ((0)
8:
n; + En a.))
9:
sim U4 c)
Compute relevance score.
c
10: L f- Select (sim u4 k=1
Eqqation
11:
return L
function MECHA EECHANIIM-TO-CONEEE DESCRIPTION())
Given feature index i.
13:
f A None then
b Describe SAE feature
14:
a, Weec(f el)) + bpr
Equation TO
15:
else
Describe neuron
a, e(1
17:
n, + (ā:)
for Cj E do
19:
Cj + Eolg) sim(u4, G)
20:
a.
Compute relevance score. b Equation
[n. |e.
21:
Dr, t SclectTop-T (sim(u; (4 9j lgEEc
22:
return D
B.3 THEORETICAL CONSISTENCY OF SINGLE- UNIT LOCALIZATION.
potential concern regarding the localization phase is whether applying the neuron encoder F,, to a single basis vector () (representing a single neuron or feature) constitutes an Out-of-Distribution (OOD) shift, given that the encoder is trained dense activation vectors We demonstrate here that the linearity of the projection E, guarantees that the training and testing phases are mathemati- cally consistent.
Recall that the training objective is based on the symmetric InfoNCE loss. Focusing on the neuron- image alignment term, for positive pair (n, where F,,(a), the loss in direction is:
exp(sim(n,xx/71
lIiONCE = --log log Σx exp (nm
(6)
The key resolving the OOD concern lies in the linearity of E,. The activation vector is El a; - e(i) E RR where {e(t) } is the standard basis. Due to linearity, the output decomposes into projected basis vectors:
En = En Σ a3e((
=
Σ a
(e))
(7)
Single Unit Embedding
16

---

Published as a conference paper at ICLR 2026
To analyze the similarity composition, following prior work semantic decomposition (?), assume the CLIP image embedding space is spanned by a dictionary of semantic basis vectors {Ux ) such that an image is represented decompositions into the alignment calculation, the similarity function becomes a weighted sum of component-win wise similarities:
Σx ky Vk. where k2 are the coefficients. Substituting these
sim x) oc sim {Σ
as (e)
Σ U
Σ
Packe 2,
(im (e (e) vk.
Single Seuron-Semantic Align
This decomposition demonstrates that even though the training inputs a are combinations of multi- ple features, the contrastive loss explicitly maximizes the correlation between the specific activation component (the projected basis vector E, (e())) and the specific visual feature present in the im- age. Therefore, the single-neuron projection E, (e(+)) is explicitly optimized during the contrastive training process, ensuring it is not OOD during analysis.
Scale Invariance of Mechanism Analysis. A related consideration is whether investigating vectors of unit length is valid, given that the scaling of neuron activations is not standardized. We clarify that method is robust this because employ cosine similarity for all alignment measurements in both training and inference stages:
{u, c) |||||
sim u. c)
|u||
Cosine similarity is inherently independent of vector magnitude. Since E, is linear projection, scaling the imput vector by scalar a results in a linearly scaled embedding: En (aee()) (E,(e( When calculating similarity, this scalar a appears both the numerator (dot product) and the de- nominator (norm) and cancels out. Thus, our analysis relies solely on the angular direction of the embeddings, making the method invariant to the magnitude or scaling the original neuron activa- tions.
C IMPLEMENTATION DETAILS
C.I FEATURE DISENTANGLEMENT
In addition to aligning model neurons with semantic concepts, MICLIP can adopt widgets to dis- entangle the neuron space into sparse feature space, to address the issue of polysemanticity Mu & Andreas (2020), where single neuron may encode multiple unrelated concepts. In this sec- tion, we introduce k-Sparse Autoencoders (k-SAE) Makhzani & Frey (2014) that transform obscure neuron activations into sparse features that expose more interpretable relationships with human- understandable semantic concepts. It is applicable together with MICLIP, mentioned in Sec- tion
For representation E R" from specific layer in the model, k-SAE follows the dictionary learning paradigm and conducts encoding and decoding stages as shown as follows:
f = [e1, U2 ......, 2m] Top-k(W ene (a (pr)) á
W dee + bpec
(10)
encoding
decoding
The k-SAE encodes into m-dimensional feature f RD and then reconstructs it à, combination of the features, where the encoder and decoder matrix Wens Wen E mx Wass dec E Raxm and the bias bpe E IR are learnable parameters, denoted together by OSAR. Top-k(-) is operator that selects the top ks largest feature values and leaves them unchanged while setting the remaining features to zero. This introduces the sparsity constraint on the features Makhzani & Frey 20014). requiring || < k.
The k-SAE is trained by minimizing the representation reconstruction error while enforcing the feature sparsity constraint. The objective is formulated in Equation II
minimize L(OSAE: a) ||a - all subject to || < k. Osn
(11)
17

---

Published as a conference paper at ICLR 2026
Table 5: Comparison of computational cost (FLOPs) and resource usage to analyze mechanisms over 100,000 images using ViT-B-16. "-" or "N/A" indicates no training phase is required.
FLOPs (T-FLOPs)
Time
Memory
Method
Training
Neuron Loc.
Feature Loc.
Total
Training
Inference
VRAM
Storage
MICLIP (Ours)
7290.39
0.10
186.60
7477.09
30 min
2 min
16 GB
94 MB
Act- Values
3524.01
4256.42
7780.43
30 min
16 GB
97 MIB
CLIP Dissect
7270.72
7769.20
15039.92
30 min
16 GB
9.7 GB
C.2 TRAINING DETAILS
Training Details of Mechanism-Coneee Alignment. Our MICLIP trained for single epoch on a subset of 100,000 images sampled from the ImageNet- Ik training set. We employed the Adam
optimizer with a fixed learning rate of 10 4. The training was performed with batch size of 1024.
Alignment Loss Curves for Different Target Models
ResNet-50
VIT-B/16
CLIP
14
Mean Algnment Loss
14
Mean Allynment Lots
14
Mcan Alignment Loss
Contidence Interval (1: std)
Contidence interval 181 std)
Alignment Loos
12
Confidence interval (s1 sto)
Alagmment
1.0
Aligmment
-0-
20000
40000 5000a nonco 100000
0
48000 nooce accon 100000 Number of Images Processed
20000
20000 40000 cocaa sono0 100000 Number of images Processed
Number of Images Processed
Figure 7: Alignment loss curves for three target models. The curves represent the mean alignment loss and its standard deviation calculated across three random seeds for each model during training.
Training Details of k-SAF. Our k-SAE was trained on the entire ImageNet-1k 1k training set. We employed the Adam optimizer with a fixed learning rate of 10-3, The training was performed with a batch size of 32 (see Figure 8).
Training Efficiency. As shown in Table our MICLIP achieves comparable training and inference efficiency to the basclines, despite requiring one-time training phase.
k-SAE Training vs. Training Images
0.8
Treining Less
0.6
0.4
D EXPERIMENTAL SETTINGS
0.2
D.1 BASELINES
320000
640000
960000
1280000
Sct-Values is fundamental baseline identifies class- specific mechanisms (neurons features) by ranking them based on their average activation for a given class. The top-ranked mechanisms are then used for interven- tion tasks (Section 4.3
Training Images
Figure 8: Training loss curve of k-SAE for ViT-B-16 model.
Network Dissection (Bau et al. 2017) is a classic interpretability method that identifies concept detectors by measuring the alignment between a neuron's 's or feature's activation map and a concept's semantic segmentation. We follow its core principle of using Intersection over Union (IoU) as the alignment metric. However, to enable analysis on broader range of concepts in ImageNet- Ik (Russakovsky et al. 2015), we leverage Grounded SAMiRen et al. 2024) to generate concept masks.
CLIP-Dissec' (Oikarinen & Weng 2023) uses CLIP to describe the function of individual neurons. For our description accuracy comparison (Section 4.2), follow its original procedure. For inter- vention experiments (Section[4.3) we rank the neurons or features identified by -I.S-DDssect, then select those with the highest similarities for a target class.
18

---

Published as a conference paper at ICLR 2026
V-Interp (Zhang al. 2024) is inspired by text-based methods where large models explain smaller ones (Bills et al. 2023; Paulo et al.] (024). We implement a visual equivalent based on (Zhagg et al. 2024). This approach identifies the image patches that cause the highest neuron or feature activations and feeds them to a powerful "explainer" Large Multimodal Model (LMM). The LMM's zero-shot descriptions of these patches serve the final captions of neurons features. Since this process yields a fexe-text description for each neuron or feature, we then use the CLIP/ViT-B- text encoder select the unit whose description most semantically similar given target concept.
D.2 SEMANTIC ANALYSIS
D.2.1 VISUALIZATION OF LOCALIZATION FEATURES
In this section, we provide details for the analysis of semantic geometry of SAE features in MI- CLIP'S learned embedding space. We select a set of concepts from the labels of ImageNet-Ik, then categorize them into four groups using WordNet (Miller 1995) that are "mammal", "non-mammal", "tool", "vehicle".
To be more specific, "mammal" and "vehicle" refer to synset "mammal.n.0f" and "vehicle.n O.0" respectively; "non-mammal" refers to the combination of "bird.n.01", "fish.n.01", "reptile.n.01", "amphibian.n.011 .n00" and Uiverteeraate..01. "tool" refers to the combination of synsets "tool.n.01", "appliance.n.01", "furniture.n.01 and "instrument.n.01..
D.2.2 ATTENTION MAP VISUALIZATION
To qualitatively verify that our localized SAE features correspond to distinct and correct regions of an input image, we introduce a visualization technique based on feature-conditioned attention. This method adapts the attention map visualization from DINO (Caron al., (02I) but introduces a critical intervention step. Instead of merely observing the model's natural attention, we constrain the model's activations lie within the subspace of specific, pre-selected features. This allows us to generate a saliency map that directly reveals the spatial regions the model focuses on when its reasoning is guided only by the features associated with a given concept.
The process consists of three main steps: partial forward pass, targeted activation intervention, and the generation of the final attention map.
1. Partial Forward Pass and Activation Extraction Given an input image and a target intervention layer I, we perform forward pass up to that layer. We extract the activations for all tokens, separating the activation vector for (1)
the CLS token, acis R", from the matrix of activation vectors for the P image patch tokens, A putches E Pxn, Our intervention is applied exclusively to the patch tokens; the CLS token's 's activation remains unmodified.
2. Activation Intervention on Patch Tokens
This step is the core of our method. Given set of SAE feature indices ii), that have been localized to a concept e (using Equation 3. we intervene on the patch activations by passing them through the SAE and filtering the resulting sparse code.
(1) from (1) patches is passed through the pre-trained SAE encoder, Wene: to obtain its corresponding sparse feature activation vector Zp Rd where d, is the dictionary size:
First, each patch's activation vector a
2p W Wene (a bbrc)
(12)
Next, we generate masked sparse vector. žp. by retaining only the activation values at the indices corresponding to our target concept and setting all others to zero. This can be expressed as element-wise product with a mask vector m. E {0, 1]d,, where (m.); if i E L. and otherwise:
2
Zp m
(13)
Finally, this filtered sparse code is passed through the SAE decoder, Waecr to reconstruct the activation vector in the original activation space. This yields the modified patch activa- tion vector a!!
(1) W dec bpr
(14)
19

---

Published as a conference paper at ICLR 2026
This operation is applied to all patch tokens, resulting in matrix of modified activations, patches
a .....äpp IT.
3. Ceature-Conditionel Attention Map Generation The full set of activations for subsequent layers is reassembled by concatenating the origi- nal, unmodified CLS activation with the modified patch activations. The forward pass then resumes from this reassembled state. We compute the self-attention weights in subse- quent block, focusing on the attention between the CLS token and the now-modified patch tokens.
For a specific attention head h (out of H total heads), the attention weights (h) RRP. are calculated as:
a softmax (qCLS) T TK (h)
patches
(15)
(h) where the query vector 9CLS E Rdd is derived from the original CLS activation acss and
(!)
the matrix of key vectors K (h) E _xdd is derived from the intervened patch activations
patches
A
patches
The final 2D saliency map S is generated by averaging these attention weights across all attentions heads and reshaping the resulting P-dimensional vector to match the spatial layout of the image patches.
E MORE RESULTS
E. SEMANTIC ANALYSIS
To further demonstrate the effectiveness and robustness of our method in localizing features that correspond to specific visual concepts, this appendix provides more extensive set of examples across various classes. As detailed in the Analysis 5, we verify the spatial grounding of our learned features by visualizing their corresponding attention maps.
We follow the same visualization methodology described in Analysis 5. Specifically, we leverage the self-attention maps from the 8th layer of the CLIP/ViTTB-16 model, computed between the CLS token and all image patch tokens. The resulting saliency maps highlight the precise image regions to which the identified features attend. The visualizations shown in Figure and Figure [0] are generated for localocal features across a diverse set of classes, using images from the ImageNet- 1k validation set.
The results in Figure and Figure 10 demonstrate that our MICLIP consistently localizes features to the relevant semantic regions across a wide array classes. For each class, the attention maps reliably highlight the areas that define the visual concept, and this localization remains stable across different images within that class. This consistency underscores the robustness our ap- artifacts of specific images but are genuinely
proach, confirming that the identified features tied to the underlying visual semantics of the category.
E.2 ADDITIONAL CASE STUDIES OF UNDERSTANDING FLAWED VISUAL REASONING
In this section, we provide additional case studies to further validate the diagnostic capabilities of MICLIP, introduced in Section 4.6) Our approach pinpoints reasoning failures within the CLIP/ViT-B-16 model by tracing the semantic alignment of image's internal representations on layer-by-layer basis.
As a brief recall, the methodology involves projecting layer-specific activations into a shared se- mantic space using the neuron encoder E,. We then compute the cosine similarity of these evolving representations against the text embeddings of both the ground-trunte (GT) label and the label the model incorrectly predicted (the misclassified label). A divergence point, where the representation' similarity to the misclassified label surpasses that of the GT, marks the specific layer where the model's 's reasoning process falters.
20

---

Published as a conference paper at ICLR 2026
Class:
bookshop
Origial
Original
(MrCua))
(MiCuee)
Normalized Attention
Original
CLIP- Dissect
(MICU)
CLIP-Disset
(MDLLP)
Original
CLIP- LUP-DDisect
Our (MiCppp
Class: black swan
Origiol
CLiP-Dieent
Oues (MiC (P)
Origiall
CLIP-DAN CLIP-Dionct
Oun (M.CrPP)
Or
CLIP-Diowet
Oun (MIDLP)
Normalizee Attention
Oriinal
CLIP Dissect
Ours (MICUI))
Orignal
CLIP Dissect
Ours (MrCur))
Orinnal
CLIP Dissect
Our (MiCur))
30
Class: radio
Oriinal
CLiP-Diesect
Ours (M)CUr)
Origial
CLIP- Dip-eDtsect
Ours (MiCUP))
Or
PiP--Deeect Our (MICwP))
Normalized Attention
Origina
CLIP Dissect
Ours (MiCar)
Original
CLIP Dissect
Ours (MOCaP)
Orignal
CLIP Dissect
Ours (MiClr))
80
Class: zebra
Origial
CLIP Dissect
Ours (MrCUr))
Origiall
CUP Dissect
Ours (MrCur))
gnni
CLIP Dissect Our (MiCur)
Normalized Attertion
Origina
Origina
(MICuiP
(MiCur))
Class: mangoase
Original
CLIP-Diewet
Original
CLIP-Disect
CLIP-Diawet (MiCLPP)
Normalinn Attention
Original
CLIP -issects
(MiCUP))
CLP Disect
(MBCuP))
CLIP. Disectt
Ours (MiCun
Figure 9: More results for visualizing the spatial grounding of top-5 localized features.
21

---

Published as a conference paper at ICLR 2026
Class:
black grouse
Orignal
(MICCr))
Orignal
(MiCun)
Original
(MrCur))
Normalized Attention
Oriinal
CLIP-Disect
(MICU))
CLIP-Disett (M0CuP)
CLIP-Disect (MCCuP)
Class: centipede
Orighal
CLIP Dissect
(MiCUPP)
Oriinal
Dissctt
(3BCuP))
CLIP
Our (MiCur)
Normalized Attontion
Originll
CLIP-Diewct
Ours (MICUP)
Oriinul
CLIP-Diand
Oura (MICLIP)
Orijial
CLIP-Diasct
Our (MICLIP)
convertible
Orginal
CLIP-Diesect
Ours (M)CUP)
Origial
CLIP-Disend
Ours (MiCUP))
Orggnl
DiP-eDe Our (MICLP))
alizedd Attention
Original
CLIP-Diewet
Ours (MUCUP)
Orginal
CLIP-Dianct
Oun (MICUP)
Orginal
CLIP-Disnct
Our (MICLP)
Class: caim
Origiall
CLIP Dissect
Ours (MrCUr))
Origiall
CLI Diss-Discct
Ours (MrCur))
ginal
CLIP Dissect
Our (MiCwr)
Normalized Artentton
Original
(MICa))
iiginal
ginaal
(MuCur)
Class: Chihuahua
Original
CLIP (MiCur
Original
CLI (Micaa
Original
(MiCu))
Original
CLIP-Disect
Ques (MiCUP))
Origiaal
CLIP-Disset
Ours (MICUP))
Original
CLIP-Diseect
Dum (MiCuP)
00
Figure 10: More results for visualizing the spatial grounding of top-5 localized features.
22

---

Published as a conference paper at ICLR 2026
Figure I presents four additional examples of this flaw analysis. Each case demonstrates distinct failure mode, yet our diagnostic method consistently identifies the critical layer where the semantic trajectory shifts away from the correct interpretation. These examples underscore the reliability of MICLIP as a tool for gaining precise insights into model failures, confirming that the crossover phenomenon is general indicator of flawed visual reasoning.
Similarity betweed een Layer wise Represeniations and Concepts: "chain mail (GT)' and "necklace (Misclassified)
Similarity beiween Layer wise Representations and Concepls:
"Old English sheepdog (GT)' and "Irish wolthound (Misclasstheed) td)
Original Image
Correct Case
Original
Correct Case
2.00
Similanty
3.20
Sim taantttty
11
Indes
Laytr
Similarity
Simearity
Mis.
2:25
Similarity between Layer-wise Representa tions and Concepts: "lab (GT)' "stethoscope (Misclassified)
Similarity between Layer-wise Representations and Concepts:
mower (GT)" and dish (Misclassified)
Original Image
Original Image
Similartyy
6.38
GT
Similaritt
Mb...
0.20
8.28
Layer
Layer
Incorrect Case
Incoreet
Similartt
0.50
GT
Similartty
0.22
GT
Ms.
6.2
0.20
0.22
Layer
Figure 11: Additional results for flaw analysis. Each plot shows the cosine similarities between layer-wise representation embeddings and the text embeddings of the ground-truth (GT; blue) and the misclassified label (Mis.; orange). The layer where the GT similarity has the largest lead is marked by C, while the critical failure point where the misclassified label's similarity overtakes GT's is marked by @.
E.3 ADDITIONAL VISUALIZATION OF Low LEVEL FEATURES
To further substantiate MICLIP'S capability in identifying and localizing fundamental visual prim- itives, this appendix provides extended visualization results focusing low-level features. While Analysis demonstrated the localization of color concepts (e.g., "green"), here we examine whether method can effectively ground more complex low-level features, such shapes, colors and tex- tures, which are typically encoded the early layers the model.
We follow the identical visualization methodology described in Analysis 7. Specifically, we visual- ize the self-attention maps from the 3rd layer of the CLIP/ViT-B-16 model.
Figure 12 and Figure 13 illustrates more spatial grounding of feature identified by MICLIP as cor- responding to a "shapes", "colors" or "textures" pattern. The results show that this feature con- sistently attends to structures, such "red", "grid", or "round", across a diverse of images. The results from object semantics verifies MICLIP'S effectiveness in interpreting the model's behavior at different levels of granularity.
E.4 USER STUDY OF LOCALIZED FEATURES
To quantitatively evaluate the quality of the interpretability heatmaps produced by our method, conducted user study to compare the ability of MICLIP and CLIP-Dissect to local-
23

---

Published as a conference paper at ICLR 2026
ize relevant image regions. To ensure fair comparison, adhered strict blind testing protocol. An example a question from our survey is shown in Figure [4
Participants and Recruitment. We recruited participants with backgrounds in computer vi-
Table 6: User study results: User preference for explanations generated by each method.
sion. None of the participants were authors of this paper, ensuring independence. We col- lected a total of 13 participants with 130 re- sponses. The results, summarized in Table show a clear user preference for MICLIP over CIIIP-Disect..
Method
User Preference ( (%)
CLIP-dissect
23.08%
MICLIP (Ours)
76.92% %
Which method consistently activates the same concept: "tib"?
An example from our user study questionnaire. Participants were shown feature maps generated by two different methods (Method A and Method B. in random order) and were asked to select which one better explains the image's classification.
Figure 14:
E.5 VERIFYING MECHANISM LOCALIZATION VIA INTERVENTION ON LARGE VISION-LANGUAGE MODEL
To demonstrate the scalability of MICLIP, we extend our intervention analysis to the llava- hf/llava-vl fattaell.--mistra------- model. We evaluate the model on the ImageNet-1k 1k closed-set classi- fication task (original accuracy 11.78%) using a specific prompt containing all 1,000 class labels (foptions.text))
We use the following prompt instructs the model: "What type object is in this photo? Below is the exact list of choices. Each choice is enclosed in <> brackets for clarity. foptions.text Instructions: Think step by step and then output the label. Choose EXACTLY ONE from the list above. Output the label inside the <> brackets. For example, if the answer is 'sea snake", output < sea snake >. Your response label must be a perfect and totally match to one of the labels."
Accuracy is calculated via exact string matching. The model's original classification accuracy is 11.78%. We perform interventions on SAE features within the 22nd layer of the LLaVA vision tower. We compare MICLIP against CLIP-Dissect (Oikarinen Weng (023) using enhancement (x2 scaling) and removal (x0 scaling) settings, averaged over three random seeds.
Table 7: Accuracy deviations (AAce %) from enhancement and removal interventions features on Mlava-hf/llav--vll6-oa 6--ms--a---. model.
Method
Enhancement (t)
Removal (J)
CLIP-dissect
0.00 (+t 0.06)
0.03 (+t 0.03)
MICLIP (Ours)
1.11 (+ 0.01)
-1.87 (+: 0.05)
24

---

Published as a conference paper at ICLR 2026
As shown in Table 7 MICLIP effectively controls model behavior, achieving 1.11% gain in hancement and a 1.87% drop in removal. contrast, the baseline CLIP-Dissect shows negligible impact, verifying that MICLIP conrectly identifies functionally relevant mechanisms in large-scale vision-language model.
25

---

Published as a conference paper at ICLR 2026
Concept: red
Original
Orginal
CLipeDi
100
Original
CLIP Dissect
Ours
Origiral
CLIP Dissect
(urr/MirCur)
Original
CLIP Dissect
Ours/MD:Ur))
Concept: grid
Original
CLIP Dissect
OurliXtiClue
Origiral
CLIP Dissect
Origial
CLIP Dissect
Ours/
Original
Diisett
Original
CLiP-Dissect
Oura (iCup
Original
CLIP-Disuet
McCu)
Attention
Concept:
Original
CLIP -iioeit
Original
CLIP Dissect
Oura/XiiCup)
Original
CLiP-Diseect
MCCr))
Original
Origial
Ouno MMCUPP)
Originl
Qune MiDU))
Concept:
Original
CLIP- ELIP-Dianct
Ouev MiCCup
Original
CLiP-Disset
(MCCPP
Origina
CLiF-Diveet
Ouno MiCup)
Normalinn
Origial
Origiall
Orignal
Concept: striped
Original
CLIP Dissect
Origiral
CLIP Diseect
Ours (MCCrr))
Oriinal
CLiP-Diosect
Dwn MiUua))
10
0.2
Original
Ows//fiClur
Origiral
Disect
Mirsi
Origial
CLIP Dissect
Ours(M(CCu))
101
Attention
Figure 12: More results for visualizing the spatial grounding of color, shape and texture features.
26

---

Published as a conference paper at ICLR 2026
Concept:
Origiall
Origial
Digiial
Original
CLIP Dissect
Origiral
CLIP Dissect
Ours/MrrUur)
Original
CLIP Dissect
Ours MiCUr)
Attention
HEERSS
Concept: vertical line
Original
CLIP Dissect
Origiral
CLIP Dissect
(urs
Original
CLIP Dissect
Ours/MiUUr)
Original
CLIP L.P--Disect
Original
CLIP CLIP-Dissct
Oural MiCUPP
Original
CLIP-Disuett
McCus
Attention
Concept: orange
Original
CLIP DiP-Disect
Original
CLIP Disscit
Qurai (aCCUP)
Original
CLiP-Diseet
Ouri MCCs
Original
Ounx(MiChue)
Origial
Ounx/MiCUup
Origial
Oun MiDUs)
Concept:
Original
CLIP-Dianct
Original
CLiP-Diseet
Ounn (MCUPP
Original
CLiF-Diswet
Oued MiCup)
Normal
Original
Original
(urCur))
Original
Concept: fabric
Original
CLIP Dissect
Origiral
CLIP Disenct
Ours (uTCUr))
Origial
CLiP-Diosect
Owni Mitur))
0.2
CLIP Ouss/SiiCiur
Origiral
Originl
CLIP Ours/MDCUa)
10.
Attention
Figure 13: More results for visualizing the spatial grounding of color, shape and texture features.
27


Reviews:
[{"review_id": "n1HRG1cfKG", "reviewer": "Reviewer_sbm3", "summary": "This paper introduces MiCLIP, a novel framework for mechanistic interpretability of vision models that aligns internal representation units (neurons or sparse features) with human-understandable concepts in CLIP's semantic space. The key innovation is using contrastive learning to create a shared embedding space that connects model internals with both input images and output predictions, moving beyond the traditional \"activation-magnitude assumption.\" MiCLIP enables bidirectional interpretation (concept-to-mechanism localization and mechanism-to-concept description) and supports model steering through targeted interventions. Extensive experiments on ResNet-50, ViT-B/16, and CLIP/ViT-B-16 demonstrate superior performance in interpretation accuracy, intervention effectiveness, and generalization to unseen concepts compared to established baselines.", "questions": "See weakness", "limitations": "", "rating": 4, "confidence": 3, "soundness": 2, "presentation": 2, "contribution": 2, "strengths": "1. **Novel and Well-Motivated Approach:** The dual input-output grounding through contrastive learning represents a significant advancement over existing input-centric interpretability methods. Moving beyond the activation-magnitude assumption addresses a fundamental limitation in current mechanistic interpretability research.\n\n2. **Comprehensive Technical Framework:** The method provides a unified approach that works with both neurons and sparse autoencoder features, offering flexibility across different granularities of model internals. The integration with k-SAE for feature disentanglement is particularly valuable for addressing polysemanticity.\n\n3. **Thorough and Multi-faceted Evaluation:** The paper presents an extensive experimental evaluation covering:\n   - Quantitative interpretation accuracy across multiple models and concept sets\n   - Intervention studies demonstrating precise model control\n   - Generalization to unseen concepts and datasets\n   - Semantic geometry analysis and attention map visualizations\n   - Failure diagnosis through layer-wise semantic trajectory analysis\n\n4. **Practical Utility:** The framework enables meaningful model steering and provides insights into flawed reasoning processes, making it valuable for both understanding and improving vision models.", "weaknesses": "1. **Dependence on CLIP's Semantic Space:** The interpretability power is inherently limited by CLIP's semantic coverage and potential biases. A more detailed discussion of these limitations and how they might affect interpretation fidelity would strengthen the paper.\n2. **Architectural Scope:** While the method is evaluated on several discriminative vision architectures, its applicability to generative models or larger vision-language models remains unexplored. Some discussion of potential challenges in scaling to these domains would be valuable.\n3.**Computational Cost Analysis:** Although Table 5 provides FLOPs comparison, a more comprehensive analysis of training time, memory requirements, and scalability to larger models would help practitioners assess practical deployment.\n4. **Lack of Human Evaluation for Interpretability:** While the paper provides extensive automated evaluation, it lacks human studies to validate whether the discovered concept-mechanism alignments are actually meaningful and useful to human users. Automated metrics like CLIP score don't necessarily correlate with human-judged interpretability quality. The approaches of ACE (Towards Automatic Concept-based Explanations) and CRAFT(CRAFT: Concept Recursive Activation FacTorization for Explainability) to measure the interpretability quality could be considered", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "Mtl4f79xdj", "reviewer": "Reviewer_E7Mu", "summary": "This paper proposes a mechanistic interpretability approach for vision models, by projecting model internal representations to CLIP embedding space. The authors train an encoder to project model activations to CLIP embedding space via two contrastive losses: (1) between the projection and the CLIP embedding of the model output and (2) between the projection and the CLIP embedding of the image itself. The idea is to then project one-hot activations (resembling individual neurons) to CLIP space and thus learn which concept a unit represents. One can then conduct causal interventions on models, by modulating the units relevant for a concept. The authors validate their method by applying it to the final classification layer of various models and showing that the projections are close to CLIP / Mpnet embeddings of the ground-truth labels.", "questions": "- I assume you keep the target model frozen and only train the encoder, but just to make sure: Which parts of the training pipeline are frozen?", "limitations": "", "rating": 2, "confidence": 4, "soundness": 2, "presentation": 2, "contribution": 3, "strengths": "- The motivation of the work is very good, both the field's reliance on the activation-magnitude assumption and the fact that existing approaches focus on input-centric explanations too much is valid.\n- The work is well-placed in the related literature.\n- The proposed method is applied to multiple models of different architectures, and compared against multiple reasonable baselines.", "weaknesses": "- The implicit assumption behind this work, which is never addressed in the text, is that CLIP embedding space is somehow interpretable. To what extent this is true is hard to say, but my intuition is that while the method may work for the kinds of object-level representations one finds in late layers, it likely won't transfer to earlier layers which detect more primitive features.\n- The clarity of the writing could be improved. For example, from figure 2 one might think that the target model is trained or fine-tuned (because while the clip-encoders are show to be frozen, the target model does not) and the paper generally spends more time making claims than backing them up (eg, line 198). The \"Common-Nk\" datasets in table 1 are never explained, I have to assume they are subsets of COCO?\n- One technical problem that I see is that $E_n(a_i \\cdot e^{(i)})$ is probably OOD for the encoder. The $a_i$ would have to be post-relu activations of very sparse layers for training on real activation vectors to generalize to these base vectors. Investigating vectors of unit length is also questionable because this assumes that the scaling of neuron activations is somewhat standardized, which might not be the case.\n- The validation of the approach in table 1 is in my opinion insufficient, because the final classification layer is special w.r.t. the two issues outlined above: It represents high-level semantic concepts, and is likely much sparser than other layers.\n- The reporting of significance in table 1 is superfluous: At 100k test images, even very small differences are significant. I would save the space and write in the caption that all differences are significant.\n- In principle, I like the analysis in table 2, but I would have done it differently: You are currently not evaluating the specificity of interventions. When you are removing units responsible for a concept, you are probably not only hurting performance on the target class, but also on other classes. The relevant performance metric should not just be the absolute delta in classification accuracy for the target class, but the relative drop in accuracy for the target class relative to other classes. For example, it's possible that the -17% average accuracy delta for the target class you find in ResNet-50 is not different from all other classes, which could also be dropping by 17%. Higher specificity would mean that the target class accuracy drops by 17% while the average accuracy of all other classes drops by only 1%, for example.\n- I thus don't think findings 2 and 3 are sufficiently supported by evidence (in Table 3 the CIs overlap and the absolute deltas are very small).\n- Minor point, but in section 4.6 the example in the text doesn't match the example in figure 5.\n- At the very end of section 4.6 you claim that your method points out \"where the model's view shifts and why it fails\". The first part is reasonable, the latter should be removed: We have no idea why the similarity to the wrong label surpasses that to GT, we can just observe that it happens. \n\nOverall, I am quite unimpressed with the paper. I initially liked the pitch of overcoming the activation-magnitude assumption, but I don't think the approach convincingly shows that it achieves any of its objectives. It is a general issue with the field of interpretability that evaluation is hard and usually not properly done, but I find these evaluations particularly insufficient.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "EILQIwpLsi", "reviewer": "Reviewer_op3v", "summary": "This paper proposes MiCLIP, a method that trains a CLIP neuron encoder to align with the visual and textual encoders of CLIP such that a neuron can be converted into an interpretable embeddings space (CLIP's embedding space) that can be queried via text. It aligns intermediate neuron activations of a target model (e.g., ResNet50) to the input image, and also adds the alignment of the activation values the predicted class labels for improved faithfulness. The method performs well compared to baselines.", "questions": "At the current stage, there are many problems in the paper (see weaknesses). Therefore my decision will be borderline reject for now. My decision could potentially be adjusted in the rebuttal.", "limitations": "", "rating": 4, "confidence": 4, "soundness": 3, "presentation": 4, "contribution": 2, "strengths": "- The paper is clearly written and easy to follow, it was an interesting read. I would like to thank the authors for the comprehensive appendix also. \n- The authors tackle the \"input-centric\" paradigm (usually unfaithful to the mechanism of the model's decision-making process), and which often fail in cases of incorrect predictions (because they are explicitly trained on human-desired concepts). They do this by considering the model’s output decision. \n- The proposed method can work with SAE features without actually training on those features (this means the SAE features are basis vectors of the dense features from the residual stream)\n- The idea of representing a single neuron in the CLIP embedding space (via the Neuron encoder) is very interesting. \n- The analysis is Figure 5 is interesting as it allows us to see the layer where the model fails.", "weaknesses": "- The paper assumes that the target layer they operate on, actually contains concepts they align to (here, the concepts are the predicted class labels). This does not really make sense to me. Different layers have different functions. For example, early layers represent low-level features such as textures, blobs, edges...etc. But the authors align them with high-level concepts (class labels). In other words, we never know the actual concepts that a target layer uses, so we cannot assume that these concepts are the predicted class labels. For me, this is a major weakness. \n- The work of [R2] is very similar, it uses CLIP to learn interpretable embeddings (also sparse) which is exactly what the authors do. Although this work investigates CLIP, the general idea of learning a CLIP model to represent interpretable embeddings is the same.  Furthermore, [R3] also investigates the same idea of interpreting intermediate representations of models (including fine-grained neurons) with text by generating text in an autoregressive manner, and is trained on MILANNOTATIONS dataset [R4], a human-curated dataset for neuron annotation\n- Why dont the authors include MILAN [R4] as a baseline? It is a popular baseline and should be reported. \n- I am not convinced with the motivation of the paper. The claim in L52-53: \"An increase in activation value does not necessarily imply the occurrence of the corresponding concept during inference. Conversely, even negative activations can positively influence the model’s\nprediction of certain concepts.\" Is there any reference for this? How do the authors come up with such a claim? It has been well-established since 2013 that high activation values correspond to concepts (or parts of concepts). Potentially, they could be biases (e.g., dumbells and hand occur together so the \"hand\" concept will activate on dumbell images). However, high activation values still *drive* the propagation to other future layers and eventually the prediction. The authors repeatedly mention this problem all over the paper, but to me this problem is not well justified. \n- Since the method aligns the neuron representation in the CLIP space, it therefore inherits CLIP's biases. For example, it may inherit the text-spotting bias in CLIP, even though this bias is not present in the target model.  \n- There is an evaluation method proposed for neuron annotation works [R1], have the authors considered this evaluation method? Right now, most of the evaluation involves interventions. \n- In Table 1, why aren't the other baselines (Act-Values, Network Dissection and V-Interp) reported? \n- In Table 2 I dont understand why dont authors report the overall accuracy of the model with enhancement? For example, ResNet-50 imagenet accuracy is around 76%. The the 5% boost, does that mean it is 81% ? \n- The method seems to work less well for transformer models, according to Table 2. \n- The description of Figure 5 is wrong. It mentions a CD player but it is actually a sea anemone and feather boa. \n- Minor: In Table 1, what is the need of \"Sig.\" ? Is it it fill space? \n\n[R1] CoSy: Evaluating Textual Explanations of Neurons\n[R2] STAIR: Learning Sparse Text and Image Representation in Grounded Tokens\n[R3] DeViL: Decoding Vision features into Language\n[R4] Natural Language Descriptions of Deep Visual Features", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "zsKrY5RSk0", "reviewer": "Reviewer_4V8N", "summary": "The paper presents MICLIP, a framework that interprets internal units—either neurons or SAE-derived sparse features—by learning a lightweight encoder that maps their activations into CLIP’s semantic space. Training uses a symmetric InfoNCE objective that aligns each unit’s representation simultaneously with the input image embedding and the model’s predicted-label text embedding, replacing activation-magnitude heuristics with representation-level alignment. Once trained, the shared space enables concept→mechanism localization (finding units for a queried concept) and mechanism→concept description (annotating a chosen unit), and it naturally extends to SAE pseudo-activations to mitigate polysemanticity. The authors validate behavioral relevance through interventions that amplify or ablate selected units and consistently shift model outputs in the predicted direction, and they report strong results across ResNet/ViT/CLIP backbones with out-of-distribution generalization to texture concepts (DTD). Additional analyses, including layerwise error trajectories, demonstrate that MICLIP offers a practical, general recipe for unit-level interpretation without relying on unstable gradients or purely input-centric assumptions.", "questions": "Low-level features. Can you provide qualitative low-layer examples (e.g., “white horizontal line,” “45° diagonal line”) showing agreement between SAE top-activations and mechanism→concept retrieval? It would be a great aid to show MICLIP can effectively explain SAE features.", "limitations": "", "rating": 8, "confidence": 2, "soundness": 3, "presentation": 3, "contribution": 4, "strengths": "Gradient-free “causal” proxy via CLIP. Instead of using unstable gradients or handcrafted causal scores, the paper detours through CLIP to obtain a robust semantic target, then validates behaviorally via interventions. \n\nTruly dual-anchor alignment (novel). Aligning activation→text and activation→image (not just label text) is a clean, original design that reduces input-centric bias and avoids the “activation-magnitude == importance” heuristic.\n\nWorks on SAE pseudo-activations. The description/localization pipeline applies neatly when only a single SAE feature is “turned on” via the decoder, which is practically valuable for polysemantic layers.\n\nThorough, convincing experiments. The paper hits standard metrics, reports state-of-the-art, includes interventions (↑ with amplify, ↓ with ablate), spans multiple backbones, and shows the method also works at OOD textures.", "weaknesses": "Not causal by design. The loss aligns to CLIP (image/text) semantics rather than optimizing a constraint tied to model logits or a formal mediator test.(Although this may be unstable) - so the causality holds between CLIP output and layer output, not between model output and layer output. The method demonstrates empirical causal relevance via interventions, but the mechanism is not guaranteed by the objective.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)