Boosting the visual interpretability of CLIP via adversarial fine-tuning
Authors: Shizhan Gong
Haoyu LEI
Qi Dou
Farzan Farnia
Venue: ICLR 2025 Poster (2025)
Keywords: interpretability
vision-language models
CLIP
adversarial fine-tuning
Primary area: interpretability and explainable AI

Abstract: CLIP has achieved great success in visual representation learning and is becoming an important plug-in component for many large multi-modal models like LLaVA and DALL-E. However, the lack of interpretability caused by the intricate image encoder architecture and training process restricts its wider use in high-stake decision making applications. In this work, we propose an unsupervised adversarial fine-tuning (AFT) with norm-regularization to enhance the visual interpretability of CLIP. We provide theoretical analysis showing that AFT has implicit regularization that enforces the image encoder to encode the input features sparsely, directing the network's focus towards meaningful features. Evaluations by both feature attribution techniques and network dissection offer convincing evidence that the visual interpretability of CLIP has significant improvements. With AFT, the image encoder prioritizes pertinent input features, and the neuron within the encoder exhibits better alignment with human-understandable concepts. Moreover, these effects are generalizable to out-of-distribution datasets and can be transferred to downstream tasks. Additionally, AFT enhances the visual interpretability of derived large vision-language models that incorporate the pre-trained CLIP an integral component. The code of this paper is available at [the CLIP_AFT GitHub repository](https://github.com/peterant330/CLIP_AFT).

Published as a conference paper at ICLR 2025
BOOSTING THE VISUAL INTERPRETABILITY OF CLIP VIA ADVERSARIAL FINE-TUNING
Shizhan Gong, Haoyu Lei, Qi Dou & Farzan Farnia
Department of Computer Science and Engineering The Chinese University of Hong Kong [szgong22, hylei22, qdou, farnia]ecse.cume cwa.essse.cck.eeeeee .k
ABSTRACT
CLIP has achieved great success visual representation learning and becoming important plug-in component for many large multi-modal models like LLaVA and DALL-E. However, the lack of interpretability caused by the intricate im- age encoder architecture and training process restricts its wider use in high-stake decision making applications. In this work, we propose an unsupervised adver- sarial fine-tuning (AFT) with norm-regularization to enhance the visual inter- pretability of CLIP. We provide theoretical analysis showing that AFT has im- plicit regularization that enforces the image encoder to encode the input features sparsely, directing the network's focus towards meaningful features. Evaluations by both feature attribution techniques and network dissection offer convincing evidence that the visual interpretability of CLIP has significant improvements. With AFT, the image encoder prioritizes pertinent input features, and the neu- ron within the encoder exhibits better aligmment with human- un--e-ttandabble con- cepts. Moreover, these effects are generalizable to -uttr--distributionn datasets and can be transferred to downstream tasks. Additionally, AFT enhances the vi- sual interpretability of derived large vision-language models that incorporate the pre-trained CLIP an integral component. The code of this paper is available at AFT
actps://github wwt.
INTRODUCTION
Recent advancements in vision-language foundation models have successfully facilitated multi- modal representation learning that aligns heterogeneous inputs into unified embedding space. A prominent example is CLIP (Radford et al. 2021), which employs contrastive learning on large dataset paired text and images to associate images with their textual descriptions. This method effectively positions similar concepts close together and demonstrates high efficacy in down- stream tasks such as zero-shot classifications (Saha et al., 2024) and open-vocabuliy object detec- tion (Minderer et al.) 2024). Additionally, CLIP serves
built-in component in several large vasion-language models including DALL-E (Ramesh et al. 2021 and LLaVA (Liu et al. (024).
The flexibility and generalizability of CLIP render it highly attractive for high-stake decision- makinn processes such as medical diagnosis (Kim et al. (024) and autonomous driving (Xu et al. 2023b
In these safety-critical applications, neural network interpretation becomes paramount. However, several studies (Chefer et al. 2021a Li et al. 2022 2023) have demonstrated that CLIP has unsat- isfying interpretability. This deficiency can be partly attributed the inherent nature of ViT (Fel et al., (022) and partly to the non-smoothness induced by false negatives during the contrastive training (Gao et af., (202T. Wu al., 2021). To elucidate this, we present the results of two pop- ular explainable AT techniques in Fig. [ namely saliency maps (Simonyan et al.) 2013] Selvaraju ctt al., 2017) and network dissection (Bau et al. (017).. Both the simple gradient map and Grad-Cam saliency map from the CLIP's ' image encoder look noisy and random. Additionally, the number of concept detectors within the encoder, a common measure of the model interpretability, is limited. Given that CLIP's image encoder is frequently employed as plug-in component of various tasks, enhancing its interpretability could significantly improve the reliability downstream applications.

---

Published as a conference paper at ICLR 2025
Interpretability (# of concept detector)
* Interpretabillity (saliency maps)
A work station in use inside an office.
Text Enooder
Ti
T5
T
42 ())
Image Encoder
12
Interpretability (# of concept detector)
Interpretability (sallency maps)
41"" (7(3)
Image
159 (1)
Encoder W/AFT
5
Figure 1: After adversarial fine-tuning (AFT), the image encoder of CLIP can generate more sensible saliency maps and contain more concept detectors, becoming more interpretable.
Adversarial training (AT, Madry et al., 2017), initially designed to bolster the adversarial robustness of models, has been empirically and theoretically demonstrated across numerous studies to also
enhance the interpretability of neural networks (Ross & boshi-Velez. 2018 Xu et al. 2023a). It has been shown suppress irrelevant features and accentuate the discriminative features in saliency maps (Ross& & Doshi Velez 2018 Kim et al. 2019,Etmann Etmann et al. 2019 | Chalasani et al. 2020 Shah al. Waag et al. |021 Gong al. 20024). and enable networks with different architectures to encode common causal patterns (Ren et al. [2023). Furthermore, by enforcing smoothness in net- work predictions within the local neighborhood of data points, AT improves the local linearity the network, widely-used metric for model interpretability (Li al. 2020 Khan et al. 2024). Regard- ing the application of AT to multi-modal models such as CLIP, the recent FARE method (Schlarmann et al. 2024) utilizes AT to improve the robustness of the CLIP model against norm-bounder per- turbations. However, extending the application of AT to boost the interpretability of multi-modal vision-language models remains underexplored and is still limited uni-modal classification set- tings in the literature. Additionally, it is unclear whether the AT's effect on interpretabiliy exhibits zero-shot generalizability can be transferred downstream tasks.
In this paper, we systematically study the relationship between AT and the visual interpretability of CLIP. We focus on adversarial fine-tuning (AFT) to mitigate the computational demands and optimally utilize the extensive pre-trained models available. We begin with an objective function minimizing the Læ--dittacce between visual-language similarity scores w/wo perturbations. We demonstrate that AFT with designed norm regularization can be reformulated as standard training with dual-norm regularization on the input gradients. Therefore, to promote the sparsity of gradient-based saliency maps, we specifically consider a Huber loss function as the dual norm in the optimization, which leads to a piecewise quadratic penalty function the proposed AFT min- max optimization. The designed AFT optimization problem concentrates the fine-tuned model, which differentiates our proposed method from the existing FARE method leveraging AT to improve the robustness of the CLIP model against for norm-bounded adversar- ial perturbations. Subsequently, we derive an upper bound for the minimization objective function, which is independent of the text embedding. By minimizing this upper bound, we can isolate the text encoder and fine-tune only the image encoder. The fine-tuned image encoder remains compatible with the original text encoder and downstream large language models tuned with CLIP embeddings.
the interpretability of
We empirically compare the visual interpretability w/wo AFT, The results demonstrate that AFT sig- nificantly enhances the quality of the saliency maps (see Fig. DT. The improvement independent of the specific saliency map techniques employed, encompassing fundamental feature attribution meth- ods (e.g., Simple Gradient, Grad-Cam as well as more advanced approaches (e.g.. M2IB (Wang et al. 2023). Grad-ECLIPP (Zhao et al.) 2024). ). It is also transferable to out-of distriiuutt data and applicable to various downstream tasks, even when tuned on relatively small datasets. We also eval- uate the interpretability of the image encoder through clip-dissect (Oikarinen & Weng 2022) and network dissect (Bau et al.) 2017). Our findings reveal that AFT enhances the alignment of neu- ral activations with human- undaanstnnntsteaablllleeee concepts and promotes more object-centric activations. Additionally, we demonstrate that AFT improves the visual interpretability of large vision- vision-language models when integrated with CLIP plug-in component. Our main contributions are as follows:
2

---

Published as a conference paper at ICLR 2025
We propose unsupervised AFT with norm-regularization that can enforce the pre-trainnee trained CLIP to encode visual features sparsely and thus improve its visual interpretability.
We theoretically prove the duality relation between the regularized norms of adversarial perturbations and the input gradients to explain how AFT improves visual interpretability.
We provide various quantitative and qualitative experimental results to support that AFT improves the visual interpretability of CLIP and its derived large vision-language models.
2 RELATED WORK
Adversarial Robustness of CLIP Model. While imperceptible perturbations may significantly change the network prediction (Szegedy et al., 2013 Athalye al. 2018). existing literature have proposed many methods (Madry et al.) 2017: Sinha et al.) 2017: Zhang et al. 2019 Cohen et al.) 2019) to improve the robustness of neural networks towards such adversarial attacks. Studies (Mao) al. 2022 Gu ct al. 2024) also find the CLIP model vulnerable to ad attacks. To improve the adversarial robustness of the CLIP model, Wang et al. (2024b) applies an efficient adversarial training strategy during the training phase of the CLIP; Mao et al. (2022) puts forward supervised fine-tuning strategy to improve the robustness of specific downstream tasks; |Wang et al. (2024a) (2024)
retrains more robust image encoder guided by the original image encoder; [Schlarmann
proposes an unsupervised fine-tuning method that achieves robustness with only the image branch being fine-tuned. Deviated from their motivation, our work explores the possibility of building more interpretable CLIP model via a tailored adversarial fine-tuning strategy.
Interpretations in Computer Vision. Interpretability is defined as the capability to provide expla- nations understandable to humans (Doshi- & Kim 2017). To understand the decision process
of the black-box neural networks, a series of work focuses on feature attribution, where saliency
maps are used to highlight the important features (Simonyan et al. [2013] Ribeiro et al.
2016 Lund-
berg & Leel 2017, Sundararajan et al. 2017: Selvaraju et al.) [201]] Shrikumar et al.) 2017 Zhang
& Farnia, 2023, Muzellec et al. 2024 Gong et al. 2024bb Ye & Farnia 2024). Another stream of
work give mechanistic interpretation to the neural network by understanding which concept each neuron is associated with (Bau et al.) [0177: Oikarinen & Weng 2022, Kalibhat et al. 2023] Ahn
ctt al. 2024, [Bai et al.
et al. (024). Networks with higher interpretability would have more concept detectors and each neural should align better with a specific concept. We apply both feature attribution and network dissection to evaluate the visual interpretability of CLIP.
CLIP Interpretability. There few existing works focus the interpretation of CLIP, with most of them focusing on concept-based explanations which ties to decompose the visual embed- dings into multiple concepts represented by text embeddings (Yun et al.] [022 Moayeri al. 2023
Chen et al. 2023 Gandelsman et al.] [2023] Oikarinen et al. [023] Yang et al. 2023 Chattopadhyay
ct al. 2024 Bhalla al. (002)). In terms of visual feature attribution, Li et al. (2022) (023) modify
the architecture of the CLIP to improve the quality of saliency maps; Wang et al. (2023) proposes
a CLIP attribution method based on the information bottleneck principle; (Gandelsman et al. (2023)
decomposes the image representation across image tokens to generate heatmaps; /hhao et al. (0244)
presents a gradient-basee method for visual explanation. While most of these work applies black- box or gray-box attributions, i.e., using no or only the gradient information of the last layers, they excel at explaining the interaction between visual and text branches with less attention paid to inter- preting the entire visual encoder. Our work, instead of proposing for interpreting the prediction of CLIP, would focus on understanding and improving the inherent interpretability of the visual encoder, which is generalizable and transferable to downstream tasks.
-attibbtaatiiitiiinn technique
3 ADVERSARIAL
FOR MORE INTERPRETABLE CLIP
In this section, we introduce the AFT algorithm and give a theoretical explanation of why it can improve the interpretability of the image encoder. The method overview is illustrated in Fig D We take a pre-trained CLIP model with any backbone and adversarially fine-tune its visual encoder using a relatively small, image-only dataset, guided by the objectives in Eq The framework allows for flexible regularization choices based on the desired properties of the saliency maps. Our findings indicate that this adversarial fine-tuning enhances the quality of the saliency maps produced by CLIP and improves the alignment of its neurons with uaman-understandable concepts.
3

---

Published as a conference paper at ICLR 2025
3.1 NORM- DORM-REGULARIZED ADVERSARIAL TRAINING FOR CONCISE INTERPRETATIONS
We first define the form of adversarial fine-tuning objective in the context of language-image con- trastive learning. The CLIP composed of an image encoder and a text encoder, which encodes the language-image -mage pair into their representations. We use x, Xx. and Tx to represent the input image, its image embedding, and the paired text embedding, respectively. Assume Ix and Tx have been normalized into unit length, the language-imas similarity is the cosine similarity between Ix and Tx, i.e., 'x AFT aims to improve the robustness of the fine-tuned image encoder fo(-) towards minor perturbations so that the aanuagg--maage similarity remains the same w/wo perturbations. To this end, we define the objective function as follows:
min Dxw max 6x
lfo
5x)1 - I2) h(6x),
where h(-) is regularization term for the perturbations. The square loss promotes the robustness towards perturbation and the regularization ensures the perturbation is imperceptible. To avoid being stuck in the non-trivial stationary point, i.e. the original pre-trained parameters, propose add Gaussian Smoothing to the original function. This can also improve the smoothness of the loss term.
We observe the first-order Taylor approximation of Eq. T can be formulated as follows:
min Ex My (0) + max &% wx 1
[TV
fo(x + z) h(6x),
(2)
8x
where mx((xx) E N2
+ 5x)] - IT and wx = |T E o0. [/o(x
z)] I. The following theorem shows the optimized 02 in Eq. and Eq, 2 are close to each other and their relative error bounded for any y-strongly-conver h(-).
Definition 1. We call RRd + R j-strroggyy-conve if for every 2 and t [0, p + (1- t)z)= S fxp(x) + +(1 11--t) (1- -))((z)) (1 - f)|lz x|I2. (Remarks: @ is not necessarily differentiable.)
Theorem 1. Assume h(.)) is p-strongly-comvex. For every 0 and x. define 8i to be the optimal
solution in Eq. [ andd 82 (x) to be the optimal solution Eq. Then, their relative difference
satisfies the following:
(x) - 520 (x)11/118 (x) - 5/(2u0²).
(3)
We defer the proof to the Appendix A.1 The bound shows that with reasonable μo² (x)
good approximation of Eq. [ Moreover, with h (-) to be the Fenchel conjugate of h(-). we can rewrite Eq. 2 as: ming Ex mx + h" VxE2z-N0, T fo(x (x++2)) 2) We notice the term (xE,~N'10. ( [T fo(x+ the SmoothGrad (Smilkov et al. 2017) of similarity score w.r.t input. Therefore, we conclude the proposed AFT has an implicit regularization on the input gradients, enforced by the function h (-).
and 82p (x) would be close, and therefore Eq.
actually
Following this general guideline, we can identify appropriate h-(-)) to enforce sparse concept encoding of the image encoder while simultaneously regulating the approximation error. We propose to regularize the SmoothGrad with the smoothed version of L1 Li-norm regularization:
h" h+( = > H, where (u
u
(1)
if |u S otherwise,
(4)
2
and the corresponding h(-) is strongly-conver and can be written as:
2c |vI if [[v|l> S
h(v)
(5)
+00, otherwise,
where e, 17 O. iss wahper-pprrmeterr controlling the strength of the regularization and r determines the smoothness of h and therefore the convexity of l(()). This sparsity-iduccing regularization forces the visual encoder to foeus on only few key features from the input samples. For one thing, it can mitigate the noisy issue of the saliency maps, making them of higher visual quality. For the other, it can boost the conciseness during the concept encoding phase of the network and potentially make the generated concept sechtecture-agnosse (Ren et al. 2023). Overall, it makes the reasoning process of the visual encoder more interpretable.
We would highlight that although enforee sparsity in this particular application, our framework is highly flexible. We can switch h² (-) to other smooth regularization (c.g. smoothed version of group- norm elastic net regularization) to enforce different properties of the encoder (see Appendix A.4).
A.44.
4

---

Published as a conference paper at ICLR 2025
orig Sesan lime
clip
clip NY AFT
brick wall
English springer
small church
white
golf ball
chocolate
yellow warbler
brain tumor
non-cancerous polyp
bulla dermoscopy
(a)
original Image
olp
clip W/
car
English springer
church
maple
gas
yellow gas pump
linear probing (d)
pump
Figure 2: (a,b) Comparison of Simple Gradients/Grad -a between CLIP w/wo AFT. AFT greatly improves the visual quality. (c) Evaluation of Simple Gradients on out-of-distribution dataset. (d)
Evaluation of Simple Gradients with linear probing. The improvements of visual interpretability stem from AFT can transfer across datasets and different tasks.
3.2 UNSUPERVISED TRAINING BY UPPER BOUND MINIMIZATION
One limitation of the objective function in Eq. is that it necessitates the calculation of Tx. This makes the fine-tuning process computationally expensive. as we still need the text embedding en- coded by the text encoder, despite the encoder being frozen. Furthermore, this constraint ne- cessitates that the fine-tuning process be conducted on datasets containing language-iman image pairs, thereby limiting the potential to fine-ture the encoder on image-only datasets, which are more read- ily available in many applications, such the medical imaging domain. Additionally, the perfor- mance sensitive the quality the text descriptions. Fine-tuning on small scale image-text pairs can degenerate the zero-shot ability (Mao et al., 2022). However, have observed that the inner maximization objective of Eq. [ has uniform upper bound independent of the text embedding:
max m (6x) h(66)) < max } 6x
1lm. 6x
8x) 1x12 h(5x).
Therefore, minimize this upper bound instead bypass the demand for embeddings:
min Ex Dem max [e z &x) - 1x1 h((x))
The optimization objective is now the Li-distance between the perturbed image embedding and the original image embedding. In the Appendix A..4, we empirically study the effects of this approxi- mation. Moreover, we have the following observation showing after this unsupervised fine-tuning, the smoothed image embedding exhibits alignment with the original text embedding.
Observation 1. For every image x, its original image embedding Ix. embedding of text prompt Tx, and fine-tuned network parameters 0, if [|E=~N(0,03)) ffoxx + 2)] Ixll - X. then IT |TTE oo +2] TId Txl X.
5

---

Published as a conference paper at ICLR 2025
Imagenette
CUB 200-20111
Calterh-256
200
accuracy (%)
accuracy (%6
accuracy (96
SG [ciip]
SG [clip w/ AFT)
22
GC (clp)
GC AFT)
percentage (%) of pixels masked
percentage (%) of pixe's masked
percentage (%) pixels masked
100
accuracy (%u
accuracy (%6
(%6
8
13
33
30 25
20
10
20
20 50
23 30 percentage (%) of pixels masked
50
70
percentage (%) of pixels masked
percentage 1%) of pixels masked
Figure 3: Analysis of Remove and Retrain (ROAR) on zero-shot classification (top) and linear probing (bottom).. A more rapid drop in accuracy shows the highlighted features are more important. AFT enables CLIP's vision encoder to better capture task relatted features.
Therefore, the smoothed image embedding is adaptable to the CLIP's text encoder. Empirically, We discover fe(x) is close to Ez-N(0, (x 2)], since o will not be too large. Therefore fo(x) can also be compatible with the text encoder, and potentially downstream large vision-language models tuned using the CLIP''s embeddings. Note the framework can be generalized to any foundation model that entails intermediate embedding layer linking modalities. We will show numerical results on applying this framework to multi-modal models other than CLIP in the Appendix
4 EXPERIMENTS
We fine-tuned the CLIP using the ImageNet (Deng al. 2009) training set for 2 epochs. We
experimented with both VIT (Dosovitskiy et al.) 2020) and ResNet (He et al. (016) architecture. Without special illustration, the results are based on ViT-B/16 with o = = 255 and 4/255. More comprehensive results can be found in the Appendix [A.4| For AFT, we applied PGD (Madry et al. 2017) for 10 steps. All experiments were conducted on NVIDIA GeForce RTX 4090 GPUs.
4.1 IMPROVEMENTS OF SALIENCY MAPS THROUGH AFT
We first examine how AFT can improve the quality of the saliency maps. Our experiments primarily focus on two most fundamental saliency maps: Simple Gradients (SG) and Grad-Cam (GC). Both methods rely on unaltered gradient information for local explanation, eschewing complex design and post-processing aimed at quality enhancement. We believe this can better reflect the inherent interpretability of the network. Superior saliency maps should be indicative of an improved inter- pretability of the network, rather than the result of intricate manipulations of the saliency maps.
Visual Quality. We visualize several saliency maps for images selected from the validation set of ImageNet in Fig. (a). The SG of the original CLIP looks largely stochastic without highlighting any meaningful patterns. It also exhibits minimal variation when different prompts are used. Conversely, after AFT, the SG demonstrates substantial improvements, accurately capturing the objects within the image related to the prompt. It also displays increased sensitivity to the prompt variations. This trend is similarly observed for GC (Fig. (b)). Although the quality of GC for the original CLIP is superior to SG, it still suffers from background noise interference. AFT enables the GC to align better with the objects in the foreground. The results show that the original CLIP would rely on patterns that are obscure to human perception for Docision-making. After AFT, the model leverages more human- interarrarerreetbble concepts for prediction, thereby enhancing its interpretability.
Transferability. We then investigate the transferability of the improvements in interpretability to diverse datasets and downstream tasks. We visualize the SG for several images sourced from fine- grained classification and medical image datasets (Fig. 2 (c)), which are not included in the AFT phase and exhibit domain gaps with the training data. Moreover, their corresponding labels are absent from the ImageNet dataset. Nevertheless, the SG continues to exhibit meaningful patterns on

---

Published as a conference paper at ICLR 2025
orgiinal Image
tench
English springer
clip
clip WTAF
Rollout
M21B
Grad-ECLIP
Rollout
M42B
Grad-ECLIP
Figure 4: Left: Visualization of attention maps of VIT. AFT helps produce more fine-grainee fea- tures. Right: Visualization of saliency maps produced by Rollout, M2IB, and Grad-ECLIP. AFT consistently improve the quality of saliency maps.
Table 1:
Evaluation of localization ability using the Point Game (PG and PG-energy) and Segmen- tation test (Pix. Acc., AP and MaskloU) on the ImageNet-Seggentaaiee validation dataset.
Tasks
Zero-shot Classification
Lincar Probing
Saliency maps
PG-
Pixel Acc.1
mask- loUt
PG- energyt
Pixcl Acc.f
mask-
CLIP
PG
APt
PG1
AP
energyt
loUf
wlo AFT
25.77
31.32
65.51
34.45
2.16
27.75
32.62
65.45
35.22
2.51
SG
w/ AFT
71.15
53.98
66.67
59.85
4.31
65.79
51,20
66.53
56.52
4.06
w/o AFT
51.00
43.71
61.79
49.18
16.19
33,29
36.28
60.55
38.85
9.89
w/ AFT
73.98
63.76
67.44
62.34
19.39
33,59
37.02
59.20
40.33
10.62
w/o AFT
86.61
60.54
71.19
76.82
25.70
NA
NA
NA
NA
NA
Grad-ECLIP
w/ AFT
87.40
62.77
72.23
78.34
27.97
NA
NA
NA
NA
NA
images. We further conduct linear probing as a common downstream task of CLIP and visualize the SG for the resulting models. In Fig. (d), the SG maintains high quality. This shows that the improved visual interpretability can be transferred to downstream applications.
these
Feature Importance. To quantitatively analyze whether the saliency maps highlight the task related regions of the input image, we conduct Remove and Retrain (ROAR) analysis (Hooker et al.) 2019) on the saliency maps. ROAR reflects the drop in the predictive power of the dataset when increasing the proportion of pixels removed according their importance score in the saliency maps. A steeper decline in performance indicates that the model effectively captures task-related features from the inputs. We retrain networks with the top k% of the pixels removed from each image and record the accuracy on the test set. Each measurement is based on three trials with random initialization. We perform the analysis and out-of-din rit-of--ditribution datasets (CUB-200 (Wah et al.) 2011) and Caltech-256 (Griffin et al. (007)). ) We apply ROAR on saliency maps generated from both zero-shot prediction and liner probing. The results in Fig. 3 show an obvious pattern that the drop is more rapid when the CLIP is adversarially fine-tuned. It holds for both SG and GC, showing AFT enables CLIP focus on important features.
both in-distribution datasets (Imagenette, ten-class subset of ImageNet)
Improvements on Raw Attention Maps and Advanced Saliency Maps. We visualize the last attention maps of the VIT Fig. 4 which illustrate the attention score between the [cls] token and other image tokens. It is evident that after AFT, the attention maps show more fine-grained features, indicating that the model can better capture the details from the input image. We further showcase several saliency maps generated by more recent feature attribution methods, including techniques designed for transformer-based models such as Rollout (Abnar Zuidemal 2020), and state-of- the-art CLIP interpretation techniques like M2IB (Wang et al. (023) and Grad-ECLIP (Zhao al.) 2024]. With modern explanation techniques, we observe improved saliency maps even when using the original CLIP visual encoder. However, AFT consistently enhances the visual quality of the saliency maps, making the saliency maps sparser and better aligned with the object described in the text prompts. Therefore, the improvements of AFT are explanation-agnostic.
7

---

Published as a conference paper at ICLR 2025
RN50
ViT-B
ViT-L
ViT-B
ViT-B
0.8
0.8
0.0
supervised
supervised
supervised
clip wio AFT
0.7
Cip wlo AFT
clip wio AFT clip
0.7
0.6
AFT
0.6
similarity
0.5-
0.5
E (/255)
iterations
0.4
0.4
4kc
0.3
0.3
10k
0.2
0.2
0.1
20k
0.00
0
100
200
50
100
100
200
6.0
02
0.
0.0
0.7
04
count
count
count
similarity (d)
similarity (1)
(b)
(c)
(e)
Figure 5: (a-c) Comparison of neuron-concept alignment w/wo AFT via CLIP-dissect. (d,e) Effects of perturbation strength/unning iterations on neuron-concept alignment.
Pointing Game Finally, we assess the localization capabilities of the saliency maps using the (Gao et al. 2022) validation set, which includes segmentation annotations for 12,419 images across 919 categories from ImageNet. We use the Point Game (PG) as a standard metric to measure the accuracy of visual explanation localization. The PG metric assigns a hit score if the point with the highest value on the text-specific heat map falls within the object region, defined by the class segmentation mask. The PG accuracy is then calculated by averaging all sample scores. Additionally, we employ the energy-PG (Wang al., 2020). which measures the ratio of heat map energy within the ground truth mask to the entire map, providing better reflection of the heat map's s spread. Following Chefer et af. (201aabb) we treat the heat maps as soft-segmentation results and calculate pixel accuracy (Pixel Acc.), average precision (AP), and averaged mask intersection over union (maskloU) for evaluation. The results are shown in Table which cover evaluation for both SG, GC and one advanced saliency map technique, i.e., Grad-ECLIP. The evaluation is based on saliency maps generated by both zero-shot classification and linear probing. For most of the metrics, AFT brings improvement over the original model. For example, the PG of zero-shot classification improves by 45.38% and 22.98% for SG and GC respectively. The results show that AFT makes the saliency maps better localize the object with the correct category as the text.
4.2 INTERPRETABILITY QUANTIFICATION VIA NETWORK DISSECTION
To evaluate the interpretability of the visual encoder, we further conduct network dissection and eval- uate the quantity and quality of the concept detector within the visual encoder. The concept detector refers to the neuron that exclusively responds to specific concepts understandable to human beings. We apply CLIP-dissect (Oikarinen & Weng 2022) to discover the concept detectors. Specifically, use the the broadly and densely labeled (Broden) dataset (Bau et al., 2017) probing dataset, which covers comprehensive human- understandable concepts including objects, scenes, parts, tex- tures, materials, and colors. Then we have a concept set comprised of 20,000 most common English words. For each concept c, we calculate its similarity score with all the images within the probing dataset using pre-trained CLIP model, which results in similarity array S.. For specific neuron k. we calculate its activation value with each image in the probing dataset as input, which also gen- erates an array Ag. The alignment between neuron l and concept is measured by comparing the similarity between A& and S. The neuron k is called a concept detector of concept C₁ if C₁ has the largest alignment with neuron among all the concepts within the concept set. Moreover, the larger the similarity is, the better alignment is achieved between the neuron and the concept.
Improvements over Original CLIP. We first show that AFT can improve the interpretability of the CLIP visual encoder. To achieve this, we evaluate the alignment score of each neuron in the [cls] token of the last layer with its corresponding concept and visualize the scores via histogram. For comparison, we draw the plots for CLIP w/wo AFT, as well as for network trained using super- vised learning on the ImageNet dataset. Additionally, we conduct experiments on three different CLIP architectures: Re (He et al. 2016), VIT-B, and ViT-L (Dosovitskiy et al., 2020). The results are shown in Fig.5] The data reveals a clear trend indicating that CLIP exhibits superior align- ment with concepts compared to supervised learning, corroborating previous findings by (Goh al.)
8

---

Published as a conference paper at ICLR 2025
2021]. which demonstrated that neurons in CLIP align more effectively with specific words due to the model's 's objective of aligning image representations with text representations. Moreover, our proposed AFT can further improve the alignments. AFT mitigates the non-smoothness introduced by the false negative samples during contrastive learning, thereby promoting the alignment between image representations and their corresponding concepts. This trend be more pronounced as the complexity of the network architecture increases.
We then conduct Network Dissec- tion (Bau et al. 2017) on the penul- timate layer of ViT-B and visualize the activation map of certain 'cat' detector neuron towards several top activated probing images in Fig. [ The results also illustrate that the ac- tivation map of the model after AFT
Original Image
Activation Map
ciip
could be more object-centric.
Effects of Hyper-Parane Parametrrss We
clip WFT
further study the relationship be-
tween training hyper-parameters and the model interpretability. Specifi- cally, we study the influence of ad- versarial perturbation strength and fine-tuning iterations.
Figure 6: Comparison between activation maps of the same cat detector. The detector corresponds to the neuron with the highest IoU score with samples belonging to the cat concept.
According
to the results in Fig. 5 (d). the aligmment with the concept improves with stronger adversarial perturba-
tions. However, stops further improving when e becomes greater than 4/255. In Fig. (e), we find that the concept alignment can be improved with only small amount of tuning iterations, showing the efficiency the proposed AFT scheme.
4.3 EXTENDED EXPERIMENTS
Interpretability Improvements for Large Vision-Language Models. As the pre-trained CLIP is an integral component for many large vision-language models (e.g.. LLaVa (Liu et al., 2024)). )
we postulate the improvements of the CLIP visual interpretability can be propagated downstream models. To substantiate this, we visualize the attention maps of LLa Va, correspond- ing to various tokens from the generated sentences. Specifically, we integrate the attention weights of the LLM with those of the VIT, which produces a composite attention map over the input image. We compare the attention maps of the LLaVa equipped with the original CLIP visual encoder and the one after AFT. We show several results from the COCO dataset (Lin et al. (014) in Fig. 7 The at- tention maps with AFT show much cleaner saliency maps, prominently and sparsely highlighting the objects corresponding to the specific tokens. This indicates that interpretability improvements can indeed be transferred to vision-language models without compromising the quality of the generated sentences. The interpretation can also help explain some of the hallucinations. For example, in the third image of Fig. ? the interpretation map implies the model mistakes the black bag as a cat. In the Appendix A.4 we show quantitatively how AFT benefits the downstream VLMs' interpretability.
vision-languee
Trade-On with Zero-Shot Accuracy. Finally, we analyze the trade-off between interpretability and zero-shot accuracy. Following the same benchmark with Mao et al. (2022), we assess classification accuracy on the ImageNet test set and 15 zero-shot recognition tasks, ensuring no overlapping with the fine-tuning dataset. We report both clean accuracy and robust accuracy under the first two attacks of Auto.Attack (Croce & Hein 2020). i.c., APGD with cross- cros--nntropy and DLR loss. We use the ViT- L/14 version of the CLIP to be compatible with previous work (Schlarmann et al., 2024). The results are shown in Table (and the complete results are in Appendix (.4). The results show certain drop in clean accuracy. However, a minimal e like 1/255 can result in slight accuracy decrement while still substantially enhancing interpretability (see Appendix A.4). Beyond interpretability, AFT can also markedly improve the robustness of the CLIP model, which is another crucial factor to consider in high-stake applications. Furthermore, training set reveal a smaller accuracy drop for ImageNet compared to other datasets. This suggests that the trade-off can be further mitigated by AFT the CLIP with more diverse data.
experiments involving AFT on CLIP with the ImageNet
9

---

Published as a conference paper at ICLR 2025
Prompt
LLaVa output using original
LLaVa sutputusing CLIP AFT
people image are iting around ciiing enjoying
meal together They are gathered
family dinner, and there are
The people the image are siting around a daing table, enjoying a posing for
What people doing
multile P sins of food, cups, and utonsis on the mable. The
atmosphere appears to be warm and friendl, as the group of
they celebrating special
meal tngether,
suggests that smply enjoying others company. The table is filled with pitte food. indicattnng
people smiling and posing for picture The presence of couch
the barkground suggests that the diring living space.
larger
they are sharing meal.
The place could city or urban area, there multipe people walking around, food truck, and presence
The plcee could be city or town center, as there are muitiple people walking around, cars driving and Bus passing The monument significatt landmark popular tourist destination The overall abmosphere indicates that it is busting urban area where people gather, socialze, and engage various activites.
the The bustiing, commercial or publi space where people gather, eat, and travel The fact that there are many people in the area also indicates thatitis a popular of and lively location.
could place
bukk suggests
suggests
location might
buiding,
bed, Iicvision, The unmade, and there are various scattered around, including remote contal, sup, and bottle. The presence of and a window. The bed is covered with Stiped blanket, and there these items suggests that the mom bedroom, and the occupant is gal sitting on the bed. may be in the process of organizing tiying the space.
and chair. The
What inside the room
room,
with Ielevision top of dresser,
Comparison of interpretation maps for LL with the visual encoder replaced by CLIP visual encoder w/wo AFT. The attention maps correspond to the tokens highlighted with the same color as the border of the attention maps.
Figure 7:
Table 2:
Accuracy evaluation on image classification datasets of CLIP model. The superscripts on the upper right side show the value of (/255) during the adversarial fine-tuning.
Zero-hot Datesets
Vision Encoder
ImageNer
CIFARI
5TL STL.10 10
CIFARI00
Cans
Calledh
Flowas
DTD
EHOSAT
POVC
PCAM
ImagoNNr--
LonggeNNe
Average
Zeno-shot
CLIP
74.90
95.
99.31
71.08
37.94
83. 13.30
93.21
79.17
15.21
62.65
31.77
$2.00
87.86
59.59
72.95
dlan
75.92
54.47
70.11
AFT
74.60
89.71
98. 50
69.66
70. 15
85 03
91.09
70.34
50.00
24.74
27.57
50.02
85.45
59.58
67.06
APT'
70.88
78.47
96.28
63.36
84.73
86.97
42.93
22.35
$0.39
56.91
60.99
2/256
CLIP
0.00
0.00
0.00
0.00
0.00
0.00
0.00
0.00
0.00
0.00
0.00
0.00
0.00
0.10
0.00
AFT
47.36
61.70
90.80
37.40
25.50
73. 50
68.00
31.70
26.50
8.60
5.90
46.90
$7.40
38.70
44,12
53.68
38.00
30.70
30.20
28.80
62.10
43.10
47255
CLIIP
0.000
0.00
0.00
0.00
0.00
0.00
0.00
0.00
0.00
0.00
0.00
0.00
0..00
0.00
0.00
AFT
18.34
26.90
63.40
14.30
5.40
47.
30.50
6.90
12.50
1.50
0.50
19,70
27.00
23.10
21,46
35.28
36.40
75.70
21.20
65.00
13.00
17.60
11.30
50.20
41.00
31.60
33.14
LIMITATIONS AND CONCLUSION
In this work, we propose an unsupervised adversarial fine-tuning scheme for improving the visual
interpretability of the CLIP. We provide theoretical analysis to explain the underlying mechanisms driving these improvements. Through comprehensive quantitative and qualitative evaluations, we demonstrate the proposed method enables the CLIP visual encoder to focus more effectively salient input features, and the neurons exhibit improved alignment with human-understandable con- cepts. Moreover, the effects are both generalizable and transferable.
Our work is subject to certain limitations. We focus on fine-tuning with a single datasets, without investigating the impact of data quantity and data diversity on performance. Additionally, have not explored the effects of different minimax optimization algorithms. Our future work entails more comprehensive studies regarding these aspects.
10

---

Published as a conference paper at ICLR 2025
ACKNOWLEDGMENTS
The work Farzan Farnia is partially supported by grant from the Research Grants Council of the Hong Kong Special Administrative Region. China, Project 14209920, and is partially supported by CUHK Direct Research Grants with CUHK Project No. 4055164 and 4937054. This work was also supported in part by a grant from the Research Grants Council of the Hong Kong Special Administrative Region, China (Project No. T45-401/22-N), Finally, the authors would like to thank the anonymous reviewers for their insightful suggestions and feedback.
REFERENCES
Samira Abnar and Willem Zuidema. Quantifying attention flow in transformers. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pp. 4190-4197, 2020.
Yong Hyun Ahn, Hyeon Bae Kim, and Seong Tae Kim. Www: A unified framework for explaining what where and why of neural networks by interpretation of neuron concepts. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 10968-10977, 2024.
Anish Athalye, Nicholas Carlini, and David Wagner. Obfuscated gradients give false sense of curity: Circumventing defenses to adversarial examples. In International conference on machine learning, pp. 274-283. PMLR, 2018.
Nicholas Bai, Rahul A Iyer, Tuomas Oikarinen, and Tsui-War Weng. Describe-and-dissed In- terpreting neurons in vision networks with language models. arXiv preprint arXiv:2403.1377 137711, 2024.
David Bau, Bolei Zhou, Aditya Khosla, Aude Oliva, and Antonio Torralba. Network dissection: Quantifying interpretability of deep visual representations. Proceedings of IEEE conference computer vision and pattern recognition, pp.
2017.
Usha Bhalla, Alex Oesterling, Suraj Srinivas, Flavio P Calmon, and Himabindu Lakkaraju. Inter- preting clip with sparse linear concept embeddings (splice). arXiv preprint arXiv:2402.10376, 2024.
Prasad Chalasani, Jiefeng Chen, Amrita Roy Chowdhury, Xi Wu, and Somesh Jha. Concise expla- nations of neural networks using adversarial training. In International Conference Machine Learning, pp. 1383-1391. PMLR, 2020.
Aditya Chattopadhyay, Ryan Pilgrim, and Rene Vidal. Information maximization perspective of orthogonal matching pursuit with applications to explainable ai. Advances Neural Information Processing Systems, 36, 2024.
Hila Chefer, Shir Gur, and Lior Wolf. Generic attention-model explainability for interpreting bi- modal and encoder-decoder transformers. In Proceedings of the IEEE/CVF International Confer- ence on Computer Vision, pp. 397-406. 2021a.
Hila Chefer, Shir Gur, and Lior Wolf. Transformer interpretability beyond attention visualization. In Proceedings of the IEEE/CVF conference computer vision and pattern recognition, pp- 782-791, 2021b.
Chen Chen, Bowen Zhang, Liangliang Cao, Jiguang Shen, Tom Gunter, Albin Madappally Jose, Alexander Toshev, Jonathon Shlens, Ruoming Pang, and Yinfei Yang. Stair: learning sparse text and image representation in grounded tokens. arXiv preprint arXiv: 13081, 2023.
Mehdi Cherti, Romain Beaumont, Ross Wightman, Mitchell Wortsman, Gabriel Ilharco, Cade Gor- don, Christoph Schuhmann, Ludwig Schmidt, and Jenia Jitsev. Reproducible scaling laws for contrastive language-iman image learning. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 2818-2829, 2023.
Mircea Cimpoi. Subhransu Maji, Iasonas Kokkinos, Sammy Mohamed, and Andrea Vedaldi. De- scribing textures in the wild. In Proceedings of the IEÉE conference on computer vision and pattern recognition, pp. 3606-3613, 2014.
11

---

Published as a conference paper at ICLR 2025
Adam Coates, Andrew Ng. and Honglak Lee. An analysis of single-layer networks in unsupervised feature learning. In Proceedings of the fourteenth international conference on artificial intelli- gence and statistics, pp. 215-223..0 JMLR Workshop and Conference Proceedings, 2011.
Jeremy Cohen, Elan Rosenfeld, and Zico Kolter. Certified adversarial robustness via randomized smoothing. international conference on machine learning, pp. 1310-1320. PMLR, 2019.
Francesco Croce and Matthias Hein. Reliable evaluation of adversarial robustness with an ensemble of diverse parameter-free attacks. In International conference machine learning, pp. 2206- 2216. PMLR, 2020.
Timothée Darcet, Maxime Oquab, Julien Mairal, and Piotr Bojanowski. Vision transformers need registers. arXiv preprint arXiv:2309. 16588, 2023.
Jia Deng, Wei Dong, Richard Socher, Li-Jia Li, Kai Li, and Li Fei-Fei. Imagenet: A large-scabe scale hi- erarchical image database. In 2009 IEEE conference on computer vision and pattern recognition, pp. 248-255. leee, 2009.
Finale Doshi Velez and Been Kim. Towards rigorous science of interpretable machine learning- arXiv preprint arviv: 170208868, 2017.
Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xiaohua Zhai, Thomas Unterthiner, Mostafa Dehghani, Matthias Minderer, Georg Heigold, Sylvain Gelly, al. An image is worth 16x16 words: Transformers for image recognition at scale. arXiv preprint arXiv:2010. 11929, 2020.
Noam Eshed. Novelty detection and analysis in convolutional neural networks. 2020.
Christian Etmann, Sebastian Lunz, Peter Maass, and Carola-Bibian Schönlieb. On the con- nection between adversarial robustness and saliency map interpretability. arXiv preprint 1XXiv:905.004722 2019.
Li Fei-Fei, Rob Fergus, and Pietro Perona. Learning generative visual models from few training examples: An incremental bayesian approach tested on 101 object categories. In 2004 conference on computer vision and pattern recognition workshop, pp. 178-178. IEEE, 2004.
Thomas Fel, Ivan F Rodriguez Rodriguez, Drew Linsley, and Thomas Serre. Harmonizing the object recognition strategies of deep neural networks with humans. Advances in neural information processing systems, 35:9432-9446, 2022.
Yossi Gandelsman, Alexei A Efros, and Jacob Steinhardt. Interpreting clip's image representation via text-based decomposition. arXiv preprint (rXiv:2310.054)) 2023.
Yossi Gandelsman, Alexei A. Efros, and Jacob Steinhardt. Interpreting the second-order effects of neurons in clip, 2024.
Shanghua Gao, Zhong-Yo Yu Li, Ming-Hsuan Yang. Ming-Ming Cheng, Junwei Han, and Philip Torr. Large-scale unsupervised semantic segmentation. 2022.
Tianyu Gao, Xingeheng Yao, and Danqi Chen. Simese: Simple contrastive learning of sentence embeddings. arXiv preprint 2oXiv:2104.000 0882/, 2021.
Gabriel Goh, Nick Cammarata, Chelsea Voss, Shan Carter, Michael Petrov, Ludwig Schubert, Alec Radford, and Chris Olah. Multimodal neurons in artificial neural networks. Distill, 6(3):e30, 2021.
Shizhan Gong, Qi Dou, and Farzan Farnia. Structured gradient-based interpretations via norm- regularized adversarial training. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 11009-11018, 2024a.
Shizhan Gong, Jingwei Zhang, Qi Dou, and Farzan Farnia. A super-pixel-based approach to the stable interpretation neural networks. arXiv preprint arXiv:2412. 14509, 2024b.
Gregory Griffin, Alex Holub, Pietro Perona, et al. Caltech- object category dataset. Technical report, Technical Report 7694, Californa Institute of Technology Pasadena, 2007.
12

---

Published as a conference paper at ICLR 2025
Xiangming Gu, Xiaosen Zheng, Tianyu Pang, Chao Du, Qian Liu, Ye Wang, Jing Jiang, and Min Lin. Agent smith: A single image can jailbreak one million multimodal Ilm agents exponentially fast. arXiv preprint 2xX1:22402.08567, 2024.
Ziyu Guo, Renrui Zhang, Longtian Qiu, Xianzheng Ma, Xupeng Miao, Xuming He, and Bin Cui. Calip: Zero-shot enhancement of clip with parameter- free attention. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 37, pp. 746-754, 2023.
Hyeongrok Han, Siwon Kim, Hyun-Soo Choi, and Sungroh Yoon. On the impact of knowledge distillation for model interpretability. In International Conference on Machine Learning, pp-
12389-12410. PMLR, 2023.
Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recog- nition. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp- 770-778, 2016.
Patrick Helber, Benjamin Bischke, Andreas Dengel, and Damian Borth. Eurosat: A novel dataset and deep learning benchmark for land use and land cover classification. IEFE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, 12(7):2217-2226, 2019.
Dan Hendrycks, Steven Basart, Norman Mu, Saurav Kadavath, Frank Wang, Evan Dorundo, Rahul Desai, Tyler Zhu, Samyak Parajuli, Mike Guo, et al. The many faces of robustness: A criti- cal analysis of out-of-distribution generalization. Proceedings of the IEEE/CVF international conference on computer vision, pp.
2021.
Sara Hooker, Dumitru Erhan, Pieter-Jan Kindermans, and Been Kim. A benchmark for interpretabil- ity methods in deep neural networks. Advances neural information processing systems, 32, 2019.
Hsuan-An Hsia, Che-Hsien Lin, Bo-Han Kung, Jhao-Ting Chen, Daniel Stanley Tan, Jun-Cheng Chen, and Kai-Lung Hua. Clipeam: simple baseline for zero-shot text-guided object and action localization. In ICASSP 2022-2022 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pp. 4453 4457. IEEE, 2022.
Shih- Shih-Cheng Huang, Liyue Shen, Matthew P Lungren, and Serena Yeung. Gloria: A multimodal global-local representation learning framework for label-efficient medical image recognition. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 3942-3951, 2021.
Niall Hurley and Scott Rickard. Comparing measures of sparsity. IEEE Transactions on Information Theory, 55(10))4723-4441, 2009.
Alistair EW Johnson, Tom J Pollard, Seth J Berkowitz, Nathaniel R Greenbaum, Matthew Lun- gren, Chih-ying Deng. Roger G Mark, and Steven Horng. Mimic-cxr, de-identified publicly available database of chest radiographs with free-text reports. Scientific data, 6(1):317, 2019.
Neha Kalibhat, Shweta Bhardwaj, C Bayan Bruss, Hamed Firooz, Maziar Sanjabi, and Soheil Feizi. Identifying interpretable subspaces in image representations. In International Conference Machine Learning, pp. 15623-15638, 15638. PMLR, 2023.
Zulqarnain Q Khan, Davin Hill, Aria Masoomi, Joshua Bone, and Jennifer Dy. Analyzing ex- plainer robustness via probabilistic lipschitzness of prediction functions. In International Confer- ence on Artificial Intelligence and Statistics, pp. 1378-1386. PMLR, 2024.
Beomsu Kim, Junghoon Seo, and Taegyun Jeon. Bridging adversarial robustness and gradient inter- pretability. arXiv preprint arXiv: 1903. 1626, 2019.
Chanwoo Kim, Soham U Gadgil, Alex J DeGrave, Jesutofunmi A Omiye, Zhuo Ran Cai, Roxana Daneshjou, and Su-In Lee. Transparent medical image ai via an image-text text foundation model grounded in medical literature. Nature Medicine, pp. 1-12, 2024.
Sunnie SY Kim, Nicole Meister, Vikram V Ramaswamy, Ruth Fong, and Olga Russakovsky. Hive: Evaluating the human interpretability of visual explanations. In European Conference on Com- pufer Vision, pp. 280-298. Springer, 2022.
13

---

Published as a conference paper at ICLR 2025
Jonathan Krause, Michael Stark, Jia Deng, and Li Fei-Fei. 3d object representations for fine- grained categorization. In Proceedings of the IEEE international conference on computer vision work- shops, pp. 554-561, 2013.
Alex Krizhevsky, Geoffrey Hinton, et al. Learning multiple layers of features from tiny images. 2009.
Nicholas Kurtansky, Veronica Rotemberg, Maura Gillis, Kivane Kose, Walter Reade, and Ash- ley Chow. Isic 2024 skin cancer detection with 3d-tbp. httts:://kaggl/ com/
2024. Kaggle.
Zinoviy Landsman, Steven Vanduffel, and Jing Yao. A note on stein's lemma for multivariate ellip- tical distributions. Journal of Statistical Planning and Inference, 143(1 (3311))201-2022, 2013.
Jeffrey Li, Vaishnavh Nagarajan, Gregory Plumb, and Ameet Talwalkar, A learning theoretic per- spective on local explainability. arXiv preprint arXiv:201 1.01200, 2020.
Yi Li, Hualiang Wang, Yiqun Duan, Hang Xu, and Xiaomeng Li. Exploring visual interpretability for contrastive language-image pre-training. arXiv preprint wrXiv:2209.07046, 2022.
Yi Li, Hualiang Wang, Yiqun Duan, and Xiaomeng Li. Clip surgery for better explainability with enhancement in open-vocabulary tasks. arXiv preprint arXiv:2304.05565, 2023.
Tsung-Yi Yi Lin, Michael Maire, Serge Belongie, James Hays, Pietro Perona, Deva Ramanan, Piotr Dollár, and C Lawrence Zitnick. Microsoft coco: Common objects in context. In Computer Vision-ECCV 2014: 13th European Conference, Zurich, Switzerland, September 6-12, 2014, Proceedings, Part V 13, pp. 740-755. Springer, 2014.
Yuqi Lin, Minghao Chen, Wenxiao Wang, Boxi Wu, Ke Li, Binbin Lin, Haifeng Liu, and Xiaofei He. Clip is also an efficient segmenter: A text-driven approach for weakly supervised semantic segmentation.
Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 15305- 15314, 2023.
Haotian Liu, Chunyuan Li, Qingyang Wu, and Yong Jae Lee. Visual instruction tuning. Advances in neural information processing systems, 36, 2024.
Jiaxiang Liu,, Tianxiang Hu, Yan Zhang, Xiaotang Gai, Yang Feng, and Zuozhu Liu. A chatgpt aided explainable framework for zero-shot medical image diagnosis. arXiv preprint arXiv:2307.01981. 2023.
Ilya Loshchilov, Frank Hutter, al. Fixing weight decay regularization in adam. arXiv preprint arXiv:1711.05100 5, 2017.
Scott M Lundberg and Su-In Lee. A unified approach interpreting model predictions. Advances in neural information processing systems, 30, 2017.
Aleksander Madry, Aleksandar Makelov, Ludwig Schmidt, Dimitris Tsipras, and Adrian Vladu. Towards deep learning models resistant to adversarial attacks. arXiv preprint 2017.
Subhransu Maji, Esa Rahtu, Juho Kannala, Matthew Blaschko, and Andrea Vedaldi. Fine-grained visual classification of aircraft. arXiv preprint arXiv: 2013.
Chengzhi Mao, Scott Geng, Junfeng Yang, Xin Wang, and Carl Vondrick. Understanding zero-shot adversarial robustness for large-scale models. arXiv preprint arXiv:2212.07016, 2022.
Matthias Minderer, Alexey Gritsenko, and Neil Houlsby. Scaling open-vocabulary object detection. Advances in Neural Information Processing Systems, 36, 2024.
Mazda Moayeri, Keivan Rezaei, Maziar Sanjabi, and Soheil Feizi. Text-to-concept (and back) via cross-model alignment. In International Conference on Machine Learning, pp. 25037-25060. PMLR, 2023.
14

---

Published as a conference paper at ICLR 2025
Sabine Muzellee, FEL Thomas, Victor Boutin, Léo Andéol, Rufin VanRullen, and Thomas Serre. Saliency strikes back: How filtering out high frequencies improves white-bon box explanations. In Forty-first International Conference Machine Learning. 2024.
Aaria-Elena Nilsback and Andrew Zisserman. Automated flower classification large number of classes. In 2008 Sixth Indian conference on computer vision, graphics & image processing, pp. 722-729. IEBE, 2008.
Zachary Novack, Julian McAuley, Zachary Chase Lipton, and Saurabh Garg, Chils: Zero-shot image classification with hierarchical label sets. In International Conference on Machine Learning, pp. 26342 263622. PMLR, 2023.
Tuomas Oikarinen and Tsui-Wei Wei Weng. Clip-dissect: Automatic description of neuron representa- tions in deep vision networks. arXiv preprint arXiv:2204. 10965,, 2022.
Tuomas Oikarinen, Subhro Das, Lam M Nguyen, and Tsui-Wei Weng. Label-free concept bottle- neck models. arXiv preprint arXiv:2304. 06129, 2023.
Omkar M Parkhi, Andrea Vedaldi, Andrew Zisserman, and CV Jawahar. Cats and dogs. 2012 IEEE conference computer vision and pallern recognition, pp. 3498-3505. IEEE, 2012.
Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et al. Learning transferable visual models from natural language supervision. In International conference machine learning, pp- 8748-8763. PMLR, 2021.
Aditya Ramesh, Mikhail Pavlov, Gabriel Goh, Scott Gray, Chelsea Voss, Alec Radford, Mark Chen, and Ilya Sutskever. Zero-shot texi-to-image generation. In International conference on machine learning, pp. 8821-8831. Pmlr, 2021.
Jie Ren, Mingjie Li, Qirui Chen, Huiqi Deng, and Quanshi Zhang. Defining and quantifying the emergence of sparse concepts in dnns. Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp. 20280-20289 2023.
Marco Tulio Ribeiro, Sameer Singh, and Carlos Guestrin. why should i trust you?" explaining the predictions of any classifier. In Proceedings of the 22nd ACM SIGKDD international conference on knowledge discovery and data mining, pp. 1135-1144, 2016.
Andrew Ross and Finale Vesli-Velez. Improving the adversarial robustness and interpretability of deep neural networks by regularizing their input gradients. In Proceedings of the AAAI conference artificial intelligence, volume 32, 2018.
Oindrila Saha, Grant Van Horn, and Subhransu Maji. Improved zero-shot classification by adapting vims with text descriptions. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 17542-17552, 2024.
Christian Schlarmann, Naman Deep Singh, Francesco Croce, and Matthias Hein. Robust clip: Un- supervised adversarial fine-tuning of vision embeddings for robust large vision-language models. arXiv preprint arXiv:2402.12335 2024.
Ramprasaath R Selvaraju, Michael Cogswell, Abhishek Das, Ramakrishna Vedantam, Devi Parikh, and Dhruv Batra. Grad-cam: Visual explanations from deep networks via gradient- based local- ization. In Proceedings of the IEEE international conference on computer vision, pp. 618 618-626, 2017.
Harshay Shah, Prateck Jain, and Praneeth Netrapalli. Do input gradients highlight discriminative features? Advances in Neural Information Processing Systems, 34:2046-2059, 2021.
Avanti Shrikumar, Peyton Greenside, and Anshul Kundaje. Learning important features through propagating activation differences. In International conference machine learning, pp. 3145- 3153. PMIR, 2017.
Karen Simonyan, Andrea Vedaldi, and Andrew Zisserman. Deep inside convolutional networks: Vi- sualising image classification models and saliency maps. arXiv preprint
2013.
15

---

Published as a conference paper at ICLR 2025
Aman Sinha, Hongseok Namkoong, Riccardo Volpi, and John Duchi. Certifying some distributional robustness with principled adversarial training. arXiv preprint drXiv:1710.10571, 2017.
Daniel Smilkov, Nikhil Thorat, Been Kim, Fernanda Viégas, and Martin Wattenberg. Smoothgrad: removing noise by adding noise. arXiv preprint arXiv:1706.03880 2017.
Mukund Sundararajan, Ankur Taly, and Qiqi Yan. Axiomatic attribution for deep networks. In International conference on machine learning, pp. 3319-3328 PMLR, 2017.
Christian Szegedy, Wojciech Zaremba, Ilya Sutskever, Joan Bruna, Dumitru Erhan, Ian Goodfellow, and Rob Fergus. Intriguing properties of neural networks. arXiv preprint 1312:66112.6199 2013.
Bastiaan S Veeling, Jasper Linmans, Jim Winkens, Taco Cohen, and Max Welling. Rotation cquivariant cnns for digital pathology. In Medical Image Computing and Computer Assisted Intervention-MICCAl 2018: 21st International Conference, Granada, Spain, September 16-20, 2018, Proceedings, Part II 11, pp. 210-218. Springer, 2018.
Catherine Wah, Steve Branson, Peter Welinder, Pictro Perona, and Serge Belongie. The caltech-uesd birds-200-201 dataset. 2011.
Haofan Wang, Zifan Wang, Mengnan Du, Fan Yang, Zijian Zhang, Sirui Ding, Piotr Mardziel, and Xia Hu. Score-cam: Score-weighted visual explanations for convolutional neural networks. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition workshops, pp. 24-25, 2020.
Haohan Wang, Songwei Ge, Zachary Lipton, and Eric P Xing. Learning robust global representa- tions by penalizing local predictive power, Advances in Neural Information Processing Systems, 32, 2019.
Sibo Wang, Jie Zhang, Zheng Yuan, and Shiguang Shan. Pre-trained model guided fine-tuning for zero-shot adversarial robustness. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 14502-24511) 2024a.
Ying Wang. Tim GJ Rudner, and Andrew G Wilson. Visual explanations of image-text representa- tions via multi-modal information bottleneck attribution. Advances in Neural Information Pro- cessing Systems, 36:16009-1600 2023.
Zeyu Wang, Xianhang Li, Hongru Zhu, and Cihang Xie. Revisiting adversarial training at scale. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp- 24675-24685, 2024b.
Zifan Wang, Matt Fredrikson, and Anupam Datta. Robust models more interpretable because attributions look normal. arXiv preprint arXiv:2103.11257, 2021.
Xing Wu, Chaochen Gao, Yipeng Su, Jizhong Han, Zhongyuan Wang, and Songlin Hu. Smoothed contrastive learning for unsupervised sentence embedding. arXiv preprint 1rXv::2099.0421,, 2021.
Rui Xu. Wenkang Qin, Peixiang Huang, Hao Wang, and Lin Luo. Scaat: Improving neural network interpretability via saliency constrained adaptive adversarial training. arXiv preprint arXiv:2311.05144 2023a.
Zhenhua Xu, Yujia Zhang, Enze Xie, Zhen Zhao, Yong Guo, Kenneth KY Wong, Zhenguo Li, and Hengshuang Zhao. Drivegpt4: Interpretable end-to-end autonomous driving via large language model. arXiv preprint arXiv:2310.01412, 2023b.
Yue Yang, Artemis Panagopoulou, Shenghao Zhou, Daniel Jin, Chris Callison-Burch, and Mark Yatskar. Language in a bottle: Language model guided concept bottlenecks for interpretable im- age classification. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 19187-19197, 2023.
Zhuorui Ye and Farzan Farnia. On the trade-off between stability and fidelity of gaussian- smoothed saliency maps. arXiv preprint (tt::2211.0083777 2024.
16

---

Published as a conference paper at ICLR 2025
Rumpeng Yu, Weihao Yu, and Xinchao Wang. Attention prompting image for large vision- language models. arXiv preprint 10X1:2249..71143, 2024.
Tian Yun, Usha Bhalla, Ellie Pavlick, and Chen Sun. Do vision-language pretrained models learn composable primitive concepts? arXiv preprint 1.Xx::2203.172711 2022.
Hongyang Zhang, Yaodong Yu, Jiantao Jiao, Eric Xing, Laurent EI Ghaoui, and Michael Jordan. Theoretically principled trade-off between robustness and accuracy. In International conference on machine learning, pp. 7472-7482. PMLR, 2019.
Jingwei Zhang and Farzan Farnia. Morcaugrad: Sparse and robust interpretation of neural networks via moreau envelope. In Proceedings the IEEE/CVF International Conference Computer Vision, pp. 2021-2030, 2023.
Chenyang Zhao, Kun Wang, Xingyu Zeng. Rui Zhao, and Antoni B Chan. Gradient-based visual explanation for transformer-base clip. In International Conference on Machine Learning, pp- 61072-61091. PMLR, 2024.
17

---

Published as a conference paper at ICLR 2025
A APPENDIX
In this Appendix, we provide proof for the theorems, implementation details, additional experimen- tal results, and discussion on the potential application of more interpretable CLIP.
A.1 PROOFS
Proof of Theorem To prove this theorem, we apply the multivariate version of Stein's lemma by Landsman et al. 2013). which shows that for bounded function 9 Rd |gl S
M and Gaussian noise
M -mooth.. Therefore, .NO0, o²T), Ex- + 2)] is o² 1 -mmoohh,, and (TT E2n (fo T Ix)² 20 -smooh...
Ex- [ fox [(xxx=)]] According to the properties of smooth function, we have:
mx (0) + max oT Wx VxEx~ fo z)] (h(6) + 402
[18112) I max h(6)
5
S (0) + max 8" wx V fo z)] - (h(0) 402 (1813)
(8)
We reparameterize the obj as:
max 5 Wx V xx (n fo + 2)] (h(6) + E11813),
(9)
where
5 SES 5 462 By definition, we have: 102
(x) == arg max 8t Wx VxEe--(00 [T fo(x + z)] - (A(0) + =1001?);
(10)
(x) == arg max 8t we EZ I fo + 2)] M(8).
(11)
As (x) and 8āg (x) are stationary points of the corresponding objectives, we have:
Ex TT fo + 2 Voh(ot,(x)) 25(x) = O,
(12)
Ba-N(o. TT fo(x + z)] - Vzh(655 = O.
(13)
Therefore, we have:
(xh(fio(x)) - Vsh(655 2
(14)
Moreover, since h(-) is pestronglyycoonexx we further have:
(Vgh(ōie (x)) Voh(O2e (8)o)))))))))) (x)) 2 pu p)l0io(xx) 820 (x)]1
(15)
Meanwhile, according the Cauchy-Schward inequality, have:
(Vsh(6io(x)) (x) (x))
(16)
(sh(8io(x)) (x)( 550
Combining 16 and 15
We have:
(x) 550 (x)ll (Vhh(fi((x) Vah(550(x))1 = 2|111151o (x)
(17)
So finally have:
< 2 <
5
[8)0(x) 85 (x)
(18)
/ 2
This completes the proof.
Proof of Observation
By Cauchy- Swchhhy--Shwarrhhh inequality, we have:
IT Eee-Wi0.-=-)/fexx 2)] T 1x1 < [| [TLlll/Be..rro.co///ll+++ - -Ixll
(19)
Since T2 is unit-length representation, we have:
T Ta z)] T Txl [[Fxx)ooo22((((((x+)))) - 1xll S X.
(20)
This completes the proof.
18

---

Published as a conference paper at ICLR 2025
A.2 CLARIFICATION ON THE GAUSSIAN SMOOTHING
The theoretical derivation involves Gaussian Smoothed term. The Gaussian smoothing ensures that the sum of the Gaussian smoothed loss and the negative of strongly-convex regularization penalty (dual norm to Huber loss) will be a concave function with unique maximizer and therefore guar- antees the convergence of gradient updates in solving the inner maximization. Although the outer minimization task over CLIP weights still remains a challenging non-convex optimization, through using the Gaussian smoothed loss function and the dual function to Huber loss, we are able to solve the inner optimization problem and converge to solution of the primal optimization problem with the Huber loss penalty, In the experiments, We draw the Gaussian noise vectors independently for each training sample at every PGD iteration.
A.3 IMPLEMENTATION DETAILS
Adversarial Training setup. All the models in the paper are trained on ImageNet (at reso- lution 224x224) for two epochs using 10 steps of PGD, with step size set to 1/255. We use AdamW (Loshchilov et al. 2017) optimizer with momenta coefficients B1 and B3 to be 0.9 and 0.95 respectively. The training was done with a cosine decaying learning rate schedule with a linear warm-up and peak learning rate be le-5. The weight decay is set to le-4 and the batch size is 128 for RN50 and ViT-B and 64 for ViT-L respectively. We sample only I image per interaction from the Gaussian distribution as a simplified implementation of Gaussian smoothing.
Zero-Shot Evaluations. We use the CLIP Benchmark and OpenCLiF Cherti et al. 2023 pro-
tocol to evaluate the zero-shot performance. The evaluation datasets include: CalTech 101 (Fei-Fa Fei
et al.
2004). StanfordCars (Krause et al., 2013), CIFAR10, CIFAR 100 (Krizhevsky et al. 2009),
DTD
Cimpoi et all.|2214]] EuroSAT (Helber et al. 2019) FGVC Aircrafts (Maji et 22013
Flow-
ers
(Nilsback & Zisserman 22008), ImageNet-R (Hendrycks et al.. 2021), ImageNet-SSetch (Wang
et al.
2019). PCAM (Veeling et al. 2018), OxfordPets (Parkhi et al., (2012), and STL-10 (Coates
ct al.
201T).W We also test performance on the validation set of ImageNet (Deng et al., 2009).
We evaluate robustness on 1000 samples each and report clean accuracy for all samples of the respec- tive dataset. The attacks applied are the first two attacks of AutoAttack (Croce & Hein 2020). i.e., APGD with cross-entropy loss and APGD with targeted DLR loss (100 iterations each). We consider Lo boundde threat models with radii 2/255 and 4/55 and evaluate robustness on all datasets.
VLM interpretation.
The visualization in Fig. i implemented by Vis--isualizer
A.4 ADDITIONAL RESULTS
Quantitative Evaluation on VLMSs Interpretations. In this section, we provide quantitative assessment of how AFT of CLIP enhances the visual interpretability of downstream VLMs. Since there are currently no established metrics for evaluating VLM interpretations, we propose a straight- forward evaluation scheme, as illustrated in Fig. g We selected test images from the Imagenette dataset, which consists of 10 easily classified categories from ImageNet. We then prompted LLa Va with the following request:
Please select one class from tench, English springer, cassette player, chain saw, church, French horn, garbage truck, gas pump, golf bull, parachute to describe the image. Only output the class name.
Typically, LLaVa responds with just the class name, such as "Church," which is generally made up of to 3 tokens. Including the initial colon and the end-of-sentence tokens, we obtain a few tokens that correlate with several attention maps. We compute various metrics for each attention map using the pointing game metric, which encompasses PG, PG-energy, pixel accuracy (Pixel Acc.), average precision (AP), and average mask intersection over union (maskloU). Additionally, caleulate the Gini Index (Hurley & Rickard 2009) to measure the sparsity of the attention map. For each case, we take the highest metric value across different tokens. Since the pointing game
hatess///ikhuus.cc///100-//////0 http://cIII.becchaaraaaaa.a benchnazk
zer
19

---

Published as a conference paper at ICLR 2025
Please select one class from tench, English springer, cassette player, chain saw, church, French horn, garbage truck, pump, galf ball, parachute to describe the image. Only output the class name.
Church
THE
Input image
"Church"
<</3>"
Figure 8: Illustration on how we utilize LLaVa to perform image classification task and generate corresponding attention map for each response token.
Table 3: Quantitative evaluation of attention maps generated by LLaVa. We measure the Gini Index (Gini), Point Game (PG and PG-energy) and Segmentation test (AP and MaskloU) on the interaction between Imagenette and ImageNet-SSegmnnattie validation dataset.
Encoder
Ginit
PG1
PG-energy!
Pixel Acc.
AP1
mask-loU1
CLIP
25.29
5.56
19.71
69.62
28.77
4.13
CLIP w/ AFT 29.16
7.64
21.82
72.14
33.30
4.78
requires a segmentation mask, we limited our analysis to images that overlap between Imagenette and SrageNett-Segmenatioon resulting in a total of 144 images.
We compare the outcomes of LLaVa using the CLIP encoder against those using the CLIP encoder with AFT. The results, presented in Table show that employing the CLIP encoder with AFT leads to sparser and more interpretable attention maps generated by LLaVa. However, it is important to note that the current metrics have limitations. For instance. VIT attention maps frequently contain numerous high-norm tokens in low-informative background areas, a well-documented issue (Darcet al., 2023) that AFT does not fully mitigate. These high-norm tokens significantly impact the proposed metrics. We view this work as a preliminary exploration and aim to identify better metrics for a more rigorous quantitative evaluation.
Comparison between Supervised AFT and Unsupervised AFT. To empirically examine the differences between optimizing [q [ and 7 we conducted experiments comparing supervised and unsupervised fine-tuning. We applied AFT train the models using data from the COCO 2017 dataset. For supervised AFT, we utilized image captions as the Tgin Eq. [ In contrast, for unsupervised AFT, we maintained the same configuration outlined in the main text, using CLIP ViT-L-14 as the backbone. For both models, we assessed: 1) The visual quality of the saliency maps (Fig. g. 2) Quantitative evaluations via pointing games (Table. 4. and 3) Zero-shot accuracy (Table. 5 The results indicate that supervised AFT provides superior interpretabilityy particularly for Simple Gradients. This is expected, as training is steered by text descriptions, allowing the network to effectively create sparse saliency maps that align with the image content. However, the generalizability of models after supervised AFT heavily dependent on the diversity and quality of the text descriptions. With limited training data, supervised AFT significantly impair the model's 's zero-shot capabilities. Conversely, unsupervised AFT tends to achieve a better balance between interpretability and generalizability, even when trained smaller datasets.
Extension to Image Encoders Other than CLIP. As noted in the main text, the proposed AFT method is versatile and can be applied to any multi-modal models that utilize embeddings to connect different modalities. It can also be extended to representation learning within single modalities. To demonstrate this flexibility, we conducted experiments with three models other than CLIP: 1) ViT-B trained by supervised learning Imagenet, 2) MONET (Kim et al. 2024). which is image-text foundation model trained on 105,550 dermatological images paired with natural language descrip- tions from large collection of medical literature. It utilizes ViT-I--14 as its backbone and is trained
20

---

Published as a conference paper at ICLR 2025
CLIP
Uncupervised Supervised
AFT
AFT
gas pump
garbage truck
cassette
church
French
tench
English springer
chain saw
golf ball
parachute
player
Figure 9: Comparison between supervised AFT and unsupervised AFT. The models use ViT-L- 14 as the backbone and are trained on MS COCO 2017 dataset (Lin et al. 2014). We visualize saliency maps generated by both simple gradient (left) and GradCam (right).
Table 4: Evaluation of localization ability between supervised AFT and unsupervised AFT using the Point Game (PG and PG-energy) and Segmentation test (AP and MaskloU). Evaluation performed on Simple Gradients, Grad-Cam, and Grad-ECLIP.
Saliency maps
CLIP
PGf
PG-energy!
APt
mask-loUt
unsupervised
28.88
31.90
38.14
2.20
Simple Gradients
supervised
49.10
41.09
47.54
5.32
unsupervised
60.41
59.02
59.66
16.49
Grad-Cam
supervised
58.02
57.86
59.02
15.79
unsupervised 86.44
61.75
75.67
Grad-ECLIP
supervised
87.55
60.40
77.24
25.46
with contrastive learning, 3) GLoRIA (Huang et al.) 2021). an attention based model designed to learn both global and local representations by contrasting sub-regions of chest X-ray images with corresponding words in paired reports. It utilizes ResNet-50 as its backbone. For simplicity, focused on applying AFT to the global representation of GLoRIA. We perform AFT these models with the training data from 1) Imagenet, 2) The ISIC 2024 Challenge Dataset (Kurtansky et al., 2024). and 3) MIMIC-CXR (Johnson et al. 2019) respectively. All other settings remained consis- tent with those outlined in the main text. The explanation results, including both simple gradients and Brad,CAM, are presented in Fig. TO The findings indicate that AFT significantly enhances the quality of saliency maps across all three models, resulting in much sparser highlighted regions compared to the original models. This improvement leads to a more focused attention on lesions or abnormal areas. Notably, the benefits of AFT are more pronounced for vision transformers than for CNNs. Additionally, since MONET and GLoRIA are tailored for healthcare applications, these results underscore the potential of our method support decision- maning high-stakes scenarios.
Comparison with FARE (Schlarmann et al., (024).. FARE a recent work that also study adver- sarial training the context of multi-modal models. While FARE aims at improving the robustness of the CLIP model to adversarial perturbations, our proposed method's primary goal is to struc- ture the saliency maps and thus improve the CLIP's interpretability. Therefore, while both these methods result in AT-based min-max optimization problems, their different goals lead to different loss functions, where in FARE there exist norm hard constraints perturbations to address norm-bounded adversarial attacks, and our proposed AFT method, the loss function is piecewise quadratic which is the Fenchel dual to the Huber loss in our original problem formulation.
To further illustrate, we conduct experiments to compare their effects on the visual interpretabilityy of CLIP. Specifically, we compare the ROAR on Imagenette and
(Fig. (T) and pointing
21

---

Published as a conference paper at ICLR 2025
Table Zero-shot accuracy evaluation on image classification datasets of CLIP model trained with supervised AFT (Sup.) and unupervised AFT (Un.).
AFT
InsgoNNr
CIFARIO
15T.
CIFARTOR
Cars
CalReh
OxfordPers
Flowers
DED
FOVE
PCAM
ImageNNe-R
ImnggeNesSS
Average Zero-sho
Un.
56.76
73.93
42.96
59.98
81.77
86.24
56.71
14,50
24.63
50.19
73.75
48.25
57.54
Sup.
45.58
52.85
81.81
26.58
54.21
63.01
82.69
54.17
33.67
23.04
18.96
51.85
57.72
38.30
48.89
orgginal
Irrege
VIT-B
VIT. wi
AFT
tench
cassette player
church
French horn
garbage
English springer
chain saw
gas pump
golf ball
parachute
original Imesge
MONET
MONET
AFT
dermato- fibroma
keratosis-like
benign
actinic keratoses
basal cell carcinoma
melanocytic
actinic keratoses
vascular lesions
basal carcinoma
dermato- fibroma
melanocytic nevi
original Image
GLORIA
GLORIA w/
AFT
lung opacity
pneumonia
ploural effusion
edema
support device
pneumothorox cardio- atelectasis cardiomegaly fracture mediastinum
enlarged
Figure 10: Extension of AFT to other image encoders. We visualize both simple gradient and
GradCam. Top: Experimental results on supervised VTT-B. Middle: Experimental results on
MONET Kim al. 2024). Bottom: Experimental results GLoRIA (Huang et al. 2021).
game on ImageNet-Segnee Segmentation validation set (Table 6 between two methods. The numerical results suggest that both algorithms improve the interpretability of the CLIP model by boosting the sparsity of saliency maps. However, there a relative improvement of the interpretability of our proposed AFT over FARE.
22

---

an obvious Published as a conference paper at ICLR 2025
Imagenette
CUB-200-2011
accuracy (%)
(FARE)
(FARE)
6C (ours)
10
percentage (%) pixels masked
percentage (%) of pixels masked
Figure 11: Comparison of Remove and Retrain (ROAR) on zero-shot classification between FARE and our method. A more rapid drop accuracy shows the highlighted features are more important. Our algorithm design enables CLIP to catch more informative features compared with FARE.
Table 6: Comparison of localization ability between FARE and our proposed AFT. We experi- ment with the Point Game (PG and PG-energy) and Segmentation test (AP and MaskIoU) the ImageNet-Segmennation validation dataset.
Salieneymaps
CLIP
PG
PG-energy'
APt
mask-loUt
FARE
30.06
31.68
38.11
2.15
SG
33.19
33.94
39.78
2.21
FARE
70.05
64.75
66.11
21.50
GC
ours
74.31
65.47
67.23
22.46
FARE
5.56
21.47
32.11
4.39
LLaVa attention
7.64
21.82
33.30
4.78
Results of Different Network Architectures. In Fig. 12 we showcase several simple gradient maps corresponding different network architectures, including ResNet-50, ViT-B, and ViT-L. We can see the trend as the network becomes larger and more complex, the saliency maps also become more difficult to understand. On the other hand, the proposed AFT can consistently improve the quality of the saliency maps, making the simple gradient more human-understandable
Results of Different Regularization. We visualize saliency maps trained on AFT with differ- functions regularization. We replace h²)-) in Eq. 4 with different functions, including the smoothed version of the elastic net:
h" (u) = =1 Σ (n( + u 0, e2 > 0,
(21)
smoothed version of goru:
h (r) = Σ (1 12), 0,
(22)
where Sj. St S (1.... .d are disjoint variable subsets. The visualization results are shown in Fig. [3 L1 norm and elastic nets have similar effects on the Simple Gradient maps, while group- norm regularization makes the saliency map more compact and connected.
Effects of Regularization Strength. We change the regularization strength and visualize the re- sulting simple gradient maps in Fig. 14 Even with a very small such as 1/255, the visual quality of the Simple Gradient maps has significant improvement, which will cause very small drops of zero-shot accuracy shown in Table 2 the main text. As increase the we pattern that the saliency maps become more and more concise.
Effects of Training iterations. We can show how the Simple Gradient maps evolve during the fine- tuning process. We showcase the simple gradient with different tinettuning iterations in Fig. [5 The Simple Gradient becomes more concise and cleaner the fine-tuning processing. However, with few menettnnng steps, the resulting saliency map can already be of good visual quality. This shows method is computationally efficient.
23

---

Published as a conference paper at ICLR 2025
RN50
Original image
clip
Clip w/ AFT
bed
people
bear
infant
cat
dog
cupcake
apple
VIT-B
Original image
clip
AFT
Clip w
SANNMN cat
apple
flower
rose
scissor
food
scissor
keyboard
VIT-L
Original image
dii
Clip w AFT
banana
cutting board
banana
book
human
cat
broccoli
wine
Comparison of Simple Gradient maps w/wo adversarial fine-tuning among different network architectures.
Figure 12:
Interpretation Robustness. We further study the robustness of the saliency maps before and after AFT. As the SG the original CLIP shows an extremely noisy pattern, we focus on GC this study. We inject small random Gaussian noise the input, which is imperceptible to humans. Then
24

---

Published as a conference paper at ICLR 2025
clip Original
image
elastic net 1, nnom
group norm
Figure 13: Comparison of Simple Gradient maps w/wo adversarial fine-tuning subject to different regularizations.
Table 7: SSIM (%) between Grad-Cam maps before and after injecting Gaussian noise with SD of a to the input.
1/255
3/255
5/255
7/255
9/255
CLIP
91.18
82.98
77.56
73.62
70.58
CLIP W/ AFT
99.99
99.96
99.79
99.37
98.55
measure the similarity of the GC with the clean GC, represented by SSIM. The results in Table 7 reflect that the GC of the original CLIP is susceptible to random noise, with tiny noise causing great changes. On the other hand, AFT makes the interpretation more robust to perturbations, with significant improvement of SSIM over the non-AFT counterpart.
Inter-class Similarity. As posited by Han et al. (2023). ) there exists a correlation between the model interpretability and class-similarity information. An interpretable model would possess greater understanding of inter-class similarity, thereby generating more similar predictions on im- ages belonging to different but related classes. To assess whether this finding extends AFT, we divided the 1,000 classes of the ImageNet dataset into 66 categories based on the coarse classifi- cation scheme proposed by [shhed (2020), excluding the 'other' category. We predict the category logit with the visual encoder via linear probing and quantify the entropy of classes within the same category, which represents the amount of information contained the model for that category. The results in Table 8 demonstrate an improvement of class-similarity information after AFT. The smoothness imposed during the AFT can enhance the semantic-leve smoothness the prediction, which increases similarity in the representations of samples from similar classes.
Zero-shot Accuracy and Robustness. Here we show the complete version of Table i Table As a reference, we include the results of FARE (Schlarmann et al. 2024), which utilizes AFT to enhance the adversarial robustness of CLIP. AFT can result in some drop in terms of clean accuracy. But with a minor e such as 1/255, the zero-shot accuracy only drops by 2.84%, while the performance on ImageNet even increases by 1.02%, while such regularization
still help improve the visual interpretability greatly, as shown in Fig. [4] Moreover, AFT can improve the robustness of the CLIP model significantly, which is another desired property for application in safety-critical industries.
25

---

Published as a conference paper at ICLR 2025
Original Image
clip
c= 255
c 255
255
E 255
8
Figure 14: Comparison of Simple Gradient maps w/wo adversarial fine-tuning with different regu- larization strength.
Table 8: Comparison of the entropy measured based on the output of all class (Entire) and output of classes the same category to the correct class (Category).
Model
Entire
Category
supervised CLIP
1.32
0.15
2.06
CLIP w/ AFT
5.44
2.09
We also discovered our regularized adversarial training achieves comparable robustness compared with the state-of-the-art adversarial training method FARE.
User Study. We conducted a small-scale user study following the methodology of Kim et al. (2022) quantify the impact of high-quality saliency maps on user decision-making, Our focus was on the "distinct" task, as illustrated in Fig. [6 Specifically, we collected 10 validation cases from ImageNet for both the original CLIP and CLIP with AFT, where each included cases with correct zero-shot predictions and with incorrect predictions. For each case, we displayed 4 Grad- CAM maps generated by the network corresponding the top predicted classes: for correct cases, these included the correct class, and for incorrect cases, we showed the top three wrong classes alongside the true class. Users were presented with the input image and the four explanations in random order, and asked them select the class they believed was correct, without revealing class names to simulate a lack of domain knowledge. The 20 cases were also shuffled randomly.
We gathered experimental results from 20 users, which are summarized in Table 10 indicate that saliency maps can aid users in their decision-makinn process. Across all metrics, results were significantly higher than random guesses (25%). For cases with correct predictions, observed that enhancing the quality of saliency maps had limited benefits for user performance. This is likely because humans can automatically de-noise information in their minds, making cleaner
Our findings
26

---

Published as a conference paper at ICLR 2025
Tuning iterations
steps
2000 steps
8000 steps
14000 steps
20000 steps
Figure 15: Evolution of the Simple Gradient maps during the fine-tuning process.
Table 9: Accuracy evaluation on image classification datasets of CLIP model. The superscripts on the upper right side show the value of (/255) during the adversarial fine-tuning.
Zen-thot Datasets
Vision Encoder
ImogeNee
CHERIO
STL 10
CIFARI00
Cans
Calfech
OhfordPer
Flowas
DTD
EROSAT
POVC
PCAM
ImagroNer-FR
ImogaNNor-
Average Zenoshoo
Los
CLIP
74.90
95.
05.20
99.31
71.08
77.94
83.30
93.21
79.17
55.21
62.65
31.77
$2.00
87.86
59.59
72.95
okann
omns
75.92
94.14
99.11
75.34
74.23
84.47
92.91
75.74
54.47
31.61
28.71
52.70
87.56
60.42
70.11
FARE
74.24
89.52
98.47
69.13
70.53
84.77
91.06
70.60
50.05
25.39
26.70
50.01
85.52
59.73
67.04
0un
27.57
59.58
FARE
70.40
77.67
96.04
56.53
63. 34
84.70
87.14
58.07
43.83
18.28
21.99
50.00
$0.20
56.73
61.16
86.97
601.9
CLIP
0.00
0.00
0.00
0.00
0.00
0.00
0.00
0.00
0.00
0.00
0.00
0.00
0.00
0.00
0.00
Loe = 2/255
FARE
29.82
$1,80
83.40
28.10
13.90
61.
$2.60
19.20
19.00
2.00
2.20
11,30
41.90
31,00
32.17
ouns
FARE?
16.10
60.60
90.30
35.60
25.70
72.50
68.50
31.70
26.60
6.20
5.90
41.90
$6.50
38,30
43.12
FARE
52.44
57.10
89
36.70
29. 80
76.30
72.50
31.50
28. 30
12.80
8.20
50.20
61.60
41.60
45. 89
53.68
57.90
900.00
35.00
30.
77.60
72.50
30.20
80
12.60
8.00
50,20
62.10
43,10
46,30
1/255
FARE
2.18
10.50
29.10
5.90
0.20
15.10
3.20
L.10
3,50
0.00
0.00
0.30
12.00
12,80
7.44
1.82
10.20
28.40
5.40
0.20
17.10
0.90
3.10
0.00
11.70
13.10
7.12
FARE
16.64
25.90
61.70
14.10
4.50
45.90
27.90
7.00
11.80
0.70
0.60
17.30
25.60
22.40
20.44
ouns
18.34
26.90
63.40
14.30
5.10
17.30
30.50
6.90
12.50
1.50
0.50
19.70
27.00
23.10
21.46
ounS
35.28
36.40
75.70
21.20
12.30
65 90
51.50
13.00
17.60
11.30
2.60
50.20
11.00
31.60
33.14
saliency map less impactful. However, for cases with incorrect predictions, we found that better saliency maps significantly improved user decision-making, Despite the model's wrong predictions, when provided with the true class name, it still produced visually appealing and semantically aligned saliency maps. Consequently, users were more inclined to select the true class. In contrast, when faced with noisy saliency maps, users tended de- fe-noise them mentally, often retaining only the largest connected component. This process could lead to the exclusion saliency corresponding to the actual object.
27

---

Published as a conference paper at ICLR 2025
For each photo, show explanations for the model's predictions,
Please select the class you think correct.
Photo
Class
Class
Class
Class
Which class you think correct?
01
Photo
Class
Class
Class
Class
Q. Which class do you think is correct?
02
O3
04
Photo
Class 1
Class 2
Class
Class 4
Q. Which class do think is correct?
01 02 03
04
Figure 16: Illustration on our user study set ups.
Table 10: Results of user study, For each study, we report the mean accuracy and standard devia-
tion of the participants' performance. The "correct" row refer cases whose labels corrected predicted by the network while "incorrect" is the opposite. Overall, explanations of higher quality somehow help the users correct the mistakes made by the network.
Backbone
CLIP
CLIP w/ AFT
correct
13.00
74.00 15.00
incorrect
40.00
Another noteworthy finding was that users often selected the saliency maps with the highest sparsity as corresponding the true class, This observation supports our motivation to promote sparsity in saliency maps.
Cost Analysis. Finally, briefly address the cost associated with algorithms. AFT is form of fine-tuning for the visual encoder of CLIP, and it does not introduce any new parameters. All aspects of the model, apart from the specific parameter values of the visual encoder after AFT, main unchanged from the original CLIP. Consequently, the inference cost for CLIP and downstream VLMs remains identical. To minimize the cost of AFT, we applied unsupervised AFT without uti- relatively small-scale datasets. Specifically,
lizing the text encoder, allowing it to be implemented we report the training time for our method in Table [T] based on training with the ImageNet dataset for 2 epochs. All experiments were conducted on NVIDIA GeForce RTX 4090 GPUs.
A.5 DISCUSSION
One of the primary objectives of this work is to enhance the quality of saliency maps. While saliency maps are among the most widely used tools for explanations, they do have limitations. For instance, they primarily indicate the presence of specific objects but struggle to elucidate more complex con- textual features, such orientation and spatial arrangement. Consequently,
numerical exper- iments focused on using saliency maps to highlight non-contextual features of the (image, text) pairs, in line with much of the existing literature. However, during experiments, discovered
28

---

Published as a conference paper at ICLR 2025
Table 11: Cost analysis. Training time is calculated training on ImageNet training set for epochs.
Backbone
Batch size
N. of GPUs used
Training time
RN50
128
2
20,48h
VIT-B-16
128
34.20h
ViT-L-14
64
94.28h
Input Imagge
Grad-Cam
stand upside down
jump
throw
holding
standing
small
large
orthogonal
left
right
Figure 17: Combining multiple modalities in the interpretation framework enables the explanations for more complicated concepts, such as interaction, size, and relative spatial arrangement. It still imperfect. For example, the CLIP explanation fails to distinguish between "left" and "right".
that integrating both text and image modalities within our CLIP-based interpretation framework can improve interpretative capabilities. As illustrated in Fig. 17 the combination of text prompts and saliency maps allows the framework to explain more complex concepts, such as interactions be- tween human and objects, relative sizes, and spatial relationships. There are, of course, instances where the explanations fall short; for example, the model may struggle to differentiate between "left" and "right." Nevertheless, these findings underscore the significance of pursuing research directions focused on modii-modal interpretations.
As for the application of more interpretable CLIP, it widely acknowledged CLIP can be zero- shot classifier for many tasks (Guo et al. 2023 Novack et al. [023 [Shha et al. 2024). When treating CLIP as a zero-shot classifier, can be important to also provide sample-level explanation as well as mechanistic interpretations to help humans make the final decision, especially in high-stake scenarios such as computer-aided diagnosis (Liu et al. (023). Additionally, the explanations can do more. There is work utilizing the saliency maps generated by CLIP for visual grounding open- vocabulary segmentation (Hsia et al.) 2022 Lin et al. 2023). For such applications, explanations with better quality can improve the model performance. We also give preliminary evidence through the pointing game experiments. More interestingly. there is recent study (Yu et al.) (024) that employs CLIP generate attention prompts. The attention prompt can point out the correct region to focus on for the downstream LLM, which has proven empirically to significantly improve the performance of the large vision-language models. It is foresceable that better localization enables the large model to better focus on the correct objects. What is more, in our main text, we also provide an example that the improved visual interpretability helps explain why the model makes certain hallucinations. Overall, it is worth the efforts to build a more interpretable CLIP.
29


Reviews:
[{"review_id": "kynGcZy9iG", "reviewer": "Reviewer_Hgif", "summary": "The paper puts itself in a new area of study — studying interpretability in the context of the vision language models. Whereas the previous works have majorly focused on the interpretation of multimodal interactions, for example, how different text tokens interact with different image regions. However, this paper tries to probe and improve the interpretability of the image encoder of the vision language model (CLIP). They further show that the enhanced interpretability of the vision encoder is transferable - plugging in the same image encoder in a different vision language model (e.g. LLAVA)  provides better attention maps for interpretation. The proposed unsupervised adversarial fine tuning (AFT) method is finally evaluated extensively against various benchmarks and provides mathematical insights into its workings.", "questions": "Refer to the weaknesses.", "limitations": "", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1. The paper is organized and very well written.\n2. The paper reports the evaluation of the proposed AFT approach with good coverage.\n3. They show that optimising the upper bound for a sparse set of concepts of vision encoder using L1 regularization is independent of the text embedding and, hence, can be optimized in isolation.\n4. They cover the evaluation of saliency map explanations quite well- using transferability, feature importance (ROAR benchmark), attention maps and pointing game.\n5. It also provides concept explanation visualization using concept detectors through network dissection.", "weaknesses": "1. There are studies that connect adversarial robustness and saliency explanations [2] that attempt to use adversarial training [1] to improve interpretability. Such studies are missing in the related works section in the entirety. Coverage of such studies and then comparing them with the proposes AFT method could significantly enhance the value of the paper and place it in the literature.\n2. Because, in real-life scenarios, the features a black box relies on might be non-trivial in image space. For example, both a 'classroom' and a 'movie theatre' have collection of chair, and what differentiates them is the orientation of the furnitures, size of the room and so on. Such concepts can not be highlighted using saliency on the image space. Concept of saliency maps only works in a simple (e.g. object detection) framework.\n3. The explanations were initially intended to help the end user (e.g. driver of a car with a deep learning autonomous driving agent) of the application to understand the system better. Such studies determine, irrespective of an explanation being more/less accurate, how much the end user is benefitted from the explanations to perform the enf task [3].\n\n\n        [1] Xu, Rui, et al. \"Scaat: Improving neural network interpretability via saliency constrained adaptive adversarial training.\" arXiv preprint arXiv:2311.05143 (2023).\n        [2] Etmann, Christian, et al. \"On the Connection Between Adversarial Robustness and Saliency Map Interpretability.\" International Conference on Machine Learning. PMLR, 2019.\n        [3]  HIVE: Evaluating the Human Interpretability of Visual Explanations (ECCV 2022).", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "C1gfjYRGZQ", "reviewer": "Reviewer_jkum", "summary": "This paper proposes an unsupervised adversarial fine-tuning (AFT) strategy for improving the visual\ninterpretability of CLIP. Understanding and improving the interpretability of CLIP is crucial since it has been utilized as a visual encoder in multiple image generation and vision-language models (VLMs). This paper demonstrates the effectiveness of the proposed method in improving interpretability, both qualitatively and quantitatively, on multiple feature attribution and neuron explainability methods.", "questions": "Please see weaknesses [1-3]", "limitations": "", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1. The paper is well-motivated, making it easier to follow the main ideas and claims.\n2. The theoretical result on the duality between the regularised norm of adversarial perturbations and the input gradients is interesting and non-trivial. Further, the proposed unsupervised objective function helps alleviate the need for paired image-text datasets.\n3. Improvements in interpretability have been demonstrated on multiple feature attribution and neuron explainability methods.\n4. The proposed method does not significantly change the zero-shot generalization ability of CLIP while improving interpretability, as demonstrated by the linear probing results.", "weaknesses": "1. One of the primary motivations of this work is to improve the interpretability and performance of VLMs. However, only qualitative results are provided in Sec 4.3 and Fig 7, and the paper does not provide any quantitative results for the same.\n2. VLMs and image generation models need vision encoders without any explicit need for encoders trained with contrastive loss. Is this method applicable to vision encoders in general or limited to CLIP?\n3. The empirical performance difference between training unsupervised upper bound in Eq 7 vs Eq 1 has not been quantified. $T_x$ calculation should not significantly increase the computation overhead on smaller datasets and quantifying the difference in performance would give a better understanding of the tradeoff with use of Eq 7.\n4. Ln 206: Should be $T_x$", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "eDMxfE83UT", "reviewer": "Reviewer_Wa4H", "summary": "The paper studies the interpretability of VLMs like CLIP and and LVLMs like LLAVa. The authors propose a adversarial fine-tuning based loss which includes a regularization term and addition of Gaussian noise in the standard adversarial training loss. The resulting method AFT, achieves better interpretability in terms of saliency maps, feature importance etc in comparison to standard ERM based CLIP.", "questions": "Some questions are already present in Weakness section. A few general queries are listed here.\n\n- The experiment with LLaVA is interesting, how much does the sparsity inducing term h(.) help here? It is interesting as now attention weights from LLM are used and this should have an effect in noise/smoothing issues mentioned earlier in the paper regarding h(.).\n\n- Is the gaussian noise sampled at every PGD iteration?", "limitations": "", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 2, "contribution": 2, "strengths": "- The paper is nicely motivated and easy to follow.\n- The set of experiments is extensive and helps to highlight the effectiveness of the proposed AFT loss.\n- The experiment cover both CLIP and LVLMs like LLaVA.\n- AFT yields better interpretable saliency maps and also attains zero-shot robustness similar to baselines.", "weaknesses": "- The novelty aspect of this work is not enough, it is known adversarially robust models yield more interpretable models [1, 2, 3], and this work also comes to a similar conclusion albeit with a different formulation of objective. Would FARE loss not yield similar interpretability?\nsee next points.  \n- The loss equation. 7 is similar to the objective of FARE [4] (upto Gaussian noise and the regularizer, h(.)) - there needs to be a discussion regarding this (which is missing in the current version - only a small sentence regarding this is found in the appendix).\n- If one removes the noise term and h(.) - one retrieves FARE objective - How does this loss perform in the same setup as the figures 2 and 3?\n- Would FARE not be an apt baseline in addition to original CLIP?\n\n- Gaussian term is motivated by \"avoid being stuck in the non-trivial stationary point\" - using PGD for AT one can start at a random location in the $\\ell_p$-ball, this randoms start already helps with not being stuck. It seems gaussian noise is sampled every iteration, as this seems an important part of the loss - the effect this term has is not discussed\n\n[1] Etmann, C., Lunz, S., Maass, P., & Schönlieb, C. B.  On the Connection Between Adversarial Robustness and Saliency Map Interpretability. ICML 2019.\n\n[2] Wang, Zifan, Matt Fredrikson, and Anupam Datta. Robust Models Are More Interpretable Because Attributions Look Normal. ICML 2022.\n\n[3] Ross, A., & Doshi-Velez, F. (2018). Improving the Adversarial Robustness and Interpretability of Deep Neural Networks by Regularizing Their Input Gradients. Proceedings of the AAAI Conference on Artificial Intelligence, 32(1).\n\n[4] Schlarmann, C., Singh, N. D., Croce, F., & Hein, M. Robust CLIP: Unsupervised Adversarial Fine-Tuning of Vision Embeddings for Robust Large Vision-Language Models. ICML 2024.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "9qkMz8zija", "reviewer": "Reviewer_cNDm", "summary": "The paper proposes an unsupervised adversarial fine-tuning (AFT) method to alleviate the issues of insufficient representation and poor interpretability in visual representation learning of CLIP. The paper provides theoretical evidence for the feasibility of the method and presents a rigorous derivation process with well-defined formulas. The experimental results provided in the paper demonstrate the effectiveness of the proposed approach.", "questions": "1. Does AFT introduce additional complexity? It would be better to provide comparative experiments regarding the parameter number, computational cost, and other factors before and after applying this method.\n2. Can the proposed method be applied to base models other than CLIP? These experiments could show the generalizability of the method.", "limitations": "", "rating": 8, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "1. There is a clear definition of the research problem, which is to enhance the visual interpretability of CLIP. The problem is visualized through the form of saliency maps.\n2. The proposed method in the paper demonstrates a good level of originality. The extensive formalized equations provide evidence of the method's rigor and credibility. The abundance of experimental data and visualization results further confirm the effectiveness of the AFT method.\n3. The paper presents a clear and well-structured expression, making it easy to read and follow.", "weaknesses": "1. From the experimental results, AFT appears to be an effective method. However, it is unclear whether this method introduces additional complexity. I would like to see comparative experiments that evaluate the parameter quantity, computational cost, and other factors before and after applying this method.\n2. Although the paper designs the AFT method based on CLIP and achieves good results on relevant datasets, it is mentioned that the AFT method can be extended to any base model that includes intermediate embedding layer connection patterns. More experiments should be provided for this claim. In other words, selecting a few base models other than CLIP and applying the AFT method to demonstrate its general applicability would be beneficial.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "DFccd7reg5", "reviewer": "Reviewer_pAYu", "summary": "In this paper, the authors propose an unsupervised adversarial fine-tuning (AFT) method with norm-regularization to enhance the visual interpretability of CLIP.  Overall, this paper proposes a comprehensive theoretical analysis of their motivation and provides extensive experiments to prove the correctness of their ideas. However, it still has some issues.", "questions": "1. Could you please provide more easy-understanding and plain descriptions of what you do? For example, the architecture of your propose method, and the final loss function of your method.\n\n2. Make more claims on your contributions. For instance, the innovation of your ideas, or the creative construction of your method.\n\n3. why do you call your methods as the adversarial fine-tuning?", "limitations": "", "rating": 8, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1. The theoretical analysis of their method is supportive\n2. The experimental results are enough to support their motivation and demonstrate its correctness.", "weaknesses": "1. Too few content is with regard to the implemental details.  At least, an overview of the proposed method should be provided.\n\n2. Though the proposed method is interesting, a single novel norm-regularization might not be enough to support the high standard of ICLR. More introduction content of your main contributions should be added. For instance, the innovation of the proposed method or the creative construction of your method.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)