Zero-Shot Natural Language Explanations
Authors: Fawaz Sammani
Nikos Deligiannis
Venue: ICLR 2025 Poster (2025)
Keywords: Natural Language Explanations
interpretability
explainability
Primary area: interpretability and explainable AI

Abstract: Natural Language Explanations (NLEs) interpret the decision-making process of a given model through textual sentences. Current NLEs suffer from a severe limitation; they are unfaithful to the model’s actual reasoning process, as a separate textual decoder is explicitly trained to generate those explanations using annotated datasets for a specific task, leading them to reflect what annotators desire. In this work, we take the first step towards generating faithful NLEs for any visual classification model without any training data. Our approach models the relationship between class embeddings from the classifier of the vision model and their corresponding class names via a simple MLP which trains in seconds. After training, we can map any new text to the classifier space and measure its association with the visual features. We conduct experiments on 38 vision models, including both CNNs and Transformers. In addition to NLEs, our method offers other advantages such as zero-shot image classification and fine-grained concept discovery.

Published as a conference paper at ICLR 2025
ZERO-SHOT NATURAL LANGUAGE EXPLANATIONS
Fawaz Sammani & Nikos Deligiannis
ETRO Department, Vrije Universiteit Brussel, Pleinlaan 2, B-1050 Brussels, Belgium imec, Kapeldreef 75, B-3001 Leuven, Belgium fawaz. sammani @ub be, ndeligia@et rovub be
ABSTRACT
Natural Language Explanations (NLEs) interpret the massinn process of given model through textual sentences. Current NLEs suffer from a severe limitation; they are unfaithful to the model's 's actual reasoning process, as a separate textual decoder is explicitly trained to generate those explanations using annotated datasets for specific task, leading them to reflect what annotators desire. In this work, we take the first step towards generating faithful NLEs for any visual clas- sification model without any training data. Our approach models the relationship between class embeddings from the classifier of the vision model and their corre- sponding class names via a simple MLP which trains in seconds. After training, we can map any text to the classifier space and measure its association with the visual features. We conduct experiments on 38 vision models, including both CNNs and Transformers. Our method outperforms supervised baselines on many metrics, while remaining comparable on others. In addition to NLEs, our method offers other advantages such as zero-shot image classification and fine-grate ane-grained concept discovery, each outperforming baseline methods. Finally, we also show that our method achieves state-of- ttth-ertt results on zero-shot image captioning.
INTRODUCTION
Consider the example in Figure I where an image classifier incorrectly identifies the input image as volcano rather than a leatherback turtle. Why was this decision made and which features of the image led to this prediction? When we apply popular attribution techniques such as the CAM family (Zhou jet al., 2015| varaju et al. 2019 Jiang et al. 202T), the explanations are uninformative; perhaps they indicate that the model focused on the main object, but they fail to provide deeper insight. What specific elements within the attributed region influenced the prediction? What features of this image category were significant? Existing attribution techniques offer high-level interpretations and do not break down the reasoning behind the decision. On the other hand, Natural Language Explanations (NLEs) (Park et al. 2018 Sammani et al.) 2022) offer detailed interpretations in a textual format. NLEs not only resolve the limitations of attribution methods but also present explanations in a more human-friendly manner, accessible even layman user. As illustrated in Figure] 1 the NLE reveals that the classifier mistook the image for a volcano due to the bright red glow, resembling crupting lava. This insight allows us deduce that the classifier is not robust to color shifts.
However, current NLE models are explicitly trained to generate textual explanations using annotated datasets. This renders NLEs as unfaithful, as they reflect what dataset annotaters desire rather than the model's true reasoning process. Other than that, NLE models are characterized with the shortcut bias problem, shown in Sammani & Deligiannis (2023). rendering the explanation meaningless despite achieving state-of-the-ar results on natural language generation metrics.
In this work, we aim to generate faithful NLEs in a zero-shot setting for a visual classifier model. This boils down to the question: how can we access the visual features via any natural text? To achieve this, we require an intermediary between visual features of the model, and natural language text. Notably, the discrete labels of the classifier correspond to class names in text format, and the classifier weights of each class serve as a embedding vector representation for that class. These embeddings effectively encode semanties; similar objects close together in the embedding space. Figure Dillstrates a visualization of those embeddings from a ResNet-50 model for certain classes, using t-sne (van der Maaten & Hinton 2008). As shown, ocean fish such different types of sharks

---

Published as a conference paper at ICLR 2025
Ground-truth: leatherback turtte
Gred-CAM
Layer-CAM
Smooth-CAM
Prediction: volcano
NLE (Ours): this valcano because shows fire burning on the volcano lava flows,, bright red glow erupts
Figure I: Comparison between some attribution methods and Natural Language Explanation (NLE) for explaining the prediction of an image as a volcano
and rays form one cluster, while freshwater fish proximally form another cluster. Similarly, small birds such as robin and jay form a distinct cluster, and so on. Given that discrete labels represent class names, the classification layer of the vision model performs closed-set visual-textual search over all possible classes, with the predicted class being the one with the highest cosine similarity between the visual features and class embeddings. The classification layer therefore retrieves the class closest to the image. This implies that those class embeddings and the visual features share a common space. Therefore, if we can learn mapping from the available text data (class names) the vector embeddings of the classes (classifier weights), we have essentially learned the visual-semantic space of the vision classifier, which then allows to translate any new text (e.g., new class, sentence, or a concept) into that embedding space and measure its association with the visual features. Once the mapping is learned, we employ off-the-shelf frozen language model and utilize inference-time prefix- tuning (Li & Liang 202T) to generate a NLE that maxi- mize the similarity with the visual features. Additionally, the learned mapping offers us with several other applica- tions, such concept discovery and zero-shot classifica- tion.
sogle
grey onl
Jeambing
aueidfinh
biblo
roble
Haggle
Janco
Joy
chicoode
enter
@dedgo burting
To summarize, our contributions are follows: 1) We are the first to generate NLEs in a zero-shot manner for visual classifiers that are also faithful to the model, achiev- ing performance that is comparable to, and in many cases surpasses supervised baselines. 2) We demonstrate the effectiveness of our approach in three additional applica- tions: zero-shot concept discovery, image classification and image captioning, showing significant improvements over baselines. 3) We evaluate a wide set of 38 vision classifiers, including both CNNs and Transformers.
stingray
atark
harmarheed
Ugar
Figure 2: t-sne visualization of a ResNet- 50 class weight embeddings
2 RELATED WORK
Natural Language Explanations: Early works on NLEs for vision and vision-language tasks include Hendricks al. (2016); Park et al. (2018): Marasović al. (2020); Kayser et al. (2021). The standard pipeline typically consists of a task model (e.g., a classifier) for prediction, coupled with an explainer model (c.g., GPT-2 (Radford et al. 2019 to generate an explanation for the prediction. NLX-GPT (Sammani et al., 2022) proposed unified approach, combining both models into a single, compact system that simultaneously generates and explains answers in a single text stream using a single causal language modeling objective, ensuring that the reasoning process for the explanation aligns with that of the prediction. Building on this, Uni-NLX (Sammani & Deligiannis/ 2023) further consolidates multiple tasks into one model, achieving the ability to perform seven tasks simultaneously. Multimodal-CC technique and instead generates rationale (explanation) prior generating answer, which serves as a reasoning step for inferring the answer. In De (Dani et al. 2023), an autoregressive generator is trained on the conceptual captions image captioning dataset (CC3M) (Sharma et al. 2018) using
(Zhang et al. 2023) builds upon the Chain of Thought Prompting
2

---

Published as a conference paper at ICLR 2025
classilier
Inference: any new leed g, descriptor)
MLP
desciie
anch
freshwater
VIT / CNN
Textual Encoder
morimnre distance
Textual Encoder
f
Prench
VITICNN
MLP
MLP
Visual
image tench an image of goldish
Encodor
cosine
Killec
0.92
Porter
image tollet tissue
(a)
(b)
(c)
(d)
Figure 3: An overview of our method to learn class embeddings. (a) We transform the visual classifier into a visual feature encoder and text encoder paired with an MLP module that outputs classifier weights. (b) The classifier weights, which are the class embeddings, serve as ground-truth data to train the MLP module.
(c) We learn the class embeddings by converting the class name into a text prompt, encoding it with the text encoder and MLP module, aiming minimize the distance between the MLP output and the moound-tuuth data. (d) Once the MLP module is trained, we can map any new text to the classifier embedding space, which now shares the same space the visual features, allowing us to measure the association between them. * indicates that the module is frozen, while indicates trainable.
a subset of features from different layers implemented by applying Dropout. However, all these methods rely on annotated datasets consisting of (prediction-explanaioo pairs. As result, the explanations generated reflect the reasoning of the annotators rather than that of the model. This is particularly evident in DeLLL, which treats image captions explanations. On the other hand, our NLEs are generated in a zero-shot manner a way that maximizes the classifier's visual feature space, ensuring that they are faithful to the classifier's reasoning process.
Zero-Shot Textual Explanations: ZS-A2T (Salewski et al. 2023 is the work most closely related to ours, it translates attention maps into textual explanations in zero-shot manner. There are also several studies (Menon & Vondrick) 2023, Shtedritski et al. 2023) which offer concept- based explanations (short textual descriptors) rather than full sentence- based explanations for model predictions. However, all of these approaches are constrained to vision- lsiounlanguage models that are trained learn shared vision-language space through contrastive objective, meaning they cannot be applied to visual classifiers. In contrast, our NLEs are classifier-agnostic, do not depend on a contrastive training objective, and provide full conenccebbssed explanations.
3 METHOD
Consider a visual classifier model M that want to interpret, which can be of any architecture (e.g., CNN, Transformer or Hybrid). M consists of a visual feature encoder My, and classification layer W. My maps an input image into feature vector f. That is: My (x) = f E Rd Usually, f is the result of average or max pooling, or is the result of the [CLS] token in some Transformer models. The visual feature vector f then fed the classifier layer W E R with C classes. Therefore, the matrix W consists of all class embedding vectors which encode the semantics of class: W (Wa W Wo wc). and |W| = C. The prediction Ep is given by arg max(f W). Unlike the visual features f. note that W is fixed within the network and independent of the input image x. We consider an off-the-shelf textual encoder Mr capable of encoding sentences, with the only exception that this textual encoder does not also encode image information (as in the case of the CLIP textual encoder (Radford et al. 2021), ). The textual encoder My takes as an input a natural text and produces a sentence embedding 8; of l dimensions. That is, My(t,) 8; E R
Notations:
3.1 LEARNING CLASS EMBEDDINGS
The visual classifier M is a closed-set classifier. Our goal is to expand into an open-set visual classifier capable of understanding any natural text. Specifically, want to access query the
3

---

Published as a conference paper at ICLR 2025
classifier via text. The simplest approach is align text encoder with the visual features of the classifier, and then access those features through text. This is the contrastive language-iman pretraining paradigm that CLIP (Radford al.] (02T) employs. However, this method is 1) not faithful to the classifier and 2) poses significant technical challenges. More details are provided in Section Mfoo the appendix. To address these issues, aim to learn a regression function that maps text to the classifier's 's embedding space. Note that this is an ill- iol-posed problem, meaning there is either no unique solution or the correct solution difficult determine. While we may not be able to learn the exact ground-truth values of the d-dimensional class embedding vectors W, we can still capture their underlying semantics.
The classification layer W contains class embeddings which share the same space as the visual features f. Drawing an analogy with CLIP (Radford et al. 2021). W can be thought of as the output of a text encoder that is trained to be aligned with the visual features f through an alignment loss such as contrastive learning. However, in practice, Mr is a pretrained off- fff-the-shelf text encoder that is not trained to align with f. Drawing inspiration from Christensen al. (202)), we introduce a small learnable module comprising a simple Multi-Layer-Pereeprre
(MLP). P). which is trained to map text to the classifier embedding space (Figure Ba). We first map every textual class name C2. where: i = 1, ... C. to a corresponding natural text prompt t,, specified as: an image of a {c]- For example, if e is the class tench, then 4 is: an image of a lench. Given that we want learn mapping from 14 to the class embedding W., E W, the only ground-truth data available to us is W. which are the fixed C' class embeddings of the classifier (FigureB)), For ImageNet-IK (Deng et al. 2009), C' = 1, 000, presenting highly scarce data scenario. Nevertheless, as we will elaborate later, this challenge can still be addressed using specific techniques. Each prompt t; is then encoded with the text encoder My to produce a text embedding representation st, which we feed to a shared MLP layer. The regressed class embedding for the class e; is defined as We, (s:).s:. The prediction Cp can then given by arg max(f W). where W = =[W., Wey....,wcll
The MLP is trained to minimize the distance between the regressed class embedding we, and the ground-truth embedding w., (Figure 33). Note that the only trainable module is the MLP, and the visual classifier My and text encoder M1 kept frozen. We employ the cosine distance instead of other functions like L1 or Mean Squared Error (MSE) loss, as cosine loss incorporates angular information. This approach yields improved performance, as demonstrated in the ablation experiments in Section Df the appendix. The loss objective L is then given by:
L Σ
W
Training the MLP takes roughly 10 seconds on a single moderate GPU, and is therefore considered negligible and can be applied instantly any visual classifier model. Once the MLP is trained, we can map any new text (e.g., a descriptor, new class, or sentence) to the class embedding space. This mapped text now shares the same embedding space as the visual features f. and we can measure the association of that new text to ʃ (Figure (tt) It is worth noting, that in coarse or fine-grained classification, some class names (e.g., "tench") may be unfamiliar a text encoder trained on general language. However, because their corresponding class embedding is close to that of another class that the text encoder understands (c.g.. "goldfish") (see Figure 2. the MLP can infer that "tench" is fish, even though the word "fish" is not part of the class name of "tench". Furthermore, it is important to note that we only use the class name, and no other supplementary information. More details regarding this can be found in Section of the appendix.
As mentioned, we are faced with an extremely scarce data scenario. We therefore utilize two simple but highly effective data augmentation techniques, both which allow us to create training points which are in proximity to the original training sample.
Input Dropout: This follows the conventional Dropout technique (Srivastava et al.) 2014). We randomly drop 50% of s; (the input features to the MLP) with a probability of 0.3. The regressed class embedding is then defined wa, MLP(ddrpp( (s:)). As shown in Section of the appendix, this simple technique already boosts performance by a large margin.
Learned Soft Dropout:
We propose to implement learned dropout mechanism, where instead of completely zeroing out features randomly, we learn how to selectively scale them with values between O and 1. We term this approach Soft Dropout. We implement this via Gated Linear
4

---

Published as a conference paper at ICLR 2025
uptade
cat
this is cat
XE Loss
jacket
this
water
this water
arm this arm
animal this animal
top-k wordss
VIT/CNN
jacket
Pretrained Language Model
ceosine
Construct Target Distributinn
Textual Encoder
Prefix
Ihis
MLP
cosine
Figure 4: An overview of how update the prefix. The example is shown for the first timestep = 1 at iteration = 1, with K = 5 and hard prompt set as {this is a). cosine indicates cosine similarity.
Unit (GLU) activation function DDuphin et al. 2016) on the MLP output. The regressed class embedding is then defined as We: GLU dr)p(s:))) Since GLU is applied independently to each dimension of the MLP ouiput, it scales different dimensions of the learned class embedding, effectively associating the space around the training point to the corresponding class. As shown in Section of the appendix. this technique further provides significant improvements in performance in such a scarce data scenario.
3.2 GENERATING ZERO-S SHROT-SOTTTO NLES
Once we establish the mapping function MLP which projects the textual features into the same space as the visual features f, we can optimize an off-the-shelf pre-trained language model
of generating coherent text-to generate a NLE that maximizes the similarity with the visual features f. We freeze the PLM to keep its powerful generation capability, and instead use prefix-tuning (Li & method which attaches learnable vectors in the embedding space of the PLM. We follow an inference-ting approach where we optimize learnable prefixes for each sample individually. Unlike supervised models which require single forward pass inference, this approach is more time-consuming. However, it allows for much greater flexibility in the generated text, operates in open-set environment (generating words outside the dataset corpus), and can be adapted to any example or classifier on-the-fly. We build upon the approach introduced by Tewel al. (202I). p). An overview of this process is shown in Figure Given a PLM (e.g., GPT-2), attach randomly initialized learnable prefixes to it to steer the frozen language model to generate explanations of the classifier. Here, the learnable prefixes are as initial key-value pairs in each attention block of the Transformer, such that each generated word in the explanation can attend to these prefixes. At each timestep we sample the top-K K tokens from the PLM output vocabulary distribution o₁ which act as K continuation tokens of the currently generated explanation. These K potential sentences are then passed to the text encoder Mr. followed by the learned MLP to produce K vectors that share the same embedding space as the visual features f. The cosine similarity between each those vectors and the visual features f is then computed, yielding similarity scores with the visual feature vector f. These scores, normalized with softmax, form a target distribution to train against or with the standard Cross-Entrops loss, and the prefixes of the PLM are updated with backpropogation. We then run the PLM again with the updated prefixes and we sample from the output distribution the most likely token. Namely, per timestep. K token are initially produced to update the prefixes but one token finally sampled (after the prefixes are updated) as the continuation of the explanation. We run the above process for L timesteps. We set L as the maximum sequence length we want for the explanation, or until the <. > token reached. After these I. timesteps, one iteration will be concluded, and one explanation will be generated. Therefore, each iteration generates tokens which have their continuation relevant to the
Liang 202 an
one complete explanation. The sampled features will have their scores increased during the generation process, while non-relevant tokens will have their scores decreased. We train the prefixes for / iterations, producing / explanations. Because the similarity between the MLP output and the visual features is used to construct the target distribution for training the prefixes, we choose a different similarity measure- not biased toward the process and metric used create these explanations-tto select the best explanation from
5

---

Published as a conference paper at ICLR 2025
the I generated ones. Specifically, select the explanation that maximizes the CLIP-Scont the final explanation. More details are provided in Section I of the appendix.
EXPERIMENTS
In this section, we present both quantitative and qualitative experiments. Implementation details are provided in Section of the appendix. We evaluate a diverse set of vision models. selection of these models is included here, with the remainder provided in the appendix. For CNNs, consider the following family of models (each with several variants): Residual Networks (ResNets)
(He al. 2015). Wide ResNets (Zagoruyko & Komodakis 2016), ResNeets (Xie et al.
2016).
Densely Connected Networks (DenseNets) (Huang et al. (2016), EfficientNetv2 Tan & Le
2021 p.
ShuffleNetv2 (Ma et al.
2018). MobileNetv3 (Howard et al. (019), ConvNeXts (Liu al.
2022) and
ConvNeXtv2 (Woo et al.
2023). For Transformers, we consider the following family of models (each
with several variants): Vision Transformers (ViTs) (Dosovitskiy al. 2021), Swin Transformer (Liu
2022). DINOv2 (Oquab et al. (2024) and the hybrid Convolution- Vision Transformer CvT (Wu al. 2021). All models are pretraiedd on ImageNet- 1K (Deng et al.) (2009).
ct al. 2021), BeiT (Bao et al.
Models with the subscript pr indicate that the model was pretrained on ImageNet-21K before being finetuned on ImageNet-1K, Models with subscript v2 are trained with the new recipe from Py PyTorch (Vryniotis, 2021). Finally, BEiT, DINOv2 and ConvNeXtv2 are pretrained in self-supervised manner before being finetuned on ImageNet-Ik:
4.1 LEARNED CLASS EMBEDDINGS
To evaluate the learned class embeddings, we employ zero-shot image classification as benchmark. Since our approach can map any new text to class embeddings (i.e., class weights), we can treat new unsen classes as input text. This implics that by mapping n new classes to class embeddings, we effectively extend our classifier by additional classes. It is important to note that we referring to the classical zero-shot classification paradigm (Lampert et al. 2014 Xian et al. 2017b), unlike CLIP (Radford et al.) 2021) which addresses zero-shot transfer. As baseline, compare against ICIS (Christensen et al.) (023), the current state-of- tftte-of-the-ee th-the--rtt in zero-shot image classification using text-only training. For fair comparison, we re-implemented their method, utilizing the same encoder, dataset splits and class names as in our approach. We use the challenging ImageNet- 1 K dataset as our benchmark. splitting its 1, 000 classes into 900 for training and 100 for testing. For validation and hyperparameter tuning, we use 100 non-overlapping classes from the ImageNet-2 dataset.
We follow the standard evaluation protocol for zero-shot image classification which includes two settings: the Generalized Zero-Shot Ser- ting and the Zero-Shot Setting. The Generalized Zero-Shot Setting is the most important as re- flects real-world scenarios. In this setting, we generate class weights for both the seen (train- ing) classes, and the unseen (testing) classes. For ImageNet, these are all the 1, 000 classes. The image is then classified into one of these classes by comparing its features f against all these classes, and the class with the highest sim- ilarity is predicted. "Train" refers to the results on the training set, while "Test" refers to the results on the testing set. We report the standard Top-1 (@) 1) and Top-5 (@5) accuracy metries. In the Zero-Shot Setting, we generate weights for the unseen (testing) classes only, and the im- age is classified solely among the testing classes. In the case of ImageNet, the image features f are compared only against the 100 testing classes, and the class with the highest similarity is predicted. However, this setting is less realistic, in practice we do not know which classes are unknown. Results are presented in Table Our method significantly outperforms ICIS on the testing set in the generalized zero-shot setting and, in most
gurtin
gian thing
getas doe
Joinsn gorteclaost
4rds greton you.
get
garcn
goefforo
Figure 5: t-sne embeddings on new unseen classes from ImageNet-21K, for ResNet-50

---

Published as a conference paper at ICLR 2025
Table 1: Quantitative Results on evaluating class embeddings via zero-shot image classification on ImageNet- IK, compared to the baseline ICIS, for several visual classifiers.
Generalized Zero-Shot Setting
Zero-Shot Setting
Model
Method
Train
Train
Test
T
Test
Test @5
ICIS
74.18
91.76
13.04
51.68
49.8
77.04
ResNe150
Ours
66.77
89.34
33.28
66.20
50.88
77.62
ICIS
76.05
92.86
11.18
43.26
46.06
73.48
ResNet101
Ours
69.42
90.19
25.92
57.80
44.36
72.96
ICIS
78.29
93.57
11.82
49.08
51.00
79.26
WidcResNe150
Ours
74.27
92.05
30.20
64.92
49.98
78.46
ICIS
78.89
93.67
9.26
46.02
46.38
75.22
WidcResNet101
Ours
75.65
92.59
26.66
60.60
48.58
75.82
ICIS
77.54
93.32
8.78
44.00
44.24
74.52
ResNcXt5002.ad
Ours
72.44
90.97
29.96
63.00
46.90
74.96
ICIS
74.22
91.53
3.30
21.44
39,56
65.56
MobileNetv3-L
Ours
68.26
89.84
18.90
43.42
39.32
66.00
ICIS
85.43
97.11
4.98
50.62
57.72
84.18
ViT-B/16
Ours
85.47
97.35
9.18
53.84
58.20
83.72
ICIS
87.79
98.32
5.72
49.62
56.96
78.84
BEIT-L/16
Ours
87.57
98.26
10.84
51.52
54.10
80.22
ICIS
81.30
96.08
13.96
44.70
46.60
72.
DINOv2-B
Ours
79.26
95.95
21.52
49.36
43.50
73.72
ICIS
87.89
98.20
6.06
48.64
56.36
76.74
ConvNeXtV2-Bp..st
Ours
87.90
98.31
9.36
50.60
56.72
77.58
cases, in the zero-shot setting well. As this is an ill-posed problem, the testing accuracies may not be particularly high. This phenomenon is standard in zero-shot learning (Xian et al., (2)) especially on the challenging ImageNet dataset. Evaluation on other models are presented in Section of the appendix. We also present analysis on test prompt sensitivity in Section of the appendix. Fīgure(5] we present t-sne visualization of the generated embeddings for new unseen classes taken from ImageNet-2IK. As seen, semantically similar classes clustered together (e.g., water sports cluster, people cluster). After tuning the hyperparameters of the MLP mapper, we train on the full set of classes (1, 000 for ImageNet) for further applications. In Section of the appendix, aeeeeerpppppiiiii
present ablation studies on Input Dropout and Soft Dropout. Our findings show that Input Dropout boosts top-1 zero-shot accuracy by 11.38% points, while Soft Dropout further boosts it by additional 16.04% points. Therefore, these augmentations together provide a significant increase of 27. 27.42% points in top-1 zero-shot accuracy. We also present ablation studies different text encoders.
4.2 ZERO-SHOT NLES
We use the ImageNet-X (Sammani & Deligiannis 2023 dataset which provides explanations for ImageNet categories, describing them with distinctive and physical features. consists of 141K training samples, 2K for validation and 1K for testing. As our method zero-shot, we only use the testing split for evaluation purposes. We compare against 4 supervised baseline NLE models which are trained explicitly to generate those explanations. NLXX-PT (Sammani et al.) 2022) and Uni-NLX (Sammani & Deligiannis, 2023). The subscript means that the NLE model is preceded by pretraining on IM image- caption pairs before being finetuned on ImageNet-X, All baselines are trained on the ImageNet-X training set.
Baselines and Dataset:
Metrics: Supervised NLE models use natural language generation (NLG) metrics such as BLEU (Papineni et al., (2022) measure the n-gram overlap between the generated NLE and the ground ground-truth one. We avoid evaluation using NLG metrics because is challenging to expect an n-gram overlap between the ground-truth annotation and zero-shot explanation. As result, resort to using
7

---

Published as a conference paper at ICLR 2025
camantic-basedd evaluation metrics that, to the best of our abilitty, reflect faithfulness. Given NLE for an image 2, we use the following metrics: CLIP-S: We use the CLIP-Score (Hessel et al. 2021 metric that leverages the powerful CLIP model as external "judge" to assess the matching score between an image-text pair. Here, we use the CLIP-S to judge how associated the image x is to its NLE. That is, CLIP-S CLIP (z, NLE). . An NLE which truly reflects the image and its content, is given a high matching score. LPIPS: Perceptual Similarity (Zhang et al. 2018) is a learned metric trained with human input that has shown to correlate very well with human judgment. measures the similarity between two images semantically using deep network features rather than focusing on pixel-level differences. To utilize this metric, we need two images. Taking advantage of the remarkable ability of current text-to-image models in generating realistic images, we first generate an image by re-formulating the NLE as a text prompt to the Stable Diffusion Text-to-Iman Image model (Rombach et al. 2 to obtain 29 (see Section[]ot of the appendix for more details on this). We then compare x against the image z using the LPIPS metric: LP LPIPS (æ, x9). . The LPIPS score is low when the two images share similar semantic features. The lower the LPIPS, the more the NLE truly reflects the visual features. We use the trained LPIPS madel&"baser on AlexNet (A) and SqueezeNet (S). Finally, Cosine is the cosine similarity between the visual features of the model for the image x and that for 29; cos [M (z), My(xx)) When the NLE is correct and faithful, the synthesized image 2 will activate the same visual features in the classifier as the original image x that generated the NLE, resulting in a high cosine similarity score. Our method, including the 4 baselines follow exactly the same evaluation protocol, using the ImageNet-X test set.
interpret
We follow NLX-GPT and Uni-NLX where we generate both the prediction and explanation. As highlighted in NLX-GPT, this is important ensure that the prediction and explanation come from the same reasoning process. Results are presented in Table 2
On all CNN models, our zero-shot NLEs outperform supervised NLEs on all metrics while remaining comparable on the cosine metric. On Transformer models, our zero-shot NLEs surpass the supervised NLEs on CLIP-S and perform comparably (and sometimes better) across other metrics. Note that the cosine metric only captures the geometrie alignment (or direction) of these vectors in the Euclidean space. As simple example, the cosine similarity between: 01 = [1, 1, 0] and U22 = (2,2.0) where 1/2 = 20, is, 1, indicating perfect alignment. Therefore, the cosine metric is suboptimal for capturing semantically relevant alignment. Evaluation on other models are presented in Section of the appendix.
In Figure 6 we provide qualitative examples of images from different ImageNet categories. The first row presents NLEs that are expressive and free from contextual errors. For example, we can see that the rings presented on the snake's 's skin are distinctive of the category king snake. In the second row, we present examples where the NLEs are expressive but contain contextual errors. Despite these mistakes, they still help users understand the reasoning process. For example, the model identifies a vine snake based on its green, arrow-shaped head, and identifies a tree frog by its green, lizard-like head. We can also reveal that ResNet-50 has associated jellyfssh with "glowing". Upon inspecting the ImageNet training images for the class jellyfish, we observe that most of them feature glowing jellyfish. This suggests that the ResNet-50 model has taken a shortcut bias in learning this class. Additional qualitative examples are provided in Section D of the appendix.
4.3 ZERO-SHOT
CONCEPT DISCOVERY
We present one more numan-friendie interpretable approach to deep learning models, which is fine-grained concept discovery where we aim to dissect the visual features into textual concepts in zero-shot manner. In this context, we refer to concepts as textual descriptors; short descriptions in natural language that describe physical features of many objects in the world. This approach stems from how humans reason. For example, to justify an image as an american robin, we would describe the bird's beak, orange belly, and black back. In this approach, we are given set of predefined concepts. We then query visual classifier to determine whether these concepts are present in the classifier's visual features. Previously, this method was restricted to CLIP models (Menon & Vondrick, 2023). Now, with our approach, it can be applied to any visual classifier. Manually writing these textual concepts can be costly and does not scale to large class sets like ImageNet. To address this, we leverage large language models (LLMs), which demonstrate remarkable world knowledge across various domains, to generate these concepts. See Section K of the appendix for more details.
8

---

Published as a conference paper at ICLR 2025
Table Quantitative Results on our zero-shot NLEs compared supervised baselines, on several visual classifiers. Results are reported on the test set of ImageNet-X.
Model
Metric
NLX-GPT
NLX-GPTn
Uni-NLX
Uni-NLX
Ours
CLIP-S
28.53
28.52
28.38
28.52
30.56
LPIPS(A)
0.721
0.719
0.720
0.716
0.706
ResNet50
LPIPS(S) 4
0.624
0.623
0.620
0.620
0.614
Cosine
0.703
0.704
0.703
0.710
0.708
CLIP-S
28.53
28.52
28.38
28.52
30.73
LPIPS(A)
0.717
0.717
0.720
0.719
0.707
ResNeXt50g2ead
LPIPS(S) &
0.621
0.620
0.622
0,623
0.616
Cosine
0.680
0.677
0.677
0.677
0.676
CLIP-S
28.53
28.52
28.38
28.52
30.56
LPIPS(A)
0.717
0.718
0,718
0.723
0.706
DenseNet161
LPIPS(S) L
0.620
0.620
0.620
0.624
0.616
Cosine
0.589
0.591
0.589
0.588
0.588
CLIP-S
28.53
28.52
28.38
28.52
30.15
LPIPS(A)
0.719
0.716
0.717
0.716
0.712
DINOv2-B
LPIPS(S) 1
0.622
0.621
0.619
0.619
0,619
Cosine
0.362
0.360
0.354
0.356
0.314
CLIP-S
28.53
28.52
28.38
28.52
30.92
LPIPS(A)
0.723
0.723
0.717
0.722
0.710
VIT-B/16
LPIPS(S) 4
0.624
0.626
0.619
0.624
0.621
Cosine
0.450
0.440
0.447
0.445
0.425
Expressive No Contextual Errors
Prediction: barrow
Prediction:
Prediction: airliner
Prediction: king snake
because it shows tractor cart buckets
because shows honey in food bee factory at farm for Honeybea Foods
bocause shows flyng overhead at high speed over airport
because serpentine ring on snake skin
Expressive with Errors
Prediction: hermit because the shell
Prediction:
Prediction: frog
Prediction: jellyfish
because shows green arrow lop
because shows the frog with head covered by green lizard on top
because shows jollyfish glow in a tube on the screen
Figure 6: Qualitative examples of our zero-shot NLEs on ImageNet for ResNet-50 classifier.
Performing this application with our method is simple, and is illustrated in Figure 3 Given large set of concepts U with [0]| Z, every concept u, encoded using the encoder My followed by the MIP to yield concept embedding w w.,. That is, w the cosine similarity between the visual features f and W., V/j L..., Z. We then take the top- B scoring concepts. Qualitative results are shown in Figure 7 for different models. This allows to reveal what concepts different model features encode. Interestingly, we see that the BeiT-L models assigns the top-concept for the "tench" prediction as "10 legs". This classification arises from the patchification process the transformer, where multiple patches comprise of the fins of the fish, which are identified as "10 legs" due their physical appearance.
MLP (M,(u,)) (u,))) We then compute

---

Published as a conference paper at ICLR 2025
ResNet-50:
WideReaNet-10
Bair/16-Large
tunch, dince tince because...
variety of fish
74.25
legs
96.0
freshwater brass or bronzee
49.25
freshwater wish
68.0
fish
48.78
large. elongated
57.75
freshwater
34.0
metal plastic
fish with a bright
leaves
31.50
wire mesh stretched
orange yellow color
brass bronze
ResNet-50:
WideeRssNet-101
BeiT/16-Large
great
vhhite shark
white shark because...
because...
large, an underwater scene
73.30
large, predatory fish
81.0
large, predatory fish
44.0
71.25
dorsal fins
powerful jaws
large. prediatory
71.25
large marine mammal
79.00
sharp
33.5
one or two dorsal fins
70.76
wings fins
78.25
large head
30.5
large marine
underwater scene
77.50
large
Figure 7: Qualitative examples of concept discovery on ImageNet for different classifiers. The cosine similarity is scaled by 2.5 for all models, in order to stretch the range of the score distribution to [0, 1]
Finally, we conduct quantitative evaluation of the detected concepts to assess their effectiveness and faithfulness the visual features. We the top-detected concepts supplementary information to regress the class embeddings. If the detected concepts are faithful, they should lead to better regression the class embeddings, and result higher accuracy. Our baseline uses only the class name to regress the class embeddings. That is, Wey MLP(Mr((4) where 4 is: an image of (class name). For the evaluation, we change t, the which now include both the class name and the detected concepts: the an image of {class name, 4s where is a concept from the top- B detected concepts. Results on ImageNet are presented Table several models. As shown, the detected concepts lead to significant improvements in accuracy, which shows the effectiveness and faithfulness of the concepts to the visual features. In this experiment, we use B 15 and average the text encoder outputs to single feature vector, before we feed it input to the MLP. We also found that setting B 10 yields to approximately same results (e.g., 73.61 Top-1 accuracy for ResNet-50). Evaluation other models provided in Section GoetionnGof. of the appendix.
4.4 ADDITIONAL APPLICATIONS
Table 3: Quantitative results of concept discovery
In Section B of the appendix, we also present zero-shot image captioning experiments on the COCO dataset (Lin et al. 2014) as additional application of our method, achieving new state- of-the-art results while also showing the gener- alizability of our method to datasets beyond Im- ageNet. Furthermore, we present experiments on zero-shot transfer to other datasets in Section of the appendix.
Baseline Top-1
Concepts
Model
Top-5
Top-1
Top-5
ResNet50
63.42
87.02
73.66
92.18
ResNet101
65.07
86.95
75.32
92.91
Wet5ResNet500.2
73.51
90.10
81.06
95.64
WideRes
75.25
90.60
82.05
95.95
ResNeXt101ssal
75.52
90.79
82.85
96.41
DenseNet161
67.84
88.58
75.37
93.11
EfficientNet2--S
76.64
91.99
84.00
96.79
EfficientNetv2-M
77.82
92.37
85.03
97.17
CONCLUSION
ShuffleNetv2-co
68.02
86.87
75.04
92.64
MobileNetv3-L.
63.32
85.20
71.76
90.62
We proposed zero-shot faithful NLE method for visual classifiers. We also presented other
ViT-B/32
68.66
85.76
75.31
92.34
ViT-B/16,maB
77.84
93.00
84.42
97.48
ViT-L/32
69.83
86.66
76.47
92.97
applications on zero-shot image classification and fine-grained concept discovery. Note that our work offers a solution to a wide range of zero-shot applications that were previously re- stricted CLIP models. Our work removes this limitation, and can now be applied to any visual classifier. Additionally, our work offers method for annotating sparse autoencoder la- tents trained on any visual classifier. Finally, as method is accompa-
VIT-L/16
72.35
88.58
79.44
94.51
Swin-S
75.40
90.66
82.45
96.34
ViT-B/16,e
76.90
92.77
82.40
96.79
BeiT-B/16
77.27
92.47
84.56
97.37
BEiT-L/16
79.90
93.58
87.06
98.22
DINOv2-B
73.49
91.29
81.99
96.37
CvT-21
74.16
90.10
81.04
95.15
ConvNeXI-B
76.54
91.66
83.72
96.79
ConvNeXt-Bi
77.89
92.88
84,98
97.66
ConyNexTv2-B
77.31
91.83
84.58
97.11
with any research work, nied by its own set of limitations, which we discuss in Section A of the appendix.
ConvNexTv2-BBr
78.93
93.32
85.96
97.87
ConyNeXtv2-BB,
BBNNNeXtv2-BB3381
80.05
93.54
87.28
98.32
10

---

Published as a conference paper at ICLR 2025
ACKNOWLEDGEMENT
Fawaz Sammani is fully and solely funded by the Fonds Wetenschappelijk Onderzoek (FWO) (PhD fellowship strategic basic research ISH7W24N). N. Deligiannis acknowledges support from the Francqui Foundation (2024-2027 Francqui Research Professorship Trustworthy Al) and the "Onderzoeksprograamaa Artificiele Intelligentie (AI) Vlaanderen" programme.
REFERENCES
Peter Anderson, Basura Fernando, Mark Johnson, and Stephen Gould. Spice: Semantic propositional image caption evaluation. Computer Vision-ECCV 2016: 14th European Conference, Ams- terdam, The Netherlands, October 11-44, 2016, Proceedings, Part V 14, pp. 382-390 -92.. Springer, 2016.
Jimmy Ba, Jamie Ryan Kiros, and Geoffrey E, Hinton. Layer normalization. ArXiv, abs/1607.0645h 2016.
Satanjeev Banerjee and Alon Lavie. Meteor: An automatic metric for mt evaluation with improved correlation with human judgments. In IEEvaluation @ ACL, 2005.
Hangbo Bao, Li Dong. Songhao Piao, and Furu Wei. BEit: BERT pre-training of image transformers. In International Conference on Learning Representations, 2022.
Anders Christensen, Massimiliano Mancini, A. Sophia Koepke, Ole Winther, and Zeynep Akata. Image-free classifier injection for zero-shot classification. 2023 IEEF/CVF International Confer- ence on Computer Vision (ICCV), pp. 19026-19035, 2023.
M. Cimpoi, S. Maji, I. Kokkinos, S. Mohamed, and A. Vedaldi. Describing textures in the wild. In Proceedings of the IEFE Conf. on Computer Vision and Pattern Recognition (CVPR), 2014.
Meghal Dani, Isabel Rio-Torto, Stephan Alaniz, and Zeynep Akata. Devil: Decoding vision features into language. German Conference on Pattern Recognition (GCPR), 2023.
Yann Dauphin, Angela Fan, Michael Auli, and David Grangier. Language modeling with gated convolutional networks. In International Conference on Machine Learning, 2016.
Jia Deng, Wei Dong, Richard Socher, Li-Jia Li, K. Li, and Li Fei-Fei. Imagenet: A large-scale hierarchical image database. 2009 IEEE Conference Computer Vision and Pattern Recognition, pp. 248-255, 2009.
Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xiaohua Zhai Thomas Unterthiner, Mostafa Dehghani, Matthias Minderer, Georg Heigold, Sylvain Gelly, Jakob Uszkoreit, and Neil Houlsby. An image is worth 16x words: Transformers for image recognition at scale. In International Conference on Learning Representations (ICLR), 2021.
Kaiming He, X. Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition. 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 770-778, 2015.
Lisa Anne Hendricks, Zeynep Akata, Marcus Rohrbach, Jeff Donahue, Bernt Schicle, and Trevor Darrell. Generating visual explanations. In European Conference on Computer Vision (ECCV), 2016.
Dan Hendrycks and Kevin Gimpel. Gaussian error linear units (gelus), arXiv: Learning, 2016.
Jack Hessel, Ari Holtzman, Maxwell Forbes, Ronan Le Bras, and Yejin Choi. CLIPScore: reference-free evaluation metric for image captioning. In Marie-Francine Moens, Xuanjing Huang, Lucia Specia, and Scott Wen-tau Yih (eds.), Proceedings of the 202I Conference on Empirical Methods in Natural Language Processing, pp. 7514-7528, Online and Punta Cana, Dominican Republic, November 2021. Association for Computational Linguistics. doi: 1/.221. mmn5
11

---

Published as a conference paper at ICLR 2025
Andrew G. Howard, Mark Sandler, Grace Chu, Liang-Chieh Chen, Bo Chen, Mingxing Tan, Weijun Wang, Yukun Zhu, Ruoming Pang, Vijay Vasudevan, Quoe V. Le, and Hartwig Adam. Searching for mobilenetv3. 2019 IEEE/CVF International Conference on Computer Vision (ICCV), pp. 1314-1324, 2019.
Gao Huang, Zhuang Liu, and Kilian Q. Weinberger. Densely connected convolutional networks. 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 2261-2263 2016.
Peng-Tao Jiang, Chang-Bin Zhang, Qibin Hou, Ming-Ming Cheng, and Yunchao Wei. Layercam: Ex- ploring hierarchical class activation maps for localization. IEEE Transactions on Image Processing, 30:5875-5888, 2021. doi: 10.1 2021//TI..22110889944
Maxime Kayser, Oana-Maria Camburu, Leonard Salewski, Cornelius Emde, Virginie Do, Zeynep Akata, and Thomas Lukasiewicz. e-vil: A dataset and benchmark for natural language explanations in vision-langee language tasks. 2021 IEEE/CVF International Conference on Computer Vision (ICCV),
pp. 1224-122222 2021.
Nitish Shirish Keskar, Bryan McCann, Lav Varshney, Caiming Xiong, and Richard Socher. CTRL A Conditional Transformer Language Model for Controllable Generation. arXiv preprint (rXXiv:909.058588 2019.
Diederik Kingma and Jimmy Ba. Adam: A method for stochastic optimization. In International Conference on Learning Representations (ICLR), San Diega CA, USA, 2015.
Christoph H. Lampert, Hannes Nickisch, and Stefan Harmeling. puribute-based classification for zero-shot visual object categorization. IEEE Transactions on Puttern Analysis and Machine Intelligence, 36(3):453-463 2014. doi: 10. 109/TPAMM.20013..
Xiang Lisa Li and Percy Liang. Prefix-tuning: Optimizing continuous prompts for generation. Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 1 1th International Joint Conference on Natural Language Processing (Volume 1: Long Papers), pp. 4582-4597, 2021.
Tsung- Yi Lin, Michael Maire, Serge J. Belongie, James Hays, Pietro Perona, Deva Ramanan, Piotr Dollár, and C. Lawrence Zitnick. Microsoft coco: Common objects in context. In European Conference on Computer Vision, 2014.
Ze Liu, Yutong Lin, Yue Cao, Han Hu, Yixuan Wei, Zheng Zhang, Stephen Lin, and Baining Guo. Swin transformer: Hierarchical vision transformer using shifted windows. 2021 IEEE/CVF International Conference on Computer Vision (ICCV), pp. 9992-10002, 2021.
Zhuang Liu, Hanzi Mao, Chaozheng Wu, Christoph Feichtenhofer, Trevor Darrell, and Saining Xie. A convnet for the 2020s. 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 11976, 2022.
Ilya Loshchilov and Frank Hutter. SGDR: Stochastic gradient descent with warm restarts. In International Conference Learning Representations, 2017.
Ningning Ma, Xiangyu Zhang, Hai-Tao Zheng, and Jian Sun. Shufflenet v2: Practical guidelines for efficient enn architecture design. In Proceedings of the European Conference on Computer Vision (ECCV), September 2018.
Ana Marasović, Chandra Bhagavatula, Jae Sung Park, Ronan Le Bras, Noah A. Smith, and Yejin Choi. Natural language rationales with full-stack visual reasoning: From pixels to semantic frames to commonsense graphs. In Findings, 2020.
Sachit Menon and Carl Vondrick. Visual classification via description from large language models. International Conference Learning Representations, 2023.
12

---

Published as a conference paper at ICLR 2025
Maxime Oquab, Timothée Darcet. Théo Moutakanni, Huy V. Vo, Mare Szafraniec. Vasil Khali- dov, Pierre Fernandez, Daniel HAZIZA, Francisco Massa, Alaaeldin El-Nouby, Mido Assran, Nicolas Ballas, Wojeiech Galuba, Russell Howes, Po- Yao Huang, Shang-Wei -We Li, Ishan Misra, Michael Rabbat, Vasu Sharma, Gabriel Synnaeve, Hu Xu, Herve Jegou, Julien Mairal, Patrick Labatut, Armand Joulin, and Piotr Bojanowski. DINOv2: Learning robust visual features with- out supervision. Transactions on Machine Learning Research, 2024. ISSN 2835-8856. URL attpps://openrr
net forum?id-a68SUt 6zFt
Kishore Papineni, Salim Roukos, Todd Ward, and Wei-Jing Zhu. Bleu: a method for automatic evaluation of machine translation. In Annual Meeting of the Association for Computational Linguisticss, 2002.
Dong Huk Park, Lisa Anne Hendricks, Zeynep Akata, Anna Rohrbach, Bernt Schiele, Trevor Darrell, and Marcus Rohrbach. Multimodal explanations: Justifying decisions and pointing to the evidence. 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 8779-8788, 2018.
Alec Radford, Jeff Wu, Rewon Child, David Luan, Dario Amodei, and llya Sutskever. Language models are unsupervised multitask learners. 2019.
Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, Gretchen Krueger, and Ilya Sutskever. Learning transferable visual models from natural language supervision. In International Conference on Machine Learning (ICML), 2021.
Nils Reimers and Iryna Gurevych. Sentence-bert: Sentence embeddings using siamese bert-networks. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing. Association for Computational Linguistics, 11 2019.
Robin Rombach, A. Blattmann, Dominik Lorenz, Patrick Esser, and Björn Ommer. High-resolution image synthesis with latent diffusion models. 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 10674-10685, 2021.
Leonard Salewski, A. Sophia Koepke, Hendrik P. A. Lensch, and Zeynep Akata. Zero-shot translation of attention patterns in vqa models to natural language. In DAGM, 2023.
Fawaz Sammani and Nikos Deligiannis. Uni-nlx: Unifying textual explanations for vision and vision-language tasks. 2023 IEEE/CVF International Conference Computer Vision Workshops (ICCVW), pp. 4636 4641, 2023.
Fawaz Sammani, Tanmoy Mukherjee, and Nikos Deligiannis. NIx-gpt: A model for natural language explanations in vision and vision-language tasks. 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 8312-8322, 2022.
Ramprasaath R. Selvaraju, Abhishek Das, Ramakrishna Vedantam, Michael Cogswell, Devi Parikh, and Dhruv Batra. Grad-cam: Visual explanations from deep networks via gradient-based localiza- tion. International Journal of Computer Vision (IJCV), 128:336-3333 359, 2019.
Piyush Sharma, Nan Ding, Sebastian Goodman, and Radu Soricut. Conceptual captions: A cleaned, hypernymed, image alt-text dataset for automatic image captioning. In ACI., 2018.
Aleksandar Shtedritski, C. Rupprecht, and Andrea Vedaldi. What does clip know about red circle? visual prompt engineering for vlms. 2023 IEEE/CVF International Conference on Computer Vision (ICCV), pp. 11953-11961 2023.
Kaitao Song, Xu Tan, Tao Qin, Jianfeng Lu, and Tin-Yan Liu. Mpnet: Masked and permuted pre-training for language understanding. 2020.
Nitish Srivastava, Geoffrey E. Hinton, Alex Krizhevsky, Ilya Sutskever, and Ruslan Salakhutdinov. Dropout: simple way to prevent neural networks from overfitting. J. Mach. Learn. Res., 15: 1958, 2014.
Mingxing Tan and Quoc V. Le. Efficientnetv2: Smaller models and faster training. In International Conference Machine Learning (ICML., 2021.
13

---

Published as a conference paper at ICLR 2025
Yoad Tewel, Yoav Shalev, Idan Schwartz, and Lior Wolf. Zerocap: Zero-shot image-to-ters generation for semual-semantic arithmetic. 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 17897-17907, 2021.
Yoad Tewel, Yoav Shalev, Roy Nadler, Idan Schwartz, and Lior Wolf. Zero-shot video captioning by evolving pseudo-tokens, In 34th British Machine Vision Conference 2023, BMVC 2023, Aberdeen, UK, November 20-24, 2023. BMVA, 2023.
Laurens van der Maaten and Geoffrey F. Hinton. Visualizing data using t-sne. Journal of Machine Learning Research, 9:2579-2605, 2008.
Ramakrishna Vedantam, C. Lawrence Zitnick, and Devi Parikh. Cider: Consensus-based image description evaluation. 2015 IEEE Conference Computer Vision and Pattern Recognition (CVPR), pp. 4566-4575, 2014.
Patrick von Platen, Suraj Patil, Anton Lozhkov, Pedro Cuenca, Nathan Lambert, Kashif Rasul, Mishig Davaadorj, Dhruv Nair, Sayak Paul, William Berman, Yiyi Xu, Steven Liu, and Thomas Wolf. Diffusers: State-of-the-art diffusion models. petps::/githhe com wwt.ccom../h....cccccc http:/hhhggggfceeee/e
diffusers] 2022.
Vasilis Vryniotis. sion's
How to train state-of-the-arr models using torchvi- latest primitives, 2021.
URL https: httpss//pptorrc..orr//bogg
vee/
Sanghyun Woo, Shoubhik Debnath, Ronghang Hu, Xinlei Chen, Zhuang Liu, In-So Kweon, and Saining Xie. Convnext v2: Co-designing and scaling convnets with masked autoencoders. 2023 IEEF/CVE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 16133-16142, 2023.
Haiping Wu, Bin Xiao, Noel C. F. Codella, Mengchen Liu, Xiyang Dai, Lu Yuan, and Lei Zhang. Cvt: Introducing convolutions vision transformers. 2021 IEEE/CVF International Conference on Computer Vision (ICCV), pp. 22-31, 2021.
Yongqin Xian, Christoph Lampert, Bernt Schiele, and Zeynep Akata. Zero-shot learning comprehensive evaluation of the good, the bad and the ugly. IEEE Transactions on Pattern Analysis and Machine Intelligence, 41:2251-2266 2017a
Yongqin Xian, Bernt Schiele, and Zeynep Akata. Zero-shot learning - the good, the bad and the ugly. 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 3077-3086, 2017b.
Saining Xie, Ross B. Girshick, Piotr Dollár, Zhuowen Tu, and Kaiming He. Aggregated residual transformations for deep neural networks. 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 5987-5995, 2016.
Sergey Zagoruyko and Nikos Komodakis. Wide residual networks. In BMVC, 2016
Zequn Zeng. Hao Zhang, Zhengjue Wang. Ruiying Lu, Dongsheng Wang. and Bo Chen. Conzic: Controllable zero-shot image captioning by sampling- polishing. 2023 IEEF/VE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 33465-23477,, 2023.
Richard Zhang, Phillip Isola, Alexei A. Efros, Eli Shechtman, and Oliver Wang. The unreasonable effectiveness of deep features as a perceptual metric. 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 586-595 2018.
Zhuosheng Zhang, Aston Zhang, Mu Li, Hai Zhao, George Karypis, and Alexander Smola. Multimodal chain-of-thought reasoning in language models. ArXiv, abs/2302.00923,, 2023.
Bolei Zhou, Aditya Khosla, Àgata Lapedriza, Aude Oliva, and Antonio Torralba. Learning deep features for discriminative localization. 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 2921-2929, 2015.
14

---

Published as a conference paper at ICLR 2025
Bolei Zhou, Agata Lapedriza, Aditya Khosla, Aude Oliva, and Antonio Torralba. Places: A 10 million image database for scene recognition. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2017.
Kaiyang Zhou, Jingkang Yang. Chen Change Loy, and Ziwei Liu. Learning prompt for vision- language models. International Journal of Computer Vision, 130:2337 2348, 2021.
15

---

Published as a conference paper at ICLR 2025
A LIMITATIONS AND ETHICAL CONCERNS
All research has inherent limitations, and this study is no exception. It is important to acknowledge these constraints, We highlight two limitations of our method.
Our method may occasionally lend to focus on the general image features rather than specific ones. Figure illustrates such cases. In the first example, the "jars" in the background influence the model to produce content related to "caffeine" " and "garlic", likely due to their proximity in the embedding space the word "jars." In another instance, the upper text above the puzzle in the image resembles a list of answers, leading the model to generate content associated with "tests." Additionally, the model hallucinates that the test related to computers. Note that when the image is solely composed of the main object without other distractions such as the background, the generated NLE is usually meaningful (e.g,, an image from the same "crossword puzzle" class in Figure ( Our method can also potentially generate inappropriate language, seen in the final example. This occurs due to the proximity of certain inappropriate terms the word "cock" in the embedding space, which refers to the bird resembling the crane in the image, but can also refer to the male's genitalia.
Another limitation of our approach its slower performance compared to supervised NLEE models. Our method relies an inference-time approach, which results in the generation of single NLE taking approximately 35 seconds on a single RTX 3090 GPU. In contrast, supervised NLE models trained once, allowing for inference occur in single forward pass, making approach more cime-consuming. Although this is the case, we emphasize that our approach offers greater flexibility, operates effectively in open-set environment, and can be adapted to any example classifier an-the-fly fly. We prioritize these key factors as they are more important in NLEs, thereby outbalancing the increased explanation generation time.
prediction: pill bottle
prediction: crossword puzzle
prediction: crane
because...
because..
because...
shows caffeine glass jars and teaspoon of garlic powder in the spice jar
shows answers questions on the basics of a computer
it shows the cock p***s crane
Figure 8: Qualitative examples of failure cases (first two examples) and inappropriate language (last example) for NLEs produced.
B ZERO-SHOT IMAGE CAPTIONING
additional application. We use the MLP trained to synthesize the full 1,000 ImageNet class weights, and apply the method presented in Section 3.2 on images from the COCO dataset (Lin et al. 2014). We report results the common "Karpathy test split" benchmark using various vision classifiers. As baselines, we compare against the current state-of-the-art sy systems on zero-shot image captioning, namely ZeroCap (Tewel et al. 202 and ConZIC(Zeng et al. 2023). both which use CLIP. We use the common NLG metries for evaluation: BLEU-4 (B 44) (Papineni et al. 2002). METEOR (M) (Banerjee Lavie 2005). CiDEr (C) (Vedantam et al., (014)) and SPICE E (S) (Anderson et al.) (016). Results are presented in Table 4 Decoding features from BEiT-L achieves state-of-the-arr results on CiDEr and SPICE, which are
We present zero-shot image captioning
two most important metrics for evaluating image captioning systems. ViT-B/16, performs best METEOR, while the baseline ZeroCap leads in BLEU-4. Notably, even with a basic Re vision encoder trained ImageNet-1K 1K of 1.2 million images, approach surpasses baseline methods CiDEr and SPICE and achieves comparable results on other metrics, despite the fact that the baseline methods the far powerful CLIP vision encoder, trained on 400 million image-text pairs.
16

---

Published as a conference paper at ICLR 2025
We also find that Transformer-base models offer advantage over CNNs this task. Note that the COCO dataset differs in distribution from ImageNet terms of context and object composition in images, demonstrating the robust transferability of our method new datasets in completely zero-shot manner.
Table 4: Zero-Shot Image Captioning Results MSCOCO Karpathy Test Split
Method
B@4
M
C
ZeroCap
2.6
11.5
14.6
5.5
ConZIC
1.3
11.5
12.8
5.2
Ours
ResNet50
1.7
11.2
14.9
6.6
ResNet101
1.8
11.3
15.5
6.6
DINOv2-B
1.8
11.6
16.2
7.1
ConyNeXtV2-Bpao pt-384
2.0
11.5
16.9
7.2
ViT-B/16pr
1.7
11.9
17.4
7.4
BEiT-L/16
1.8
11.7
17.6
7.6
C ZERO-SHOT TRANSFER
We investigate the performance of our method in zero-shot transfer, scenario where model trained on one dataset can generalize to other datasets without additional training. Specifically, we use the MLP trained to synthesize the 1,000 ImageNet class weights and test its ability to synthesize class weights for other datasets. For this evaluation, we select the Places365 dataset (Zhou et al.) 2017) specializing in scene classification, and the DTD dataset (Cimpoi et al. 2014) specializing in texture type classification. These datasets are particularly challenging; for instance, the powerful CLIP model (Radford al. 2021), trained on 400 million image-tex. pairs, achieves only 41.7% top-1 zero-shot accuracy on DTD and 37.37% on Places365. We report the Test@ and T Generalized Zero-Shot Accuracy the full validation sets of the selected datasets. Notably, in this context, the generalized zero-shot accuracy on the test data is equivalent to that in the zero-shot setting, as the evaluation encompasses all unseen classes in the Places365 and DTD datasets. It is important to note that the objective of this experiment is not to surpass CLIP's performance, as our method fundamentally different and directly comparable with CLIP. Unlike CLIP, our approach is trained on text data only, without any image data, and utilizes only 1,000 samples-400,0m demonstrate the zero-shot transfer
fewer data than CLIP. Instead, the goal of this experiment capabilities of our method. Results are presented in Table 5 We observe that state-of-the-an models leveraging large-scale pretraining significantly outperform other models on this task, with BEiT-L achieving the best results. We also observe trend consistent with the existing out-of-distribution (OOD) literature: models leveraging large-scale pretraining, particularly Transformers, demonstrate superior OOD performance. This pattern holds true across both datasets.
Table 5: Zero-Shot Transfer Performance of different models Places365 and DTD datasets
Model
Places365
DTD
Test@
@
Test@1 1
@5
ResNer50
10.76
30.92
14.36
32.29
WideResNet101
13.02
33.89
12.77
31.97
ViT-B/32
13.89
34.41
15.74
33.24
ViT-L/16
14.17
33.91
15.53
32.39
ConvNeXt-Boon -ass
15.03
35.92
18.46
34.79
DINOv2-Base
16.66
40.88
17.66
40.05
ViT-B/16 (pt)
18.46
44.53
15.85
33.30
ConVNeXtV2-B (pt-384)
18.48
42.44
19.20
35,96
BEiT-L/16
19.18
43.59
20.27
37.13
17

---

Published as a conference paper at ICLR 2025
D ABLATION STUDIES
We present ablation studies learning class
embeddings, demonstrating that each compo- nent contributes non-trivially. In Table 6 start by showing results on simple baselines to ensure that our method not purely statistical and demonstrates effective learning. For this experiment, we use the ResNet-50 model and report the top-1 and top-5 accuracy for the test shown, when the class embedding of the test set is consistently set to the mean all class embeddings from the training set, the MLP fails. We also include baseline where random values are sampled from the minimum and maximum range of the train- ing class embeddings, again showing that the MLP fails. The same outcome occurs when using randomly initialized parameters for the MLP. Our main baseline uses the MSE dis- tance loss to learn class embeddings. Replacing the MSE with a cosine loss which incorporates angular information improves top-1 accuracy by approximately points. Applying dropout to the input increases top-I accuracy by points, and addition of learned soft dropout significantly enhances performance by 16 points. Next, ablate different text encoders from Reimers & Gurevych (2019) using the Sentence Transformers library As shown, the MPNet-Bate Base,2 (Song et al. 2020) provides the best results.
Table 6: Ablation studies the method in Section 4.1, using ResNet-50
Ablation
Top-1
Top-5
Learning Class Embeddings
Mean of Train Set
0.00
0..00
split used in Section 4.1
Random Values
0.00
0.20
Random Parameters
0.20
0.62
Baseline (MSE Loss)
3.12
35.56
Cosine Loss
5.86
40.64
+ Input Drop
17.24
54.38
Soft Dropout
33.28
66.20
Different text encoders
MPNet-B B- B-QAv1
24.16
57.50
DistilRoberta.
25.34
57.62
Minil MPNet-Basee
26.94
54.74
33.28
66.20
E PROMPT ANALYSIS
In this section, we analyze the impact of prompt templates used inference time. CLIP models are known to be sensitive to prompt variations (e.g., adding the letter "a" can significantly improve accuracy, as shown in Zhou al. (2021). To explore this. we evaluated ResNet50 trained with the prompt "an image of a class" on the Generalized Zero-Shot accuracy for unseen test classes using various prompts in the text encoder. Results are presented in Table 7 for several prompts, arranged in increasing order of sensitivity. Our findings reveal that, like CL.IP models, the MLP is also sensitive to prompt variations. Adding or changing one word can result in degradation of zero-shot accuracy. However, this issue can be resolved by averaging the results of all prompts, similar the approach used by CLIP.
pretrained MLP for
Table 7: Performance with different prompts at inference
Prompt
Test
Test@5
an image a {class}
33.28
66.20
image of the (class)
32.92
65.02
an image of one (class]
31.98
64.06
an image of large {class)
31.52
63.86
an image of a nice {class]
31.18
64.86
an image of weird {class)
30.90
64.24
a cropped image of a {class} a black and white image of the {class}
30.22
63.42
30.14
61.74
hhups//sshertnnet
18

---

Published as a conference paper at ICLR 2025
F ADDITIONAL MODELS ON EVALUATING LEARNED CLASS EMBEDDINGS
In this section, we provide quantitative evaluation of the learned class embeddings for additional models not included in the main manuscript, and compare them with the baseline ICIS. Results are provided in Tables and Our method consistently outperforms ICIS by a large margin on the test set in the generalized zero-shot accuracy settings, and in most cases, in the generalized setting as well.
Table 8: Quantitative Results for additional models on evaluating class embeddings via zero-shot image classification on ImageNet- compared to the baseline ICIS, for several visual classifiers
Generalized Zero-Shot Setting
Generalized Setting
Model
Method
Train
Train @
Test
@
Test@
@est@5
ICIS
80.80
95.17
3.72
37.94
44.30
68.08
ResNet50,2
Ours
79.65
94.76
11.66
44.94
44.24
67.84
ICIS
82.09
95.70
1.30
28.60
39.24
63.52
ResNet101
Ours
81.41
95.50
6.10
37.80
40.28
64.76
ICIS
81.56
95.51
1.32
30.34
41.32
65.58
WideResNee55..2
Ours
80.49
95.33
10.66
43.04
42.82
65.88
ICIS
82.99
95.78
0.84
32.86
43.04
66.60
WideResNet101,2
Ours
82.62
95.80
8.98
43.78
44.96
68.88
ICIS
81.72
95.31
1.78
31.98
40.30
66,00
ResNeXt550ge4d.w
Ours
81.07
95.06
6.16
40.66
39.42
67.92
ICIS
83.13
96.11
0.48
23.18
38.86
68.82
ResNeXt101essdd
Ours
83.02
96.00
8.00
43.86
40.32
72.34
ICIS
79.18
94.16
11.36
45.64
49.06
73.82
Res
Ours
75.88
92.96
27.44
60.78
48.84
73.88
ICIS
82.95
95,98
2.56
30.32
41,88
66.96
ResNeXt101 v2
Ours
82.32
95.91
11.46
44.12
44.12
70.04
ICIS
72.24
90.84
9.40
41.84
43.30
69.90
DenseNet121
Ours
69.08
89.99
19.24
49.24
44.38
68.04
ICIS
76.07
93.28
9.16
42.08
45.42
73.46
DenseNet161
Ours
73.00
92.54
21.44
52.88
45.88
74.26
ICIS
84.53
96.72
5.78
48.04
52.78
76.50
EfficientNetv2-S
Ours
84.35
96.62
7.24
50.40
56.90
80.18
ICIS
85.38
96.94
7.10
50.36
56.18
77.16
EfficiemtNet2--M
Ours
85.31
96.99
10.40
50.82
53.00
77.70
ICIS
76.27
92.92
1.44
26.28
39.40
70.42
ShuftleNerv2.20
Ours
74.74
92.29
7.56
38.12
40.12
70.98
ICIS
82.78
96.16
2.34
25.94
33.12
57.14
ConvNext-Ting
Ours
82.33
96.11
2.78
26.48
33.48
56.20
ICIS
84.26
96.59
3.56
38.10
44.90
68.96
ConyNNett-Small
Ours
84.13
96.64
4.76
37.70
43.54
69.08
ICIS
84.58
96.82
4.72
43.22
49.26
71.98
ConyNext-Base
Ours
84.44
96.80
5.50
45.40
51.50
74.20
G ADDITIONAL MODELS FOR EVALUATING CONCEPT DISCOVERY
In this section, we provide quantitative evaluation of concept discovery for additional models not included in the main manuscript. Results provided in Table 10 As shown, the detected concepts
19

---

Published as a conference paper at ICLR 2025
Table 9: Quantitative Results for additional models on evaluating class embeddings via zero-shot
image classification on ImageNet- IK, compared to the baseline ICIS, for several visual classifiers
Generalized Zero-Shot Setting
Generalized Setting
Model
Method
@rain
Train @5
Test
@
Test@1
Test@5
ICIS
76.12
92.16
2.22
26.98
35.60
55.10
ViT-B/32
Ours
75.99
92.25
2.66
27.42
33,46
53.52
ICIS
77.35
92.89
1,80
28.20
36.70
58,80
ViT-L/32
Ours
77.31
93.01
2.52
29.50
36.98
58.82
ICIS
79.98
94.54
3.76
34.10
44.98
66.82
ViT-L/16
Ours
80.14
94.45
2.22
35.80
46.62
67.92
ICIS
81.29
95.58
1.74
27.78
38.76
65.06
Swin-T
Ours
81,20
95.63
2.40
30.26
38.86
64.98
ICIS
83,30
96.38
4.30
38.12
42.56
66.22
Swin-S
Ours
83.17
96.35
5.46
39.48
41.64
64.32
ICIS
83.82
96.71
16.80
53.70
51.46
80.48
ViT-B/16pt
Ours
83.21
96.67
20.08
57.66
52.22
81.62
ICIS
85.18
97.34
3.70
42.72
49.72
78.32
BeiT-B/16
Ours
85.20
97.50
5.94
47.14
53.02
79.20
ICIS
77.17
94.24
17.22
44.54
43.58
69.82
DINOv2-S
Ours
76.08
94.14
21.42
49.02
46.56
71.22
ICIS
81.99
95.29
2.50
45.66
47.80
75.58
CvT-21
Ours
81.98
95.14
3.78
44.78
47.68
75.72
ICIS
85.77
97.55
6.72
42,46
50.08
72.72
ConvNexT-B
Ours
85.59
97.64
8.56
49.98
53.48
76.12
ICIS
85.38
96.99
4.28
41.16
44.98
67.62
Con-NexTv2-B
Ours
85.30
97.13
5.42
44.08
46.66
66.92
ICIS 86.77
97.77
6.08
49.16
56.34
78.46
ConvNexTv2-BB,
Ours
86.61
97.88
9.78
52.30
56.38
77.98
lead to significant improvements in accuracy the baseline, which shows the effectiveness of those concepts, and that they are faithful to the visual features.
Table 10: Quantitative Results for additional models concept discovery
Model
Baseline
+Concepts
Top-1
Top-5
Top-1
Top-5
ResNet50,,
72.85
89.78
80.25
95.31
ResNet101
73.88
89.73
81.40
95.76
WideResNet500
69.86
89.34
77.60
93.70
WideResNNe100
70.75
89.39
77.97
93.95
ResNeXt550zzauu
68.19
88.17
76.22
93.27
ResNeXt550:2.44
73.58
89.62
80.70
95.26
ResNeXt101 12x8d
71.04
89.74
78.44
94.21
ResNeX11011 12x84,
75.24
90.73
82.39
96.22
DenseNet121
64.09
85.91
71.10
90.91
ConvNext-T
74.38
89.15
81.87
96.07
ConvNext-S
76.20
90.75
83.23
96.65
Swin-T
73.32
89.09
80.66
95.58
DINOv2-S
70.61
89.62
76.94
94.45
20

---

Published as a conference paper at ICLR 2025
H ADDITIONAL MODELS FOR EVALUATING NLES
In this section, we provide quantitative evaluation of NLEs for additional models included in the main manuscript. Results are provided in Table [T] Our zero-shot NLEs outperform the supervised baselines on the CLIP-S metric, while achieving comparable or, in many cases, surpassing scores on other metrics.
Table 11: Quantitative Results for additional models on our zero-shot NLEs compared supervised baselines. Results are reported on the test set of ImageNet-X.
Model
Metric
NLX-GPT
NLX-GPT
Uni-NLX
Uni-NLX;
Ours
CLIP-S
28.53
28.52
28.38
28.52
30.70
LPIPS(A)
0.719
0.718
0.717
0.715
0.710
WideResNet50
LPIPS(S) |
0.622
0.623
0.618
0.620
0.620
Cosine 1
0.649
0.641
0.642
0.647
0.635
CLIP-S f
28.53
28.52
28.38
28.52
30.66
LPIPS(A)
0.719
0.720
0.718
0.718
0.710
WideResNett101
LPIPS(S) |
0.622
0,624
0.620
0.623
0.618
Cosine
0.642
0.640
0.641
0.640
0.633
CLIP-S
28.53
28.52
28.38
28.52
29.83
LPIPS(A)
0.720
0.718
0.719
0.717
0.716
ConyNeXtV2-B
LPIPS(S) +
0.623
0.621
0.623
0.620
0.626
Cosine
0.458
0.443
0.444
0.445
0.364
CLIP-S
28.53
28.52
28.38
28.52
30.04
LPIPS(A)
0.723
0.718
0.715
0.716
0.716
BeiT-L/16
LPIPS(S) |
0.623
0.619
0.620
0.618
0.622
Cosine
0.440
0.429
0.427
0.432
0.372
I IMPLEMENTATION DETAILS
Text-to-Image Model: For the semantic-based evaluation, we use the Stable Diffusion v1.5 model (Rombach et al., 2021) from Hugging Face Diffusers library (von Platen et al.) 2022), and precede the text prompt by the word realistic, following best practices from the text-to-image community. We perform this for the evaluation of both our method and the basclines. The NLE consists of concepts or features text format that describe the visual features (e.g., "serpentine ring on snake skin") and formed as natural language expression. Therefore, they can be formulated that the text-to-image model can understand (e.g.. "a realistic image showing serpentine rings on the snake skin."). Therefore, the synthesized image will depict image incorporating these features.
prompt
Learning Class Embeddings: We begin by reminding readers that l: represents the dimension the text features s; produced by the text encoder Mr, and is the dimensions of the class embeddings (and the visual features f). Our MLP is composed of two layers: Wi and W2. Each layer is followed by layer normalization (Ba al. 2016). At the first layer,
GELU activation function (Hendrycks & Gimpel| 2016) tollows oowlowdd weedbbe by dropout layer with probability of 0.5. When using the learned soft dropout (see Section
Wi Ex28 and W 2 Otherwise, W1 Rxx4 and W2 E Rddxd Note that the GLU activation function implementing the learned soft dropout also uses layer normalization, which we found to important.
The ML.P is trained with full batch gradient descent for 2500 epochs using the Adam Optimizer KKiggma & Ba 2015) with learning rate of 5e-3 and a cosine annealing schedule (Loshchilov & Hutter 2017 On single RTX3090 GPU, it takes roughly 10 seconds to train. All pretrained classifiers are obtained from the torchvision library. The ImageNet-21K pretrained ViT and ConvNeXt models, well all ConvNeXtv2 models are obtained from the timm library. The BeiT, DiNOv2 and CvT models are obtained from the huggingface library. We take the ImageNet class names provided by caffe-tensorflow
21

---

Published as a conference paper at ICLR 2025
Generating NLEs: We use the smallest GPT-2 (Radford al. 2019) model of 124M parameters as our pretrained language model. The number of learnable prefixes is set to in each attention block of the GPT-2 model. As mentioned previously, the prefixes are set as key-value pairs such that each generated word in the explanation can attend to these prefixes. For example, if we have Y currently generated words, then at each timestep, the number of keys and value tokens in the attention mechanism will always be (5 + Y), while the number of query tokens will be Y. Consequently, the Y query tokens will always attend both the Y key-value tokens and the 5 prefix tokens, resulting in total of (5 - Y) tokens to attend to. We train the prefixes with the Adam optimizer (Kingma & Ba 2015) with a learning rate of 0.01 and a weight decay of 0.3 with cosine annealing schedule for T= 20 iterations. Each iteration yields generated sentence, and we select the sentence with the best CLIP-Score as the final NLE. The number of K tokens sampled at each timestep is set to 512. We use maximum NLE length of 20. We follow "Twee et al. (2023) and sample one hard prompt at each iteration, as contextual information for the language model, from the following list: [ ['Image of", 'Picture of", PPotoo of", 'Image shows'. 'Picture shows'. 'Photo shows', 'Image showing', 'Picture showing', 'Photo showing'l, J. is important to clarify that our task is not traditional image captioning, as we do not train the model to explicitly map visual features annotated captions from dataset. Instead, it can be regarded as faithful image captioning, which is also faithful NLEs, since we decode the visual features sourced from the model's internal representation, into language. To maintain the language model's coherent and fluent generation, we add the fluency loss from Tewel et al. (202I) with a weight of 0.8. This is the cross-entropy loss between the language model distribution at timestep I with the prefix and without the prefix. At each decoding step, we apply several modifications to the model's output probabilities: we discourage token repetition (Keskar et al. 2019) by reducing the scores of repetitive words by factor of 2.0. We also prevent the generation of repeated n-grams of order to avoid repetitive phrases, by setting their score to negative infinity which eliminates those tokens from further sampling. We also enforce minimum sequence length of 10 tokens by setting the <.> token score to 0, in order to prevent premature termination. We also exclude specific tokens such unneeded character symbols, from being generated. Finally, the sequence length approaches the target length, we promote ending it by increasing the score of the token.
J ADDITIONAL QUALITATIVE EXAMPLES
In Figure 9 we provide additional qualitative examples of the NLEs generated for ResNet-50 classifier. It is interesting to see that in the first example from the first row, we can infer that the model has encoded in its features the association of the harvestmmm"" spider with leaves and grassy areas-an an insight that could not be uncovered using any other attribution technique. We can also see from the first example in the second row that the model associated the prediction of a "toucan" with the bird's 's colorful parts.
K GENERATING DESCRIPTORS FOR CONCEPT DISCOVERY WITH LLMS
We directly use the textual concepts provided by Menon & Vondrick (023). This work uses GPT-3.5 for generating the concepts, using the following prompt: "What are useful vi- sual features for distinguishing a (category name) in a photo?", where (category name] is re- placed by the class name in the dataset. Additionally, the work utilizes in-context examples to guide the LLM in generating structured concepts that are short and distinctive. We refer to htttps://gitte blob/man waab//aater///e pentterr/geerate..emmm wawpatteddeerrrptteeeeee py for the full script used in Menon & Vondrick
(2023) to generate concepts.
L CLASS EMBEDDINGS WITH SUPPLEMENTARY INFORMATION
We provide ablation experiment using supplementary information of the class, taken from the original WordNet hierarchy. Specifically, we obtain two types of supplementary information: the broad parent of the object, and the more specific type of it. For example, the broad parent of the class "tench" " is animal, and the specific type is fish. Similarly, the broad parent of the class "magpie" is animal and the specific type is bird. This supplementary information assists the textual encoder in
22

---

Published as a conference paper at ICLR 2025
M
A
90
91
5
RRMS
R
105
prediction: harvestman
prediction: crossword puzzle
prediction: menu
because...
becsuse...
becsuse...
it shows spider in the genus, a species grows
shows vocabulary game with
it shows items and prices, including price information product service
piterinag
prediction: toucan because...
prediction: green Uzaad
prediction: band aid
because...
It shows to be the orange, black or
showw lizerd colors in the gardens reptilian plant, is green
shows image thet is shown to
person using the product for curing healing
Figure 9; Additional qualitative examples of our zero-shot NLEs on ImageNet for ResNet-50 classifier.
better understanding the discriminative class names, particularly when it is unfamiliar with them, as the text encoder was trained on natural text that may not include these specific names.
It is important to note, however, that refrain from using any of this supplementary information in our work. This is because the trained classifiers determine the parents implicitly on their own. Utilizing this information would facilitate more structured learning and could be considered as "unfair" or "biased". However, we still provide results on this for future research that may be relevant our work. Results presented in Table 12 As demonstrated, utilizing the broad parent does not significantly impact performance. In contrast, employing the type parent increases the Top-1 accuracy by approximately 1.5 points and the Top-5 accuracy by about points. This improvement is attributed to the type parent (c.g., fish, bird) offering considerably more information to the text encoder about the object compared the broad parent (e.g., animal).
Table 12: Ablation Studies on WordNer with a ResNet-50
Top-1
Top-5
Only Class Names (Ours)
33.28
66.20
WordNet (not used)
Broad Parent Type Parent
34.82
66.40
35.16
70.82
Finally, also refrain from providing class descriptions supplementary information, because these descriptions would leak to the explanations, which the task we aim achieve. Additionally, they would leak to other downstream tasks.
23

---

Published as a conference paper at ICLR 2025
M ADVANTAGES OVER CONTRASTIVE LANGUAGE-IMAGE PRETRAINING
An alternative option to our method is aligning a text encoder with visual features f obtained from the frozen visual encoder Mv. via contrastive language-image pretraining. We provide two intuitive explanation of why our method of regressing class weights is better, efficient, and most importantly faithful:
The output distribution of the visual classification model M across all classes given by f.W, from which the prediction is then made. Therefore, the output distribution highly depends on W, and not just the visual features f. By aligning a text encoder with the visual features f. we would be changing W, they would now be the output of the newly aligned text encoder. Therefore, this approach is not faithful to the classification model, as completely changes its whole output distribution. Our method the other hand preserves the distribution by learning to regress the weights W.
2. Apart from the above fundamental problem, there significant technical challenge. Pretraining with contrastive learning requires huge set of image-caption pairs (400 million at least) and a huge amount of computational resources, in order to reach the impressive performance of CLIP models. Performing this for each different classification model is not efficient, not ideal, and not desirable for users who wish to explain their classifiers. Our method on the other hand, can be trained on any moderate GPU (or even, a nigh-performing CPU), and takes around 10 seconds, which is considered negligible and can be applied to any visual classifier on-the-fly.
N USING SOFT DROPOUT
For certain visual classifiers, we find that using learned soft dropout slightly harms performance.
We hypothesize that this occurs when the weight embedding space of the visual classifier is already well-regularized, making additional regularization detrimental. In Table 13 we provide a complete list of our models, indicating whether or not they utilize the learned soft dropout. 20/38 models use it, while the remaining 18 does not. However, we note that the negative impact is minimal, and as such, retaining this technique across all models remains acceptable.
Table 13: Usage of Learned Soft Dropout
Model
Soft Dropout
Model
Soft Dropout
ResNet50
ResNet101
ResNet50v2
ResNet101,2
Wed5
WideResNe150.,2
WideResNNt101
WideResNe::01..2
ResNeXt5502244
ResNeXt5001x44 v2
GesNext1OlsAsaa
1e
DesNeeXt1133xxdd,
DenseNet121
DenseNet161
EfficientNetv2-S
EfficientNetv2-M
ShufflcNetv22.00
ConvNeXt-Tiny
Con-NeXt-Small
ConvNeXt-Base
MobileNetv3-L.
ViT-B/32
ViT-B/16v2
ViT-L/32
ViT-L/16
Swin-T
X
Swin-S
X
ViT-B/16gc
BeiT-B/16
BeiT-L/16
X
DINOv2-S
X
DINOv2-B
X
CvT-21
X
ConvNeXt-B,
ConyNeXtV2-B
X
No
X
ConvNeXIV22-Bg..
384
X
24


Reviews:
[{"review_id": "3PJZP9diqg", "reviewer": "Reviewer_zJsF", "summary": "The paper primarily explores generative natural language explanations for vision models. Alternatively, the paper considers learning a simple MLP to retrieve text most similar to the visual embedding of a given vision model. They claim to be the first work to this in a zero-shot manner. The authors conduct extensive ablations across various models and demonstrate strong results.", "questions": "1. Did the authors ablate the prompt template? CLIP can be sensitive to prompt templates, was a similar trend seen for the learnt MLP?\n2. Were there any biases in the generated explanations? Like bias towards color or specific features?", "limitations": "", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1. The proposed method is simple and intuitive, I liked the motivation behind the work. \n2. The paper is well written and easy to follow, this makes it easy to understand what the authors have done and the current State of The Art.\n3. The authors experiment on a wide variety of models. \n4. The authors are able to extend \"Visual Classification via Description from Large Language Models\", in a practical manner to a wide variety of models.", "weaknesses": "1. Based on Table 1, the simple method seems to work less well for bigger models such as ViT and DEIT in the generalized zero-shot setting. There is a lack of discussion of the results, a little discussion about Table 1 would be helpful.\n2. Since this is a zero-shot method, it would have been cool to see how does it work on other popular yet specialized datasets like Flowers / Aircraft / CUB etc. Although ImageNet results are good having more datasets would have given more insights.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "XcVjsLkbo5", "reviewer": "Reviewer_9tNT", "summary": "The paper proposes a new method for zero-shot natural language explanation (NLE) for classification. The method trains an MLP that maps the encoding space of an LLM text encoder to the space of class weights of arbitrary classifiers. Consequently, the method works on arbitrary classifiers. It produces NLE by iteratively adapting a prefix to the image classifier’s feature representation, and the prefix serves as a condition for generating text descriptions. In this way, the generated text is faithful, in contrast to the case of models trained on human description data specifically. It achieves the state-of-the-art performance on zero-shot NLE, and is also on par to the SOTA methods in zero-shot classification and concept discovery.", "questions": "1. The author might consider improving the presentation in section 3.2 by making the process more clear.\n2. The author might want to analyse the cause and give one explanation for the inferior \"cosine\" metric in NLE results.\n3. I wonder if this method can be applied to directly improve the robustness of image classifiers.", "limitations": "", "rating": 8, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 4, "strengths": "1. The method proposed by the paper is novel, taking a completely different perspective from that of the previous works in the field of NLE. \n2. The method is powerful, in that it achieves the state-of-the-art performance on zero-shot NLE, and is also on par to the SOTA methods in zero-shot classification and concept discovery. \n3. The method also demonstrates and utilises the possibility of matching the representation space of LLM to arbitrary vision classifier (or any models whose outputs are texts), which gives deep insights into this mechanism and can be inspiring to other fields as well.\n4. Overall, the presentation of the paper is very clear.", "weaknesses": "1. Section 3.2 (as well as Section F) is somewhat confusing to me. From my understanding, the generation of NLE is the following process: 1. there are K sentences starting with some initial prompts like “this is a”; 2. a random prefix is present to generate top-K tokens; 3. the top-K tokens are concatenated to the K sentences to get K continued sentences; 4. the representation of these sentences are compared to the image representation, and the prefix is updated to reflect the similarities, so that next step it can generate better tokens that make the sentences aligned to the image; 5. the whole process continues so that the sentences become longer. I still have the following doubts (I will update my score and confidence once I understand them better; I also think Figure 4 should be more clear on this whole process):\n    1. At the first time step, the sentences will become “this is a [object]”. How to make sure the top-K tokens actually contain the correct object, so that the continuation does not follow a completely wrong object?\n    2. How is it possible for the generated sentence to automatically have “because” in it, so that the NLE is generated to be “this is a [object], because…”\n    3. In Section F, I am a bit confused between “iterations” and “time steps”\n    4. In Section F, I am not sure why the CLIP-score is used to choose the sentences. Shouldn’t it be that the sentence is chosen based on its similarity to the image representation? And is it fair to use CLIP-score here when comparing to other methods?\n2. In the NLE comparison results, it seems that with the “cosine” metric the proposed model seems to be systematically inferior than the best models. The author might want to analyse the cause and give one explanation.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "vCfXwNxpVw", "reviewer": "Reviewer_Y6Z6", "summary": "This paper presents a zero-shot natural language explanation approach. \nThe proposed method involves two sequential steps. \nFor the first step, the authors propose to align the text encoder with that of the classifier from a visual encoder.\nSubsequently, an LLM is then equipped with a prefix that is trained by the pseudo label learned from the former stage.\nThe experiments are primarily focused on the NLE task. \nWhen compared with several baselines, the proposed method achieves improved performance on diverse vision backbones.", "questions": "One more question other than the aforementioned weaknesses: \n\nHow about the model performance using a tiny vision large language model (VLLM)?", "limitations": "", "rating": 6, "confidence": 3, "soundness": 2, "presentation": 3, "contribution": 2, "strengths": "- The studied problem, zero-shot natural language explanation, is quite practical and useful for many researchers.\n- The writing of this paper is good, and most parts of it are easy to follow.\n- The authors utilize a variety of vision encoders to validate the generalization capability of the proposed method.", "weaknesses": "- My biggest concern is why we simply remove the first stage and use the CLIP model for the alignment.\nTo me, the authors try to align the output from a text encoder with that of a vision encoder classifier. \nThis process is utilized to offer a pseudo label for the subsequent LLM prefix-tuning.\nI feel like we can directly use a pre-trained CLIP as a replacement which can generalize well to open settings.\n- Following the first concern, how did the authors address out-of-distribution classes? \nIt seems the classifiers are trained on 1,000 ImageNet classes.\nWhat if there are more fine-grained classes?\n- It is controversial to call this method a ``classifier-agnostic`` method. \n  - The authors did use a classifier in the first training stage.\n  - The MLE is evaluated on the ImageNet domain.\n- There is only 1 baseline compared in Table 1.\nAdditionally, the improvement over this method is not significant.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "4ke4eiYzWj", "reviewer": "Reviewer_N3TX", "summary": "The paper proposes a method to use natural language to explain the decisions made by an image classification model. To achieve this, they train a small MLP to map the text features to the feature space of the image classifier. Then, they use a pretrained language model and apply prefix-tuning, fine-tuning a small set of trainable parameters (added to the key-value pairs in the attention module). This enables the language model to generate explanations for a given image sample, using the image and the image classifier to extract its features.", "questions": "- Regarding the last paragraph of section 3.2: If you attach each of the $K$ tokens to the previously sampled top-K tokens, how would there be $K$ possible sentences? Wouldn't it be $K^2$?\n- In the first paragraph of section 4.1:\n  - What is the number of classes in $\\hat{W}$? (What is the dimension of $\\hat{W}$?)\n  - When training the MLP, which classes did you use? Only the 900 classes, or all of them?\n- As a metric to evaluate your method, the cosine similarity between the text embeddings of the predicted explanation and the ground truth explanation (from ImageNet-X) using the BERT model is interesting and missing. Have you tried this?\n- Do you have an intuition about why your model performs worse than other supervised models on ViT models (even though it performs better on CNNs)? Do you have any experimentations to support your intuition?", "limitations": "", "rating": 6, "confidence": 5, "soundness": 3, "presentation": 2, "contribution": 3, "strengths": "- Provides a method for generating explanations for image classifier decisions using natural language.\n- Does not require large-scale training data.\n- Is inexpensive to train (only a small MLP and small prefixes in the attention mechanism).\n- Includes good visualizations.\n- Has strong paper motivation.\n- Shows good performance on CNNs compared to supervised methods.", "weaknesses": "- Some parts of the paper are vague. For example:\n  - The section discussing learnable prefixes set as key-value pairs in the attention modules. Could you please add one sentence to this part explaining how this works?\n  - The last paragraph discussing zero-shot NLE generation (If you attach each of the $K$ tokens to the previously sampled top-K tokens, wouldn’t there be $K^2$ possible sentences instead of $K$?).\n- It requires optimization for each image sample to produce an NLE.\n- It requires loss calculation and backpropagation for each image sample to produce an NLE.\n- Although the model performs well on unseen classes, it deteriorates performance on seen classes.\n- LPIPS is not a suitable evaluation metric for this application because the NLE is not intended to generate captions for images (as T2I models require). Therefore, passing NLEs generated by this model to a T2I model to generate images does not make sense.\n- No limitations are mentioned in the main text of the paper.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)