LICO: Explainable Models with Language-Image COnsistency
Authors: Yiming Lei
Zilong Li
Yangyang Li
Junping Zhang
Hongming Shan
Venue: NeurIPS 2023 poster (2023)
Keywords: Language-image consistency
prompt learning
image classification
CNN interpretation
TL;DR: We propose a Language-Image COnsistent (LICO) learning scheme that incorporates class-aware prompts for facilitating CNN-based image classification, generating improved classification performance and explainable attention maps.

Abstract: Interpreting the decisions of deep learning models has been actively studied since the explosion of deep neural networks. One of the most convincing interpretation approaches is salience-based visual interpretation, such as Grad-CAM, where the generation of attention maps depends merely on categorical labels. Although existing interpretation methods can provide explainable decision clues, they often yield partial correspondence between image and saliency maps due to the limited discriminative information from one-hot labels. This paper develops a Language-Image COnsistency model for explainable image classification, termed LICO,  by correlating learnable linguistic prompts with corresponding visual features in a coarse-to-fine manner. Specifically, we first establish a coarse global manifold structure alignment by minimizing the distance between the distributions of image and language features. We then achieve fine-grained saliency maps by applying optimal transport (OT) theory to assign local feature maps with class-specific prompts. Extensive experimental results on eight benchmark datasets demonstrate that the proposed LICO achieves a significant improvement in generating more explainable attention maps in conjunction with existing interpretation methods such as Grad-CAM. Remarkably, LICO improves the classification performance of existing  models without introducing any computational overhead during inference.

LICO: Explainable Models with Language-Image COnsistency
Yiming Lei', Zilong Li', Yangyang Li", Junping Zhang', Hongming Shan
Shanghai Key Laboratory of Intelligent Information Processing, School of Computer Science, Fudan University Academy of Mathematics and Systems Science, Chinese Academy of Sciences
3 Institute of Science and Technology for Brain-Inspired Intelligence and
MOE Frontiers Center for Brain Science, Fudan University
Shanghai Center for Brain Science and Brain-inspired Technology
(ymlei, jpzhang, hmshan] @fudan. edu. cn, zi zilongli230m. fudan. edu. cn, yyli0amss ac
Abstract
Interpreting the decisions of deep learning models has been actively studied since the explosion of deep neural networks. One of the most convincing interpretation approaches salience- based visual interpretation, such Grad-CAA CAM, where the generation of attention maps depends merely categorical labels. Although existing interpretation methods can provide explainable decision clues, they often yield partial correspondence between image and saliency maps due to the limited discriminative information from one-hot labels. This paper develops a Language- Image COnsistency model for explainable image classification, termed LICO, by correlating learnable linguistic prompts with corresponding visual features in a searse-to-fine manner. Specifically, structure alignment by minimizing the distance between the distributions of image and language features. We then achieve fine-grained saliency maps by applying optimal transport (OT) theory to assign local feature maps with specific prompts. Extensive experimental results eight benchmark datasets demonstrate that the proposed LICO achieves a significant improvement in generating more explainable attention maps in conjunction with existing interpretation methods such as Grad-CAM. Remarkably, LICO improves the classification performance existing models without introducing any computational overhead during inference. Source code is made available at https: // thub. com/
first establish coarse global manifold
Introduction
Although deep neural networks (DNNs) have shown excellent performance in many fields, the lack of interpretability is still barrier landing in some high-stakes scenarios such as medical diagnosis, autonomous driving, etc. Therefore, the literature proposes various interpretation methods for DNNs extent.
and reveals the decision clues of DNNs
Popular interpretation methods can be roughly categorized into two types: 1) gradient back- propagation-basee based, and 2) class activation mapping based. Both of them mainly take image classification as the pretext task and then generate explainable noisy gradients and saliency maps, respectively. As illustrated in Fig. l(a), CAM-based methods often explore a better weighting scheme for integrating feature maps of given input image. The gradient-based methods, in Fig. ((b), also
"Corresponding author.
37th Conference on Neural Information Processing Systems (NcurIPS 2023).

---

Feature
Maps
logis
Back-progugttion
Weightng
Prompt embooking
Text
Encoder
Prompts
CNNS
CNNs
CNNs
Input
Input
Input
(a) Post-hoc CAM-based methods
(b) Post-boc Strdient-basee methods
(6) LICO (ours)
Figure 1: Motivation of LICO. (a) Existing post-hoc CAM-based methods focus on generating better weighting schemes to obtain weighted-sum attention maps. (b) Gradient- based methods tend to propppre gradients from logits to input space. (c) Proposed LICO incorporates learnable
prompts to enable image features to approximate semantic information in latent space.
start from the output logits while back- prockpapppgginggggg gradients to the input space. However, they are all post-hoc approaches that investigate DNN pre-trained specific dataset [1, 2, 3, 4, 5, 6, 7], and yield biased interpretations due to the limited semantic information that set of one-hot labels offer. In addition, one-hot labels often trigger overfitting of the pre-trained model that the effectiveness of existing interpretation methods could be compromised. On the other hand, the latent feature space of such pre-trained models is unable to sense real semantic space reflecting crucial parts in images. Therefore, may be futile to explore complex post-hoc techniques for interpreting a pre-trained model.
Inspired by advanced large vision-language models (VLMs) such as CLIP [8], which developed generalized image and text encoders through training huge amounts of image-text pairs, in this paper, we assume that the large VLMs can encode real-world semantic knowledge through contrastive vision-language alignments, whereas the traditional models pre-trained on relatively small datasets, such as ImageNet- 1k, are inferior in capturing the true semantic information.
Motivation. The DNN-based image classification framework generally consists of convolutional neural network (CNN) for feature extraction and a linear layer acting as classifier. Since feature representations are used as the input to the classifier, if the DNN is truncated from feature representa- tions, it becomes linear classifier that aims classify feature representations linearly into discrete one-hot label space. More specifically, the feature representation lies in the learned manifolds of high-dimensional semantic space [9]. Unfortunately, training using cross-entropy loss with one-hot labels cannot guarantee that the manifolds of image features can reflect the distribution of real-worls semantic knowledge, which hinders performance improvement of existing interpretation methods.
In this paper, we leverage language information from large VLMs to enhance current interpretation methods for achieving more explainable saliency maps while enabling promising classification performance improvements; see Fig. I(c). First, we propose Language-Imager LinageCCOnsistent (LICO) learning to facilitate the alignment between the manifolds of visual features and class-aware language information. To address the discrete nature of categorical classes, we construct learnable prompt for each class and map all prompts into continuous space using the CLIP text encoder, which is feasible to align manifolds of both image and text. Second, we impose each prompt token to correlate with certain feature maps. Considering that the feature maps are redundant to the final classification decision, we propose to encourage the context tokens to guide certain feature maps through distribution alignment using optimal transport (OT) theory.
Contributions. We summarize the main contributions of this paper as follows. (i) We propose novel framework enhance current interpretation methods by introducing language guidance from large VLMs. (ii) We model the discrete categorical class to continuous space by constructing class-aware learnable prompts, hence, enabling consistent manifold matching between image features and text features. (iii) To ensure consistent local feature alignment, we utilize OT theory to reduce distance between distributions image and text. (iv) Extensive experimental results on eight classification datasets demonstrate the superiority of the proposed LICO against current interpretation methods terms of quantitative and qualitative results. For practical applications, LICO does not introduce any computational overhead during inference while maintaining improved performance.

---

(4) End-to-end DNN clussificution
(b) Language Knowledge
A2
Classifier
CC
X
96
(x:)
Image:
Feature Extraction
Classilication
Procapt: 4
gp
(e) Coarse Manifold Matching and Fine Optimal Transporr Aligument
Parameter updated
Parameter frozen
Image encoder fa
Cor
9
Text encoder
h4
MLP
Language
Lum
MM loss
Image Manifold
Prompt Manifold
Lor
OI loss
Figure 2: Framework of the proposed LICO. (a) Conventional classification pipeline DNNs. (b) Language feature extraction with pre-trained text encoder. (c) Manifold matching among samples and optimal transport alignment between feature maps and prompt tokens within each sample.
2 Related Work
Interpretation methods. Class activation mapping (CAM) is simple approach generating
class-aware saliency maps for CNNs [1]. Inspired by CAM's effectiveness, its variants including Grad-CAM [2], Grad-CAM+ [3], Score-CAM [4], RISE [10], and Group-CAM [5], were proposed to enhance the precision and interpretability of CNNs. Gradient-based methods such as Guided Back-propatation [11], SmoothGrad [7], and Integrated Gradient (IG) [6] compute the gradients from output logits back to input space. However, they are prone to yielding attribution noise that inconsistent with human intuition. Some advanced techniques have been developed reduce attribution noise by utilizing improved path integration theory, such as adaptive path method [12] and important direction method [13]. Distinct from existing interpretation methods that focused on the post-pne post-processing of feature map weighting scheme and accurate gradients calculation based on model, our LICO aims incorporate class-aware learnable prompts that reflect real- world semantic knowledge to guide latent image features. It is worth noting that LICO is compatible with current methods and can consistently improve their performance.
Prompt learning. Prompt learning stems from natural language processing (NLP), enabling large language models to directly model the prediction probability raw text [14, 15, 16]. Following the proliferation of large VLMs, prompt learning began to surface in the realm of computer vision, significantly enhancing multi-modal tasks and text-guided image generation. To overcome the limitations of learning with fixed prompts used in CLIP, learnable prompts have been explored to provide more generalized performance on downstream tasks. CoOp firstly proposed framework that constructs learnable prompts to align image and text in latent space [17]. Furthermore, to enable CoOp with a stronger capability to perceive the knowledge of new classes, the authors proposed CoCoOp in which the prompts are learned conditioned on image features [18]. PLOT correlated one given image with multiple prompts of its class by optimal transport, leading different prompts to reflect features with respect to different regions within this image [19]. In our LICO, we propose that learnable prompt tokens in one sentence should be registered to certain parts of an image, as shown in Fig. ((c).
Optimal transport. Optimal transport (OT) is a typical metric for measuring discrepancy be- tween two distributions, requiring solving a linear programming problem [20, 21]. Thanks to some fast alternatives, OT has been widely utilized deep learning, including graph cross-domain alignment [22, 23], domain adaptation [24, 25, 26), optimal graph matching for vessel image registra- tion [27], and multi-modal distribution alignment [28, 29). Although KL-ddivergence can effectively measure the cost of which the predicted distribution approximates ground-truth distribution, given the normalized image and text distributions, intractable to guarantee they share a common metric space. Therefore, propose utilize OT to tackle fine-gran ine-grained cross-modal alignment.

---

3 Methodology
3.1 Overview of LICO
Fig. 2 presents the overview of LICO framework. Given a mini-batch of B training samples {(a,, Vi, Bi.t:));=1 the imput image x; is of size x W x C, where is the number of channels, Il and W are height and width, respectively. y/ C. denotes the class label, and & represents the text of corresponding label yt. denote the pre-trained CLIP text encoder as gø while we fixed its parameters & during training. The image encoder is denoted as fo that is to be optimized. The image encoder takes I; as input and outputs feature maps F, with N channels. Note that F; is a tensor, which is then flattened to one-dimension for each channel; i.e., F3 E RNxx' For the language modeling branch, the text encoder I6 takes input the constructed learnable prompt: t [X,, X2, X2...... -M- fz]. where !...] denotes the concatenation, & is the text corresponding to label of the i-th sample, X,, are learnable context tokens, and M 1 is the number of context tokens. Through the pre-trained text encoder, the prompt vector 4; is mapped into a continuous d-dimension space; i.e., d 512 in CLIP.
In order to measure the distances between image and language features in latent space, we append a mapping net hgg multilayer perceptron (MLP) with only one hidden layer, to map the d-dimension language features t; E xd to the space of d' d'-dimension, i.e., G, E R xd' which is the same as that of image features F, The structure of hes varies along different image encoders. We use hy (a, b] describe structure, i.e., numbers of hidden units and output units are and respectively. Note that in this paper, we do not intend to modify the structure of the existing image and text encoders so that we can obtain the feature maps and corresponding attention maps that are comparable existing methods. Consequently, we utilize F; and G; to preserve the latent space using manifold matching (Section 3.2) and fine-grained laagure- alignment by optimal transport (Section 3.3).
structure in
3.2 Language-Image Manifold Matching
To preserve the global manifold structure in latent space, we first measure the relationships, i.e. the distances similarities, among training images and corresponding class-aware prompts.
Although the class categories are discretely distributed that it is infeasible to capture their manifold, we can map them into continuous space by constructing prompt vector for each class, then mapping it to latent space using a pre-trained text encoder. Therefore, we can enforce the image manifold [30] to approximate the language manifold.
In practice, align the adjacent matrices of language and image features. Specifically, the language adjacent matrix is denoted as A B and that of image is A A
exp( exp(-D(E ((F, F2)/T)
exp(-D(G,, G₁)/T) A E exp(- G.)/T)*
(I)
Σ B s=1 ((F,, F.)/T)
where D) calculates the distance between two images two prompts, such Euclidean distance, 7 is the temperature to be learned during training. Then, each image is formulated as a distribution A.. where each dimension denotes the distance from the i-th sample to others within mini-batch. Similar to the class-wise prompts, AC implies the relationships between the class prompt of the i-th sample and those of others.
Recall our assumption that the large amounts image-text pairs used in large VLMs, such as CLIP, can lead to generalized text encoder that well establishes a real-world semantic space. Therefore, for certain downstream tasks and datasets, we aim to introduce the knowledge of this semantic space to the latent space of the target image domain. Then, we propose manifold matching loss LMM. which enables the manifold of image features to approach that of prompt features using KL--diergence:
LMM
Σ B
1
B
KL A.
We note that we not consider the inverse version KT, A [|]]] since LICO focuses on enabling image manifold to approach the manifold of language prompts.

---

3.3 Feature Distribution Alignment by Optimal Transport
While we strive for coarse alignment of the global manifold structure, is crucial establish correlation between prompt tokens and specific feature maps for each sample, which aids in mitigating the influence of redundant features on the generation of attention maps. Most importantly, the critical feature maps that are most related to the target class should possess the highest similarity with respect to the class token.
Unfortunately, it is challenging to determine or assign which of the N feature maps are correlated with certain prompt tokens. In this paper, we propose to align G; = : . IM [t,]] E IR M and F; [f, fN] E NRx for achieving consistency between feature maps and specific prompt tokens. In other words, different words in a sentence should correspond to parts of an image, and in contrast, partial regions in an image reflect the semantic information delivered by certain tokens. Therefore, is intractable to measure this distance using KL-divergence since is not strict metric, i.e., it does not satisfy the property of triangle inequality. In LICO, we use optimal transport, which is widely used in measuring distances of two distributions, to align distributions of normalized visual features and prompt features.
For the given image feature maps F; E Ng and a prompt tokens G; E IR Mxd" we construct two discrete distributions:
n=1 Un und fn:
8 gm.
m=1
where 8g., is a Dirac function centered at fn, as to 8g..: and the weights u = (u } E AN and [um]m=1 AM, A denotes the N- and M -ddimensional probability simplex, i.e., Em Un
1, mm= Um 1. Finally, the discrete OT distance for one sample is defined as follows:
M
Dor(u, inf Ej (1p.o)==[C(ff))))) [C(fgg)] TE))(A))) n= m=1
min
ΣΣ
Tm m c(fns
(4)
S.F. Tlm 44 T1, =
where C' E NxM represents the cost matrix in which each element e(fn, Im) denotes transportation cost between fn and 9m. T E xNxM is the transport plan that is to be optimized and 11(p, v) denotes the transportation polytope that contains all joint probabilities of μ and In practice, solving the optimization problem in Eq, (4) often equips with high computation cost. Thus we use the Sinkhorn algorithm, which is more computationally amenable, to solve an entropy-constraine problem [31]:
N
M
DoT v) = min
Tn.m c(fns 9m) (HI(T), S.t.
T1m = M, T1, = v. (5)
TETT(A.))
where À is Lagrange multiplier and H(T) En,m TogllogTm.o. n.m- Then after few iterations, we obtain the optimal solutions:
T" diag((" ==xiiq("""exp(-C///
(6)
where is the iteration step. μ³' and v' are updated according to following rules: o' ==(exp(--C/) p²))
n' = [//exp((C/A)oo-))
Dynamic context (DC). To endow each image with diverse prompt tokens, shuffle the learnable context tokens in each training iteration referring to the training procedure in Algorithm 1.
3.4 Final Objective Function
The final training loss function is as follows:
C LcE + aLMM BLoT.
(8)
where Lce is the cross-entropy loss, Lor = ΣE Dor, and 3 are hyperparameters for adjusting different terms. During the inference phase, we apply the trained image encoder and classifier to conduct conventional classification that yields the predicted probability of a given input image. Note that the text encoder and the MI. mapping not affect the inference procedure. The detailed algorithm can be found in Algorithm 1.
5

---

Algorithm Training Algorithm of LICO.
Require: Training set S, total epochs U, image encoder fo. text encoder Iø. MLP hi, learnable prompts t; = 11, Xy......... X 1,s-1,4,, 1z].
Return: Image encoder with optimal parameter 0*
1: for u = 1 to U do
2: Sample mini-batch of (far,, 44 Wi from S.
3: Randomly shuffling Xm and 4.
D. Dynamic context Image and text features Adjacient matrices
4:
F- fo;;), G. fyy (t)(tt).
5:
Calculate A B A B
6:
Calculate LMM according to Eq. (2).
>> Coarse alignment by manifold Optimal transport plan T by Eq. (6), then calculate Lor. Fine-grained alignment by OT
7:
8: Classifier t
F;
9: Total loss: C= LCE + aLMM + Borr update 0, sb and Xm, by gradients:
0C Oc OC
00 dej ax
end for
4 Experiments
4.1
Datasets
This paper focuses on image classification task and evaluates the proposed LICO on well-known datasets, including ImageNet-Ik [32], CIFAR-10/100 [33], and SVHN [34]. We conduct the classifi- cation experiments under the setting of limited training data in which the splits of labeled data follow the previous works for fair comparison [35, 36). Furthermore, we conduct fine-grained classification on typical benchmarks, including CUB-200 [37]. FGVC-Aircraft [38], Stanford Cars-196 [39], VGG Flowers [40]. The evaluation is carried on both the full dataset and few-shot settings, following the procedure of CGC [41].
4.2 Implementation Details
We employ the ViT-B/32 trained by CLIP [8] the text encoder and its parameters are fixed during training. The output dimension of this text encoder is 512 for each token. The image encoders in our experiments vary along different datasets and training settings. Specifically, for the ImageNet experiments, we utilize ResNet-50 as the image classifier [42]. In doing so, the convolutional layers preceding the final linear layer constitute the image encoder of LICO. For the CIFAR 10/100 and SVHN, we follow the experimental settings in [35, 36), the classification network is Wide ResNet (WRN) [43]. We further conduct the same experiments using another network, the PreActtResNet-II 18 (PARN-18) [44]. For fine-grained classifications
CUB-200, Standford-Cars, Aircraft, and VGG Flowers datasets, we applied the same settings used CGC for fair comparison [41], where the image encoder is also the ResNet-50 that has been pre-trained on ImageNet- l with CE loss. Note that for all the experiments of LICO, we only use the trained image encoder and the classifier during the inference phase. The text encoder, learnable prompt contexts, and MLP mapping net are dropped, thus, will not compromise the computational efficiency. Hyperparameters a and are set as 10 and respectively.
All the experiments are implemented by PyTorch [45]. The learning rates for ImageNet, CIFAR- 10/100, and SVHN are of 0.03 with a consine rate decay schedule, i.e., v) = ?o cos( Iak ( ). where тo denotes the initial learning rate and le is the index of training step [46]. We use standard stochastic gradient descent (SGD) optimizer with momentum of 0.9 [47, 48], and the weight decay is 0.0001. The training batch sizes are 128 and 64 for ImageNet and other datasets, respectively. Specifically, the mapping net for ResNet-50 hep [512, 49), h [512, 64] for WRN, and h,p [512, 49) for PARN- 18. The total training epoch is 90 for ImageNet and 200 for others. The experiments were trained on four NVIDIA A100 GPUs for ImageNet- 1k and one GPU for other datasets.
4.3 Comparison of Interpretation Capability
In this experiment, we compare interpretation results of popular methods, including Grad-CAM [2]. Grad-CAM+ [3], RISE [10], Score-CAM [4], Group-CAM [5], and CGC [41]. Note that LICO compatible with these post-hoc interpretation methods so that in our experiments, compare the

---

Grad-CAM
RISE
Score-CAM
Grropp-AAM
Grad CA3-6.22
Original
Stoen dale 13.90
0.4
Grss-CAt $4.55
2.6
Stree CAMLLCCO
+ +
GrAADCANL Col
Scatn-CAn 02.51
CANI
0.2
0.0
0.6
0.8
0.0
1.
Caree
Derecion Curna
Original
1.0
1.0
0.2
Group CAM-JCO, 3.49
CAM:
Score CAN
Greas CAR 72.02
0.4
4551 LEO
LICO
Greas CANLICA 06
0.2
SCORIICAM
0.2
13.29
0.0
0.2
0.6
0.6
0.8
1.0
0.0
0.2
0.4
0.6
0.8
1,0
Foodiie Carez
Dereciin Caree
Original
1.0
1.0
CraskCA JCD
Criup Croup-Gat:72.j
Gmoup-GMM-aC0
CAM LKCB
LiCO
0.4-
Greas-Cukt 93.67
Grad-ciM
0.4
K.6 LKCL
Add 21.54
Scar CAs
6:56: 66-33
Mise-udo 00.12
0. 0,6
0.8
1.0
0.0
0.0
1.5
Figure 3: Saliency maps (Left) and Insertion/Deletion curves (Right) on ImageNet- Ik validation set.
saliency maps obtained by these interpretation methods using baseline ImageNet pre-trained model and LICO trained counterparts of the same architectures.
Qualitative results. Qualitatively, we compare the saliency maps obtained by different interpreta- tion methods in Fig. 3 (Left). We can that for single object images, LICO can effectively help baseline methods cover more comprehensive and discriminative regions, e.g., the head and fur of bird. However, the baseline methods are inferior in capturing all the objects in the multi-object image. Please refer to Appendix A for more results of multi-class and multi-object cases.
Table 1: Quantitative comparisons of different interpretation methods on ImageNet in terms of Insertion and Deletion. We report the results: "Baseline/+LICO", Overall Insertion Deletion.
Method
Grad-CAM++
Grad-CAM
RISH
Score-CAM
Group-CAM
CGC
Insertionf
50.0/51.2
53.5/57.1
54.0/54.9
55.1/55.6
56,8/55.2
52.2/55.4
Deletion,
14.8/11.7
13.3/15.1
11.7/10.8
11.5/11.2
12.3/10.5
-/15.8
Overallf
35.2/39.5
40.2/42.0
43.6/44.1
42.3/44.4
44.5/44.7
-/39.6
Quantitative results. To achieve quantitative evaluation, we follow [10, 4, 5] to conduct Insertion and Deletion tests. Insertion gradually introduces class-related regions (3.6% pixels) of an original image to a blurred image according to the values of the saliency map. This process is repeated until the blurred image is fully recovered. In contrast, Deletion aims to replace related pixels (3.6%) in a blurred image with those of the corresponding original image. In Table 1, we provide the AUC of the classification score after Softmax. For most the interpretation methods, LICO consistently improves the Insertion and Deletion values. Although the insertion value of Group-CAI and deletion value of Grad-CAM is better than LICO, the LICO still achieves the best overall values. We also report the quantitative results of corresponding cases with different interpretation methods in Fig. (Right). Please refer to Appendix B for the experiment of Pointing Game [5].
Sanity checks. Sanity check for saliency maps was first proposed in [49], which is qualitative test aiming at evaluating whether the saliency maps are sensitive model parameters. We conducted two types of test: cascading randomization from top to bottom layers and independent randomizing of
7

---

Inout Looft
Convd4
Conv2
Logii
Conid4
Convd2
Convdo
Convd8
Scom Scoe-CA
U.0
Cascading randomiztioon
Indopendent randomizatin difeerent
Figure 4: Sanity checks evaluated using ImageNet pre-trained V9G-19 and LICO trained version. Each column represents saliency maps after randomizing corresponding layers following two ran- domization schemes: (a) cascading randomization from top bottom layers, and (b) independent randomization of different layers.
different layers. In this experiment, Score-CAM is selected as the baseline method, and the backbone network is VGG-19. In Fig. 4, we can see that Score-CAM sensitive the model parameters. and the saliency maps obtained by LICO-trained model also pass these two tests. From Fig. 4(a) we can see that the saliency maps of LICO are more sensitive model parameters along cascaded randomization, e.g., from Logit to Conv34, then to Conv32.
4.4 Classification Performance
In addition to interpretation comparison, we compare the classification performance on various datasets since LICO can act as regularization for enhancing image classifier. According previous studies [1, 41, 50], good interpretation method often sacrifices the discriminative ability, i.e., lower classification accuracy than baselines. As shown in Table we compare non-post- non-post-hoc methods GCC [50] and CGC [41] with our LICO, Despite their emphasis on interpretation ability, both GCC and CGC experience performance degradation compared to the CE loss pre-trained model. In contrast, LICO surpasses the baseline model and achieves the highest Top-l and Top-5 accuracy values, a notable advan- lage not shared by previous interprelation methods. Although LICO introduces relatively few amounts of training parameters, i.., MLP hy and prompt tokens X which is trivial to the enhancement of both interpretation and classification perfor- mance. Furthermore, it does not affect model architectures and introduces any computational overhead the inference phase. Please refer to Appendices C and D for further evaluation of LICO using ViTs and the potential in regularizing semantic space obtained by t-SNE, respectively.
Table Classification accuracy (%)
on ImageNet using ResNet.
Method
Top-1
Top-5
ResNet-50
76.13
92.91
+GCC [50]
74.40
92.12
+CGC [41]
74.60
92.24
+LICO
76.27
92.99
ResNet-18
69.76
89.08
+GCC [50]
67.74
88.52
+CGC [41]
66.37
88.27
+LICO 69.83
89.21
Training with limited data. Table 3, we evaluate the performance of LICO with limited training data. Following [36, 35], we conduct experiments on CIFAR-10/101 and SVHN with different label amounts. Note that we only use limited labeled data for training and do not focus whether LICO is useful under semi- aumi-supervieed setting. Across all the limited training settings, LICO consistently improves both PARNI8 and WRN basclines. Notably, in the full training setting. LICO exhibits the largest improvement on CIFAR-100 100, which is attributed to the effectiveness of prompt guidance introduced by manifold of larger amounts of classes.
Table 3: Classification accuracies (%) on CIFAR- 10/100 and SVHN datasets using different amounts of labels. "Full" means training with all labeled data.
Dataset Labels
10
CIFAR- 100
SVHN
40
250
4000
400
2500
10000
40
1000
PARNI8
22.94023
32.9
77.5
5.2+
23.7 4017
54.9
14.74o27
56.2
+LICO
23.7
36.14as1
80.1
+000
5,8
21.44044
56.5
16.0
59.5
WRN
23.4 36.9
80.8
8.64a.17
32.3
60.7 40.24
15.94az3
58.9
+LICO
24.9 40.42
41.7
81.5
10.7+
35.5
62.0
17.44034
63.7
WRN
95.6410.2
80.9 +0.18
97.9
+LICO
95.8
82.2
98.3

---

Table 4: Fine-grained classification accuracy under full and few-shot settings.
Method
I-shot
5-shot
10-shot
Full
I-shot
5-shot
10-shot
Full
CUB-200
Standford-Cars
Baseline
+3.7
51.7+003
66.4402
80.1
6. +0.2
34.3+0o
61.1
89.7+au
+CGC [41]
15.8
55.2+03
68.4403
81,5401
6.5
36.5
63. 03.0
90.3
+LICO
16.9
55.8
68,4
82.7
7.1
37.2
64.4
91.5+02
Aircraft
VGG Flowers
Baseline
7.7
25.7.04
41.4201
83.7
$2.1
85.6104
93.22a2
96.1102
+CGC [41]
8.0403
26.9404
42.9403
85.7
53.3
85.8
93,4
96.2+00
+LICO
8.4
27.5
43.1
85.6402
55.6+00
86.2
93.4
96.8
Fine-Grained classification. Furthermore, we evaluate LICO on a fine-grained classification prob- lem that requires the model to capture fine-grained features. In Table 4, except for the Aircraft, LICO enhances classification performance under all settings compared with CGC. This observation high- lights that contrastive learning among attention maps used in CGC primarily emphasizes similarities between target images and other random/augmenter images, while disregarding the measurement of semantic distinctions. For the Aircraft dataset, it is difficult for LICO to achieve significant improvements due to the categorical labels are types of aircraft, e.g., the numerical symbols such 727-200 which are challenging for CLIP text encoder achieve meaningful embedding, and CLIP has demonstrated relatively lower accuracy on some out of distribution data like aircraft and satellite [8]. To address the challenges, initial prompts as "a type of aircraft", which allows LICO to improve performance in the few-shot settings while achieving comparable results CGC under the full setting. This strategy has also been verified in CoCoOp [18].
incorporate prior text knowledge by constructing
4.5 Ablation Study
Table 5: Ablation on CMM and LoT.
LCE
EMM
Lot
Top-1
Top-5
Insert.
Delet.
Ablation on manifold matching and OT alignment. In Table 5, we inves- tigate the effectiveness of LMM and Lor- We can see that only using Lor obtains more performance drop than that of LaM- This indicates that the global consistency of manifolds guarantees the basic performance, and OT only focuses on local feature alignments so that cannot be sensitive to relationships between intra- and inter-class samples.
76.13
92.91
53.5
13.3
75.98
92.92
56.6
16.0
76.18
92.90
56.9
15.5
76.27
92.99
57.1
15.1
evaluate the performances influenced by different numbers of learnable tokens. We find that 12 is the best choice in our experiments, and the settings 16 and 20 are relatively better than those of 4 and 8.
A Ablation number of context tokens. In Table
Ablation on distance function in Eq. (1). In Table 7, we compare the similarity functions used in manifold matching. which measures distances among samples within a mini-batch. We can see that the Euclidean distance is more suitable to our LICO.
Table 6: Ablation of context tokens.
Table 7: Ablation on distance function.
no.
Top-1
Top-5
Insertion
Deletion
Dataset
Euclidean
Cosine
75.64
91.92
54.1
17.8
CIFAR-10
95.78
95.46
76.03
92.74
55.2
17.5
CIFAR-100
82.22
81.85
76.09
92.89
56.3
16,0
SVHN
98.25 4:006
98.2 +oos
12
76.27
92,99
57.1
15.1
CUB-200
82,70
82. 10
16
76.21
92.87
57.0
15.8
Flowers
96,80+aa0
96,20+0.40
76.14
92.93
56.9
15.5
Please refer to Appendices E, F, and for more ablation studies on DC, effects of different type of text encoders, and frozen parameters of prompts, respectively.

---

5 Conclusion
In this paper, we proposed LICO enhance existing visual interpretation methods by incorporating language information, which is compatible with these methods. The LICO aligns image and prompt embeddings globally using manifold matching, simultancously aligns feature maps with correspond- ing learnable context tokens by applying optimal transport alignment. Extensive experiments on evaluating interpretation capability and classification performance exhibit both quantitative and quali- tative enhancement introduced by LICO. A key limitation of LICO is that it depends on trainable MLP project language embeddings into metric space of same dimension with that of image features, where the output dimension of such MLPs varies according to different image encoders.
Broader Impacts LICO explores effective visual interpretations of DNNs by introducing language knowledge, which orthogonal to existing post-hoc interpretation methods. An important merit of LICO is to enhance interpretability while achieving competitive or better classification performance, applicable to various kinds of tasks and models effectively.
Acknowledgements
This work supported in part by Shanghai Municipal Science and Technology Major Project (No. 2018SHZDZX01) and ZJLab, Natural Science Foundation of Shanghai (No. 21 21ZR 1403600), National Natural Science Foundation of China (Nos. 62101136, 62306075), China Postdoctoral Science Foundation (No. 2022TQ0069), Young Elite Scientists Sponsorship Program by CAST (No. 2022QNRC001), Shanghai Municipal of Science and Technology Project (No. 20JC1419500), and Shanghai Center for Brain Science and Brain-inspired Technology.
References
[1] Bolei Zhou, Aditya Khosla, Agata Lapedriza, Aude Oliva, and Antonio Torralba. Learning deep features for discriminative localization. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 2921-2929, 2016.
[2] Ramprasaath R Selvaraju, Michael Cogswell, Abhishek Das, Ramakrishna Vedantam, Devi Parikh, and Dhruv Batra. Grad-CAM: Visual explanations from deep networks via gradient- based localization. In Proceedings of the IEEE International Conference Computer Vision, pages 618-626, 2017.
[3] Aditya Chattopadhay, Anirban Sarkar, Prantik Howlader, and Vineeth N Balasubramanian. Grad-CAM+H Generalized gradient-based visual explanations for deep convolutional networks. In 2018 IEEE winter Conference on Applications of Computer Vision, pages 839-847, 2018.
[4] Haofan Wang, Zifan Wang, Mengnan Du, Fan Yang. Zijian Zhang, Sirui Ding, Piotr Mardziel, and Xia Hu. Score-CAMe Score-weighted visual explanations for convolutional neural networks. In Proceedings of the IEEE/CVF Conference on Computer Vision and pattemn Recognition Workshops, pages 24-25. 2020.
[5] Qinglong Zhang, Lu Rao, and Yubin Yang. Group-CAM: Group weightee visual explana- tions for deep convolutional networks. arXiv preprint arXiv:2103. 2021.
[6] Mukund Sundararajan, Ankur Taly, and Qiqi Yan. Axiomatic attribution for deep networks. In International Conference on Machine Learning, pages 3319-3328, 2017.
[7] Daniel Smilkov, Nikhil Thorat, Been Kim, Fernanda Viégas, and Martin Wattenberg. Smooth- grad: removing noise by adding noise. arXiv preprint /706v:
2017.
[8] Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et al. Learning transferable visual models from natural language supervision. In International Conference on Machine Learning, pages 8748 8763, 2021.
[9] Yipei Wang and Xiaoqian Wang. propagation explanations. Advances in Neural Information Processing Systems, 35:9085-9097, 2022.
"why not other classes?": Towards class-contrastive back-
10

---

[10] Vitali Petsiuk, Abir Das, and Kate Saenko. RISE: Randomized input sampling for explanation of black-box models. In British Machine Vision Conference, page 151, 2018.
[11] Jost Tobias Springenberg, Alexey Dosovitskiy, Thomas Brox, and Martin Riedmiller. Striving for simplicity: The all convolutional net. International Conference on Learning Representations, 2015.
[12] Andrei Kapishnikov, Subhashini Venugopalan, Besim Avci, Ben Wedin, Michael Terry, and Tolga Bolukbasi. Guided integrated gradients: An adaptive path method for removing noise. In Proceedings of the IEEE/CVF Conference Computer Vision and pattern Recognition, pages 5050-5058, 2021.
[13] Ruo Yang. Binghui Wang, and Mustafa Bilgic. IDGI: A framework to eliminate explanation noise from integrated gradients. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2023.
[14] Pengfei Liu, Weizhe Yuan, Jinlan Fu, Zhengbao Jiang, Hiroaki Hayashi, and Graham Neubig. Pre-train, prompt, and predict: A systematic survey of prompting methods in natural language processing. ACM Computing Surveys, 55(9):1-35, 2023.
[15] Fabio Petroni, Tim Rocktäschel, Patrick Lewis, Anton Bakhtin, Yuxiang Wu, Alexander H Miller, and Sebastian Riedel. Language models as knowledge bases? arXiv preprint arXiv:1909.01066, 2019.
[16] Alec Radford, Jeffrey Wu, Rewon Child, David Luan, Dario Amodei, Ilya Sutskever, et al. Language models are unsupervised multitask learners. OpenAl blog. 188):9, 2019.
[17] Kaiyang Zhou, Jingkang Yang, Chen Change Loy, and Ziwei Liu. Learning to prompt for vision-language models. International Journal of Computer Vision, 130(9):3337-2348 2022.
[18] Kaiyang Zhou, Jingkang Yang, Chen Change Loy, and Ziwei Liu. Conditional prompt learning for vision-language models. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 16816-16825, 2022.
[19] Guangyi Chen, Weiran Yao, Xiangchen Song, Xinyue Li, Yongming Rao, and Kun Zhang. PLOT: Prompt learning with optimal transport for lision-language models. In The Eleventh International Conference on Learning Representations, 2022.
[20] Gaspard Monge. Mémoire sur la théorie des déblais et des remblais. Mem. Math. Phys. Acad. Royale Sci., pages 666-704, 1781.
[21] Gabriel Peyré, Marco Cuturi, et al. Computational optimal transport: With applications to data science. Foundations and Trends Machine Learning, 1 11(5-6)355-6077 2019.
[22] Hermina Petric Maretic, Mireille El Gheche, Giovanni Chierchia, and Pascal Frossard. GOT: an optimal transport framework for graph comparison. Advances in Neural Information Processing Systems, 32, 2019.
[23] Liqun Chen, Zhe Gan, Yu Cheng, Linjie Li, Lawrence Carin, and Jingjing Liu. Graph optimal transport for cross-domain alignment. International Conference on Machine Learning, pages 1542-1553. PMLR, 2020.
[24] Fan Zhou, Zhuqing Jiang, Changjian Shui, Boyu Wang, and Brahim Chaib-draa. Domain gener- alization via optimal transport with metric similarity learning. arXiv preprint arXiv:2007.10573 2020.
[25] Shengsheng Wang, Bilin Wang. Zhe Zhang, Ali Asghar Heidari, and Huiling Chen. Class-aware sample reweighting optimal transport for multi-source domain adaptation. Neurocomputing, 523:213-290 22-33, 2023.
[26] Rosanna Turrisi, Rémi Flamary, Alain Rakotomamonjy, and Massimiliano Pontil. Molti-source domain adaptation via weighted joint distributions optimal transport. In Uncertainty in Artificial Intelligence, pages 1970-1980 2022.

---

[27] Danilo Motta, Wallace Casaca, and Afonso Paiva. Vessel optimal transport for automated aligmment of retinal fundus images. IEEE Transactions on Image Processing, 1511))6155- 6168, 2019.
[28] John Lee, Max Dabagia, Eva Dyer, and Christopher Rozell. Hierarchical optimal transport for multimodal distribution aligmment. Advances in Neural Information Processing Systems, 32, 2019.
[29] Shraman Pramanick, Aniket Roy, and Vishal M Patel. Multimodal learning using optimal transport for sarcasm and humor detection. In Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pages 3930-3940, 2022.
[30] Richard Socher, Milind Ganjoo, Christopher D Manning, and Andrew Ng. Zero-shot learning through cross-modal transfer. Advances in Neural Information Processing Systems, 26, 2013.
[31] Marco Cuturi. Sinkhorn distances: Lightspeed computation of optimal transport. Advances in Neural Information Processing Systems, 26, 2013.
[32] Olga Russakovsky, Jia Deng, Hao Su, Jonathan Krause, Sanjeev Satheesh, Sean Ma. Zhiheng Huang, Andrej Karpathy, Aditya Khosla, Michael Bernstein, al. Imagenet large scale visual recognition challenge. International Journal of Computer Vision, 115:211-2523 2015.
[33] Alex Krizhevsky, Geoffrey Hinton, et al. Learning multiple layers of features from tiny images. 2009.
[34] Yuval Netzer, Tao Wang. Adam Coates, Alessandro Bissacco, Bo Wu, and Andrew Y Ng. Reading digits in natural images with unsupervised feature learning. 2011.
[35] Bowen Zhang, Yidong Wang. Wenxin Hou, Hao Wu, Jindong Wang. Manabu Okumura, and Takahiro Shinozaki. FlexMatch: Boosting semi-supervised learning with curriculum pseudo labeling, Advances in Neural Information Processing Systems, 18408-1448111 2021.
[36] Kihyuk Sohn, David Berthelot, Nicholas Carlini, Zizhao Zhang, Han Zhang, Colin A Raf- fel, Ekin Dogus Cubuk, Alexey Kurakin, and Chun-Liang Li. FixMatch: Simplifying semi- supervised learning with consistency and confidence. Advances in Neural Information Process- ing Systems, 33:596-608, 2020.
[37] Peter Welinder, Steve Branson, Takeshi Mita, Catherine Wah, Florian Schroff, Serge Belongie, and Pietro Perona. Caltech-uesd birds 200. SENS-TR-2011-001, 2010.
[38] Subhransu Maji. Esa Rahtu, Juho Kannala, Matthew Blaschko, and Andrea Vedaldi. Fine- grained visual classification of aircraft. arXiv preprint arXiv:1306.514 2013.
[39] Jonathan Krause, Michael Stark, Jia Deng, and Li Fei-Fei. 3D object representations for fine- grained categorization. In Proceedings of the IEEE International Conference on Computer Vision Workshops, pages 554-561, 2013.
[40] Maria-Eles Elena Nilsback and Andrew Zisserman. Automated flower classification over large number of classes. In 2008 Sixth Indian Conference on Computer Vision, Graphics & Image Processing, pages 722-729. 2008
[41] Vipin Pillai, Soroush Abbasi Koohpayegani, Ashley Ouligian, Dennis Fong, and Hamed Pir- siavash. Consistent explanations by contrastive learning. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 10213-10222, 2022.
[42] Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for im- age recognition. In Proceedings of the IEEE Conference Computer Vision and Pattern Recognition, pages 770-778, 2016.
[43] Sergey Zagoruyko and Nikos Komodakis. Wide residual networks. In British Machine Vision Conference, 2016.
[44] Kaiming He, X. Zhang, Shaoqing Ren, and Jian Sun. Identity mappings in deep residual networks. In European Conference on Computer Vision, 2016.
12

---

[45] Adam Paszke, Sam Gross, Francisco Massa, Adam Lerer, James Bradbury, Gregory Chanan, Trevor Killeen, Zeming Lin, Natalia Gimelshein, Luca Antiga, al. PyTorch: An imperative style, high-performance deep learning library. Advances in Neural Information Processing Systems, 32, 2019.
[46] Ilya Loshchilov and Frank Hutter. SGDR: Stochastic gradient descent with warm restarts. arXiv preprint arXiv: 03983, 2016.
[47] Ilya Sutskever, James Martens, George Dahl, and Geoftrey Hinton. On the importance of initial- ization and momentum in deep learning. In International Conference on Machine Learning, pages 1139-1147, 2013.
[48] Boris Polyak. Some methods of speeding up the convergence of iteration methods. Ussr Computational Mathematics and Mathematical Physics, 4(5):1-17, 1964.
[49] Julius Adebayo, Justin Gilmer, Michael Muelly, Ian Goodfellow, Moritz Hardt, and Been Kim. Sanity checks for saliency maps. Advances Neural Information Processing Systems, 31, 2018.
[50] Vipin Pillai and Hamed Pirsiavash. Explainable models with consistent interpretations. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 35, pages 2431-2439, 2021.
13

---

Appendix
A
More Results of Saliency Maps
LICO localizes more discriminative local features. Fig. provides more saliency maps, showing that LICO is able to localize fine local features of target objects, e.g., the foot and head in Fig. ((c).
Input
Brad-CAM
RISE
Score-CAM
Group-CAM
3.0
Oriiha
Gil LAN dE
6.0
Oree CANe $3A8
ScloCC UO0 36.00
6/1
Dawu Caan-1
LA-
SELLO 39
+0000
Smaw 23
02
10.33
2.0
merden Cale
Halitie Daree
Croa-aan S0i4
Serm CAN
Semerraa 25.8
On 18
Cuice
Origial
Crecn LOS
Crock Clae
1.2
1.4
LCCO
022
2.0
24
Figure 5: More samples on ImageNet-Il 1.. For each image, we provide saliency maps of different methods and Insertion/Deletion curves. LICO helps localize fine local features.
More comprehensive locations yield lower metrics. Here, we provide some failure cases obtained by our LICO, where the saliency maps of LICO exhibit more comprehensive locations target objects while resulting in lower AUC values of Insertion and higher AUC values of Deletion compared with baseline methods.
For the cases with multiple target objects Fig. 6, LICO localizes different objects separately and comprehensively, which demonstrates the consistency between language knowledge and visual features established by our LICO. However, LICO obtains inferior Insertion/Deletion evaluations, which is paradoxical to human cognition. We conjecture that the main reason is attributed the fixed text encoder, which is not specifically optimized for target tasks, i.e., the images with a single object occupy higher percentage of the dataset such as ImageNet-1k: Therefore, the located multiple objects may be harmful to saliency maps-guide -uide pixel-level insertion and deletion strategies.
In Fig. 7, we provide some examples that contain multi-class objects in one image, and for the target classes, LICO obtains more accurate localizations. Fig. 8, we verified the effectiveness of LICO on capturing more objects of sinlge class within one image, and we can see that LICO captures more comprehensive objects than those without LICO.
B Pointing Game
To verify the localization ability LICO, we conducted the pointing game on MS COCO 2017 validation set for localization evaluation. Following settings in Score-CAM and Group-CAM, quantified localization by calculating Hits- Misses assessing salient pixels fall within the annotated bounding boxes. Table shows that LICO consistently improves all the baseline interpretation methods, indicating the effectiveness of regularization by the proposed manifold OT losses.
Hits
14

---

Seare-GAM
Group-GAM
SOANTE Carre
Orama
SCANN DE
Clas JECS
1000
24
00
0.0
Original
One MCD
0.5
02
62
Oranaa
66
26:
LCO
Can Mo0 vo
24
06
20
04
26
08
Figure 6: Saliency maps of failure cases with multiple target objects ImageNet- Ik. For each image, we provide saliency maps of different methods and Insertion/Delction curves.
Imput
Grodi CAM
Score-CAM
Group-CAM
Input
Grad CAM
Score CAM
Group CAM
Originl
Criatnll
Tangel Clesse Broom
Target Class: setter
LCCO
Original
Original
Target Class: Teapot
Target Class:
Figure 7: Saliency maps of multi-class cases.
C Evaluation on ViT
We further trained ViT-Base-16 network on ImageNet- Ik dataset and presented the corresponding accuracy, insertion, and deletion in Table 9. We calculated the LM and for between language tokens and representations of patch tokens and class tokens. For attention maps, we applied Grad-CAn CAM in LICO-trained ViT-Base-16 16 by calculating gradients from outputs to the last attention layer of the class token. Table 9 further confirms that transformer model with LICO not only performs better in classification but also gains better interpretability than the one without LICO, which is in line with the finding for the CNN-based backbone.
D t-SNE Visualization of Latent Features
We further explore how LICO can regularize the latent space. In Fig. 9, we provide the t-SNE visualization of learned visual features on CIFAR-10 and SVHN. In Fig. 9(),, our LICO enables different classes with similar distribution shapes. Interestingly, if roughly categorize the ten
15

---

Score-CAM
Group-CAM
UCC Orignal
Original
Class: Busby
Class: Teapot
Class:
docking
Figure Saliency maps single-class and multi-object cases.
Table 8: Pointing game evaluation on MS COCO 2017 val set.
LICO Grad-CAM
Grad-CAM++
RISE
XRAI
Score-CAM
Group-CAM
56.740.221
57.2
54.3
10.222
51.040211 40.55
57.5
56.9
58.1
55.2
=0.29
58.2
classes of CIFAR-10 into classes, i.e., "animal" and "vehicle", LICO almost linearly classify these two classes, whereas the baseline obtains non-linear classification boundary (dashed lines). In Fig. 9b)), LICO makes the latent space exhibit manifold structure with respect to semantically ordinal classes such as from "1" to "4", and some semantically closer classes are also closer in latent space such "9" and "'10". However, the baseline method unable to capture the underlying manifold structure and we can see obviously that "7" is far away from "5", "6", and "8", so "9" and "10", which should be closer in nature. In Figs. (()) and (d). LICO also shows more discriminant results under limited training data.
LICO
W/O
Manifal
CIFAR-10
(b) SVHN
w/o
WKO
w/UC
[c] 10,
data
[d] CIFAR: 10,
data
Figure t-SNE results obtained by the models with and without our LICO. (a) and (b) are results of full setting on CIFAR-10 and SVHN, respectively. (c) and (d) show the results under the setting of limited training data.
16

---

Table 9: Evaluation on ImageNet- Ik using ViT-Base- 16.
Model
Accuracy
Insertion
Deletion
ViT-Base-16 16 w/o LICO
77.9
55.2
14.4
ViT-Base-16 16 w/ LICO
78.2
56.0
13.8
Table 10: Ablation on DC.
DC
ImageNet
CIFAR-I0 10
CIFAR- 100
SVHN
X
76.22
95.45.0.09
81.3
98. 10.10
76.27
90.80
82.2
98.3
E
Ablation Studies
Ablation on dynamic context (DC). the training algorithm LICO, we designed a dynamic context (DC) for learnable context. We provide some results ablation of DC in Tables 10 and 11. We can see that, for most of datasets we evaluated, DC consistently improves the classification performance. For few-shot settings in Table 11, DC obtains slight performance improvements, which attributed to few samples of each class. That is to say, there are insufficient visual feature variety to endow context vectors with various information.
Table 11: Ablation on DC on four fine-grained datasets.
CUB-200
Standford-Car
DC
1-shot
5-shot
10-shot
Full
1-shot
5-shot
10-shot
Full
16.9410
55.8
68.2
82.5
7.0403
37.1
64.1 104
90.3
16.9 10.2
55.8 10.8
68.4 10.4
82.7 0.3
7.1 10.
37.2 10.5
64.4 14.4
91.5 102.
Aircraft
VGG Flowers
DC
1-shot
5-shot
10-shot
Full
1-shot
5-shot
10-shot
Full
8.0
27.4.0.
42.8 102
85.6
55.3
85.7
92.7.02
96.2
8.4
27.5
43.1
10.4
85.6
55.6
86.2
93.4
96.8
Variation of KL-divergence values between prompts and visual features f()). Here, provide curves of KL-divergence with and without LICO. The KL values obtained by baseline w/o LICO are calculated between CE-trained visual features and fixed CLIP pre-trained vectors, and those obtained by w/ LICO are calculated between visual features and prompts learned by LICO. In Fig. 10, we can that LICO enables stable descent of KL values, and this indicates that the LICO learned visual features indeed approach the learned language prompts. However, the values obtained by baseline without LICO fluctuate to extent and cannot decrease further, which is attributed to the domain gap between CLIP pre-trained data and target CIFAR-10. 10. In other words, without manifold matching, is difficult to enable downstream visual features be aligned with CLIP pre-trained language knowledge.
F Different Text Encoders
Both Word2Vee (W2V) and BERT can replace the CLIP text encoder in our LICO framework. However, they may be inferior in acting language guidance in LICO: (1) CLIP text encoder was trained with huge amounts of image-text pairs, and the advanced architecture ViT-B/32 has stronger representation ability than W2V; (2) Even though BERT excels at representing language data, latent representation of BERT may not align with target image representations well; (3) Recent studies on prompt learning like CoCoOp has demonstrated the effectiveness of CLIP pre-trained text encoders for downstream tasks.
In Table 12, we conducted the experiments using W2V text encoder, mapping texts into 512-D. W2V: using fixed word embeddings and no context tokens for OT loss. W2V-P: replacing class tokens in the prompts, the context tokens are randomly initialized Gaussian. We can that W2V
17

---

Values of KL-divergence on CIFAR-10 test set
w/ LCO
w/o LICO
Kt. valuus
0.-
230
250
epoch
Figure 10: Variation of KL-divergence value w/ and w/o LICO on CIFAR-10 test set.
performed comparably to the None encoder baseline. W2V-P also fails, where the fixed random context tokens mislead the image features. In contrast, the CLIP outperforms both of them. The fixed W2V embedding vectors struggled to correlate well with target image features. Moreover, BERT surpassed W2V and W2V-P due to the generalizability of stronger pre-trained model. However, BERT still cannot achieve better performances than CLIP. Furthermore, BERT-ALIGN performs competitively and even better than CLIP, which is attributed to its larger training with image-text pairs. Consequently, from the results in Table 12 and this paper, we conclude that LICO works better with those vision- Laiiuae pre-trained text encoders. Pure text encoders like W2V. even the pre-trained BERT with frozen parameters, are inferior in image-text alignment in LICO because the pre-trained parameters are not sensitive to visual features. This deficiency may be addressed by utilizing some transfer learning and domain adaptation tricks.
noisy
Table 12: Comparison of different text encoders on CIFAR-10. 10.
Encoder
CLIP
W2V
W2V-P
BERT
BERT-ALIGN
None
Full
95.8
95.6
94.9
95.7
95.8
95.6
4000
81.5
81.0
80.2
81.3
81.7
80.9
G Frozen Parameters of Prompts
To evaluate the model performance with frozen parameters, we then conducted experiments on CIFAR-10 and ImageNet under two settings of frozen parameters: (1) frozen random initialization (Random) and (2) fixed form of 'a photo of a [CLS]' (Fixed). The insertion and deletion values are obtained by Grad-CAM + LICO.
In Table 13, we can see that the frozen random parameters cannot enable the models to achieve higher performances of accuracy, insertion, and deletion; the reason is that the well-trained CLIP text encoder is capable of sensing the human-unde uumaan--uddrsttddabble phrases and sentences while the random prompts lead to the difficulty image-text alignment and yield inaccurate semantic representations. However, the form of 'a photo of a [CLS]' performs better than frozen random parameters because this meaningful prompt more consistent with the input of the original CLIP so that the generated representations can be easily aligned with image representations.
Table 13: Evaluations on frozen parameters of prompts. "Ace." is short for "Accuracy".
ImageNet
Top- l
Insertion
Deletion,
CIFAR-
Full, Ace.
4000, Acc.
Random
75.88
53.3
17.8
Random
94.9
79.5
Fixed
76.20
55.2
17.4
Fixed
95.4
80.7
Learnable (ours)
76.27
57.1
15.1
Learnable (ours)
95.8
81.5
18


Reviews:
[{"review_id": "ltHdNgTpfy", "reviewer": "Reviewer_tr3q", "summary": "This paper introduces Language-Image-COnsistent (LICO) to get better interpretation for classification using the Vision-Language model. The proposed framework uses a frozen text encoder and a trainable image encoder to encode text and image information. The text is composed of several trainable prompt tokens and the text label for image classes. Then, the manifold matching (MF) loss is used to align the image feature latent space with the text feature latent space. In addition, another Optimal Transport (OT) loss is used to build a fine-grained correlation between prompt tokens and image features.", "questions": "see weakness", "limitations": "yes", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1. The added text encoder during training time can introduce new information from the text encoder to the image encoder, but won't influence the inference procedure. Therefore, the inference procedure will keep the same as conventional classification models. \n2. The insertion and deletion tests are used to validate the generated model interpretations from the proposed method, and LICO outperforms previous interpretation methods such as GradCAM and RISE in most of the cases. \n3. The authors conduct experiments on several image classification benchmarks. For ImageNet, LICO obtains higher accuracy than the baselines. For other benchmarks such as CIFAR and SVHN, LICO achieves better performances than the baselines using limited amounts of labels. For fine-grained benchmarks such as Aircraft, LICO also shows better performances under full/few-shot settings in most of the cases.", "weaknesses": "1. Did you compare the training time with the baselines? The inference time will be similar, but the training process will involve an extra text encoder. Therefore, it may take longer time to train the model and more space as well. \n2. In table 6, do you have the result for 0 learnable context tokens?", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for your insightful and valuable feedback, pushing us to rethink more comprehensive experiments. \n\n**Q1: Training time comparison**\n\nThank you for pointing out this issue.  We completely agree that the training time should be compared and discussed.\n\nPer your suggestion, we reported the training time of the models with and without LICO on different datasets in **Table R1**. We can see that LICO requires more training time due to the additional forward process of the text encoder and MLP.  We argue that given the model's better interpretability and classification performance, the additional training cost is acceptable and compromised as the model training can be done offline. We will certainly add this discussion to the future version and investigate a more efficient training strategy to improve the training efficiency of LICO in the future.\n\n**Table R1**: Comparison of training time (sec. per epoch) with and without LICO\n| ResNet-50 | ImageNet | Aircraft | Flowers | - | WRN | CIFAR-10 | CIFAR-100 |\n| :---: | :---: | :---: | :---: | :---: | :---: | :---: | :---: |\n| w/o LICO | 1200 | 60 | 84 | - | w/o LICO | 80 | 160 |\n| w/  LICO | 1850 | 86 | 123 | - | w/  LICO | 146 | 371 |\n\n**Q2: About 0 learnable context tokens in Table 6**\n\nWe appreciate your insightful comments. We first would like to note that  '0 learnable context tokens' indicates the setting of **no learnable parameters in prompts**, i.e., only the single class tokens are used.\n\n We provided the performance of the model learned with 0 context tokens in **Table R2**, which are relatively worse than others. This is partly due to the facts: (1) the original text encoder of CLIP was trained with prompt engineering rather than single-class tokens, and (2) CLIP has validated the superiority of learning with prompt engineering over single-class tokens. Our results are consistent with the finding in the literature that prompt learning methods like CoOp [1] and CoCoOp [2] have also demonstrated the effectiveness of learning with trainable context tokens when adapting to downstream tasks. We will update the result of Table 6 in the future version.\n\n**Table R2**: Ablation on no. of context tokens\n\n| no. of context tokens | Top-1 $\\uparrow$ | Top-5 $\\uparrow$ | Insertion $\\uparrow$ | Deletion $\\downarrow$ |\n| :---: | :---: | :---: | :---: | :---: |\n| 0 | 75.64 | 91.92 | 54.1 | 17.8 |\n| 4 | 76.03 | 92.74 | 55.2 | 17.5 |\n| 8 | 76.09 | 92.89 | 56.3 | 16.0 |\n| 12 | **76.27** | **92.99** | **57.1** | **15.1** |\n| 16 |  76.21 | 92.87 |  57.0 | 15.8 |\n|20 | 76.14 | 92.93 | 56.9 | 15.5 |\n\n> [1] Kaiyang Zhou, Jingkang Yang, Chen Change Loy, and Ziwei Liu. Learning to prompt for vision-language models. International Journal of Computer Vision, 130(9):2337–2348, 2022.\n>\n> [2] Kaiyang Zhou, Jingkang Yang, Chen Change Loy, and Ziwei Liu. Conditional prompt learning for vision-language models. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 16816–16825, 2022."}, {"review_id": "H94L8Vhvpo", "reviewer": "Reviewer_rFY8", "summary": "Most visualization interpretation methods based on saliency information often generate inaccurate saliency maps due to the limited discriminative information provided by one-hot labels. The manuscript proposes a language-image consistency model (LICO) to address this challenge. LICO utilizes a large-scale vision-language model CLIP to improve existing CAM-based vision explainable methods such as Grad-CAM. The authors assume that numerous image-text pairs used in CLIP can encode rich semantic information that can be aligned with the latent space of image domain, which establishes global manifold structure alignment and assigns local feature maps with class-specific prompts to generate more accurate saliency maps. The paper is well-organized and motivated, and the idea of leveraging language information from large VLMs is intuitive and effective. Experiments including deletion and insertion tests, sanity checks and classification demonstrate that LICO can achieve improvements over existing interpretation methods, resulting in more explainable attention maps.", "questions": "(1)\tThe total loss function in algorithm 1 is inconsistent with that in Eq. (8). The Loss ‘LKL’ in algorithm 1 may need to be changed to ‘LMF‘.\n(2)\tTable 2 shows a decrease in classification accuracy for the base model ResNet when combined with GCC and CGC. The previous work cited suggests that good explanatory methods often sacrifice discriminative abilities. It would be helpful to explain how the proposed methods exhibit both good explanatory and discriminative abilities.\n(3)\tIn Table 5, experimental comparisons based on Lce should also be carried out in order to better investigate the effectiveness of ‘LMF’ and ‘LOT’.\n(4)\tWhat is the effect of MLP used in the text encoder? Since the predicted probability only relies on the trained image encoder and classifier during inference, does there exist another more efficient way of processing the language features?", "limitations": "N/A", "rating": 5, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "(1)\tThe paper is well-organized and easy to understand.\n(2)\tThe motivation behind leveraging language information from large VLMs is clear and effective.\n(3)\tThe use of prompt information and multi-modal techniques on explainable methods may be with good value. ", "weaknesses": "(1)\tThe effectiveness of LICO requires further verification. The sensitivity of LICO's saliency to model parameters raises concerns about its locality and uncertainty. Evaluations of more complex images and assessing the model's robustness would be beneficial.\n(2)\tThe results of the saliency map are not sufficient to support the conclusion of better coverage of comprehensive and discriminative regions. Comparisons with more complex multi-target and multi-class images are preferable.\n(3)\tThe details of the ablation studies are not clearly presented. The saliency maps generated by CAM-based methods + LICO may be insufficient to qualitatively show the salient regions that the classification model focuses on. It would be better to additionally provide results of “multi-objects single-class” and “multi-class”, except for “single-object”. Moreover, it would be better to conduct additional experiments to evaluate the segmentation and localization performance of LICO for a more comprehensive comparison with other explanation methods.\n(4)\tSome basic experiment settings, such as the datasets and base backbones, are expected to be explained clearly. ", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for appreciating the motivation and the good value of this work.\n\n**Q1: Complex images evaluation & model robustness**\n\nGreat feedback. In addition to Fig. 3, we have provided more results in Figs. 1 and 2 of the Supplementary Material (SM). For example, in Fig. 1(c) of SM, LICO captures more details, such as foot and head, and in Fig. 2 of SM, the attention maps of LICO are more explainable for human vision.\n\nFor robustness assessment, we evaluated the standard deviations of the Pointing Game (**Table R1**) of different methods. LICO exhibits better results than baseline methods, indicating that LICO is more stable in localization. We will add more attention maps of complex images and assessments to the future version.\n\n**Q2: Experiments under settings of 'multi-object single-class' and 'multi-class'**\n\nWe appreciate your insightful comments and suggestions. Regarding the **multi-object single-class** cases, we would like to draw your attention to Section 1 of Supplementary Material for the results. Furthermore, Fig. 2 also demonstrates LICO's capability to capture multiple entities of identical classes through its attention maps.\n\nFor **multi-class** setting, we conducted the pointing game on MS COCO 2017 validation set for localization evaluation. Following settings in Score-CAM and Group-CAM, we quantified localization by calculating $\\frac{\\text{Hits}}{\\text{Hits} + \\text{Misses}}$, assessing if salient pixels fall within the annotated bounding boxes. **Table R1** shows that LICO consistently improves all the baseline interpretation methods, indicating the effectiveness of regularization by the proposed manifold OT losses. We will certainly add these results in our future version.\n\nIn the **Global Response PDF** file, we provided more attention maps under ‘multi-object single-class’ and ‘multi-class’ conditions.\n\n**Table R1**: Mean accuracy values $\\pm$ standard deviation of Pointing Game on MS COCO 2017 val dataset\n\n| Methods | Grad-CAM | Grad-CAM++ | RISE | XRAI | Score-CAM | Group-CAM |\n| :---: | :---: | :---: | :---: | :---: | :---: | :---: |\n| w/o LICO | 56.7$\\pm$0.225 | 57.2$\\pm$0.227 | 54.3$\\pm$0.205 | 55.1$\\pm$0.232 | 51.0$\\pm$0.211 | 57.5$\\pm$0.202 |\n| w/  LICO | **56.9$\\pm$0.221** | **58.1$\\pm$0.215** | **55.2$\\pm$0.201** | **56.7$\\pm$0.229** | **52.5$\\pm$0.205** | **58.2$\\pm$0.197** |\n\n**Q3: More details and basic experimental settings**\n\nWe will add more details about ablation studies and experimental settings to the future version.\n\n**Q4: Inconsistent total loss function between Algorithm 1 and Eq. (8)**\n\nThank you for pointing out this issue. The loss ‘LKL’ in Algorithm 1 should have been ‘LMF‘, which will be fixed in the future version.\n\n**Q5: Explanations on how the LICO exhibits both good explanatory and discriminative abilities**\n\nVery interesting observation. Based on our assumption that the CLIP text encoder implies 'real-world' semantic space, the text embedding capability is closely associated with the corresponding image space. Hence, $L_{\\text{MF}}$ makes the models sensitive to 'real-world' semantic information, guaranteeing discriminative ability towards different classes. Once the image representation is globally aligned with the real-world semantic representation, $L_{\\text{OT}}$ facilitates the correlation between visual feature maps and class-specific prompt tokens. For distribution alignment by OT within one image, $L_{\\text{OT}}$ leads relatively redundant visual features to those learnable context tokens, highlighting the association between key features and class tokens.\n\nOne critical reason for decreased classification performance by GCC and CGC is that there are only categorical one-hot labels during training. In contrast, our LICO incorporates generalized semantic information for guiding the image classification network. On the other hand, CGC is tailored for optimizing attention maps through contrastive learning but ignores preserving the discriminative ability of feature representations.\n\n**Q6: Experimental results about  $L_{\\text{CE}}$**\n\nFirst, we would like to clarify that the Ablation studies in Table 5 are obtained by combinations of $L_{\\text{CE}}$ and manifold and OT losses.\n\nIn **Table R2**, we added the results obtained by $L_{\\text{CE}}$. $L_{\\text{MF}}$ guarantees global manifold correlated with class texts, while only using $L_{\\text{OT}}$ rarely enables local alignment between feature maps and prompt tokens, ignoring class information. We observed that $L_{\\text{OT}}$ decreases the performance of $L_{\\text{CE}}$. This suggests that without global manifold alignment, OT struggles to enhance the discriminative ability of target classes. The reason is that OT focuses on aligning context tokens and feature maps within an individual image. Without correlation to a class-centric manifold, OT will misalign feature maps, resulting in suboptimal performance.\n\n**Table R2**: Ablation on $L_{\\text{MF}}$ and $L_{\\text{OT}}$\n\n| Loss | Top-1 $\\uparrow$ | Top-5 $\\uparrow$ | Insertion $\\uparrow$ | Deletion $\\downarrow$ |\n| :--- | :---: | :---: | :---: | :---: |\n| $L_{\\text{CE}}$ | 76.13 | 92.91 | 53.5 | **13.3** |\n| $L_{\\text{CE}}$ + $L_{\\text{OT}}$ | 75.98 | 92.92 | 56.6 | 16.0 |\n| $L_{\\text{CE}}$ + $L_{\\text{MF}}$ | 76.18 | 92.90 | 56.9 | 15.5 |\n| $L_{\\text{CE}}$ + $L_{\\text{OT}}$ + $L_{\\text{MF}}$ | **76.27** | **92.99** | **57.1** | 15.1 |\n\n**Q7: Effect of MLP used in text encoder**\n\nThe MLP ensures that the text representations from CLIP can be aligned dimensionally with latent visual features of various image encoders in our experiment. In practical scenarios, it is possible to find an alternate method to process the text representation, given a fixed dimension. However, we emphasize that such an MLP would only bring minor training costs while not compromising inference efficiency."}, {"review_id": "Wm4fRsePSH", "reviewer": "Reviewer_V7Pm", "summary": "This paper introduces LICO, a model that aligns visual encoder to language features. This model incorporates a frozen text encoder, a trainable image encoder, a classification loss, a manifold matching loss and an optimal transport loss. Experiments shows improvements over existing interpretation methods.", "questions": "1. How would authors compare the manifold loss to a cross-entropy with softmax temperature > 1? \n2. What is the necessity of the text encoder being from CLIP? Would a pure text embedding model (word2vec, bert, etc) work?\n3. Would a model trained with LICO work for words that are not included in the training class label set?", "limitations": "The authors discuss about their limitation on L303, which is not a true limitation compared to some in the Weaknesses I listed. Further, this limitation of \"some training overhead\" is not discussed in the paper. In concept, such an MLP would only bring insiginificant training cost when compared to the image encoder.", "rating": 5, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "1. This paper is well motivated, and the method is clearly stated in text as well as figures.\n2. Quantitative results are better than baselines.\n3. The experiments are comprehensive in terms of the number of datasets and baselines to compare to.", "weaknesses": "1. LICO obtains class label text embeddings via a text encoder pretrained with internet-scale image-text pairs, thus bringing additional information. Further, the image encoder is trainable whilst in baseline methods the image encoder are all frozen. These two aspects add up to an unfair comparison to other baselines.\n1. The optimal transform loss is motivated by aligning partial regions to prompt tokens, but there's no quantitative/qualitative experiments analyzing that specific effect.\n2. The architectures used in this work are generally outdated, and exact version of the architecture (ResNet18/50) are less performant ones.\n3. In Figure 3, the qualitative results from LICO are not better than baselines.\n4. In Table 2, the improvements are not significant.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for appreciating the motivation and comprehensive experiments.\n\n**Q1: On comparison fairness**\n\nWe apologize for not making it clear that our comparison to baselines is fair.\n\n- **Additional textual information**. Regarding the textual information of LLM as real-world semantic space is our assumption, motivating us to enhance target models. For downstream tasks, previous interpretation method did NOT consider this critical point, so their results are biased to limited semantic space.\n- **Learned encoders of LICO and baseline methods**.  LICO and baseline encoders are trained on the same dataset from *scratch* and used the same CE loss. Even for Grad-CAM, their target encoders are trained similarly.\n\n**Q2: Analysis of aligning partial regions to prompt tokens**\n\nThis question highlights an essential aspect of our research. To the best of our knowledge, LICO is the first work to align specific tokens with correlated feature maps for better interpretation. The optimal transport plan $T^\\* \\in \\mathbb{R}^{N \\times M}$ of OT implies the relationship between $M$ prompt tokens and $N$ feature maps. The corresponding row of $T^\\*$ can be treated as the weights of corresponding feature maps. However, there needs further exploration on whether the weighted sum of feature maps can represent the heatmap of specific tokens, which will be studied in the future. Notably, our ablation study in Table 5 has verified the effectiveness of OT, and MF + OT outperforms MF alone both in classification performance and quantitative interpretation.\n\n**Q3: Concerns on the simple architectures**\n\nFirst, we followed prior interpretation methods to use these architectures for a fair comparison. Second, LICO's design is model-agnostic and mainly focused on language-consistent interpretation, which can be flexibly extended to advanced architectures.\n\n**Q4: Results in Figure 3 and Table 2**\n\nIn Figure 3, although LICO and Original cover similar rough regions, LICO helps to localize more accurate details. In addition, the corresponding insertion and deletion curves quantitatively illustrate that the highlighted regions of LICO are more sensitive to model decisions. In Table 2, unlike previous methods that compromise model discriminative performance, LICO is the first method that *simultaneously* improves the interpretation and classification.\n\n**Q5: Manifold loss vs cross-entropy with Softmax temperature > 1**\n\nThis question is interesting and instructive. First, for mathemtics, $L_{\\text{KL}}(A^{G}||A^{F}) = - L_{\\text{CE}}(A^{G}, A^{F}) + H(A^{G})$, where $H()$ denotes entropy. When the entropy $H(A^{G})$ is a constant value, KL-divergence equals CE loss. When temperature > 1, the entropy term becomes larger, resulting in the inaccurate prompt manifold. If the entropy $H(A^{G})$ is small enough, $A^{G}$ is one-hot, so we can regard KL-divergence as an equivalent of cross-entropy. Hence, MF and CE are similar to some extent.\nSecond, the MF of LICO is effective in cross-modal alignment. However, CE often works with categorical one-hot labels and is inferior in matching two distributions.\n\n**Q6: CLIP text encoder vs pure text embedding model**\n\nBoth Word2Vec (W2V) and BERT can replace CLIP text encoder in our LICO framework. However, they may be inferior in acting as language guidance in LICO: (1) CLIP text encoder was trained with huge amounts of image-text pairs, and the advanced architecture ViT-B/32 has stronger representation ability than W2V; (2) Even though BERT excels at representing language data, latent representation of BERT may not align with target image representations well; (3) Recent studies on prompt learning like CoCoOp has demonstrated the effectiveness of CLIP pre-trained text encoders for downstream tasks.\n\nIn **Table R1**, we conducted the experiments using W2V as a text encoder, mapping texts into 512-D. W2V: using fixed word embeddings and no context tokens for OT loss. W2V-P: replacing class tokens in the prompts, the context tokens are randomly initialized as Gaussian. We can see that W2V performed comparably to the None encoder baseline. W2V-P also fails, where the fixed random context tokens mislead the image features. In contrast, the CLIP outperforms both of them.  The fixed W2V embedding vectors struggled to correlate well with target image features. \n\n**Table R1**: Comparison between CLIP text encoder and W2V on CIFAR-10\n| Encoder | CLIP | W2V | W2V-P | None |\n| :---: | :---: | :---: | :---: | :---: |\n| Full | **95.8** | 95.6 | 94.9 | 95.6 |\n| 4000 | **81.5** | 81.0 | 80.2 | 80.9 |\n\n**Q7: Would LICO work for words not included in the training?**\n\nThis comment motivates us to consider the robustness of LICO. We conducted additional experiments of zero-shot inference: training on ImageNet and inference on CIFAR-10, Flowers, and FGVC Aircraft test sets.  The inference probability is the same as that image-text matching used in CLIP. At the inference stage, we replace the class tokens of ImageNet with those of test sets, and the trained context tokens are randomly selected for constructing new prompts. In **Table R2**, we report the mean accuracy of five independent tests. Compared with CLIP trained on vast amounts of image-text pairs, our LICO obtains some performance drop on CIFAR-10 and Flowers and relatively small drop on FGVC Aircraft, because CLIP training set also does not contain large amounts of aircraft images and texts. We also fixed the pre-trained encoder and fine-tune a classification head named LICO-F, which improves the zero-shot performances.\n\nIn summary, our LICO can work and achieve comparable results.\n\n**Table R2**: Comparisons of zero-shot classification accuracy (%)\n| Method | CIFAR-10 | Flowers | FGVC Aircraft |\n| :---: | :---: | :---: | :---: |\n| CLIP | **75.6** | **65.9** | **19.3** |\n| LICO | 63.8 | 55.7 | 17.2 |\n| LICO-F | 70.4 | 61.1 | 18.7 |\n\n**Q8: On the limitation**\n\nThanks. We will discuss the Weakness you listed in the future version.\n"}, {"review_id": "WFXEBQA9h4", "reviewer": "Reviewer_r2cD", "summary": "This paper proposed LICO, which leverages the textual and semantic knowledge learned by large language models to guide latent image features. By matching relationships among images with KL-divergence globally, and distances between specific feature maps and prompt tokens with OT, the feature space of image is aligned with the prompt tokens in LLM. ", "questions": "see the \"Weaknesses\"", "limitations": "N/A", "rating": 6, "confidence": 4, "soundness": 4, "presentation": 3, "contribution": 3, "strengths": "1. It provides a novel idea that uses the semantic knowledge of LLM to guide the feature representations in the image classification model. Since the feature expression ability of the model is improved, both the classification performance and interpretation maps generated by XAI methods are better.\n2. Thorough experiments are conducted on multiple datasets and show the effectiveness of the proposed method. \n3. The paper is clearly written and easy to follow.", "weaknesses": "1. My main concern is that the model trained with the help of LLM is not the original model, so the improved interpretation is for the later model which has a better performance. So I'm unsure whether this can be regarded as \"enhancing existing visual interpretation methods.\" since the interpreted model is already changed. It's more like LICO enhances the classification model so that the XAI results are improved. LICO is not an XAI method itself, but it helps the model better learn latent features by introducing knowledge from LLM. \n\n2. Some small problems:\n- What’s the transport cost used in OT? \n- Since fixed promotes are in the original CLIP but learnable X1 to Xm-1 are added in the language model input, can the model work well with frozen parameters and without fine-tuning?  How are the learnable prompts initialized?\n- Missing introduction in Sec.1 and Sec.2 for perturbation-based explanation methods like RISE, which is also compared in the experiments.\n- The legend for g and f in Figure 2 are reversed.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We appreciate your thorough summary, encouraging feedback, and constructive suggestions.\n\n**Q1:  Concerns that the model trained with LICO is not the original model**\n\nYour understanding is correct, and we completely agree with you. The proposed LICO is not strictly an XAI method but offers a more explainable classification model from scratch while maintaining its discriminative capacity. In other words, LICO acts as a *plug-and-play* training strategy, where the resulting models are enhanced with semantic knowledge of LLM through alignments of global manifolds and local features. We will make it clear in the future version.\n\n**Q2: On the transport cost used in OT**\n\nWe would like to clarify that the transport cost used in OT is cosine distance, which will be clarified in the future version. \n\n**Q3: On the learnable prompt initialization and model performance with frozen parameters**\n\nThank you for your insightful comment. We would like to note that the learnable prompts in our LICO are randomly initialized according to Gaussian distribution, the same as the setting in [1].\n\nTo evaluate the model performance with frozen parameters, we then conducted experiments on CIFAR-10 and ImageNet under two settings of frozen parameters: (1) frozen random initialization (Random) and (2) fixed form of 'a photo of a [CLS]' (Fixed). The Insertion and Deletion values are obtained by Grad-CAM + LICO. \n\nIn **Table R1**, we can see that the frozen random parameters cannot enable the models to achieve higher performances of accuracy, insertion, and deletion; the reason is that the well-trained CLIP text encoder is capable of sensing the human-understandable phrases and sentences while the random prompts lead to the difficulty in image-text alignment and yield inaccurate semantic representations. However, the form of 'a photo of a [CLS]' performs better than frozen random parameters because this meaningful prompt is more consistent with the input of the original CLIP so that the generated representations can be easily aligned with image representations.\n\n**Table R1**: Evaluations on frozen parameters of prompts\n\n| ImageNet | Top-1 $\\uparrow$  | Insertion $\\uparrow$ | Deletion $\\downarrow$ | - | CIFAR-10 | Full, Accuracy $\\uparrow$ |  4000, Accuracy $\\uparrow$ |\n| :---: | :---: | :---: | :---: | :---: | :---: | :---: | :---: |\n| Random | 75.88 | 53.3 | 17.8 | - | Random | 94.9 | 79.5 |\n| Fixed | 76.20 | 55.2 | 17.4 | - | Fixed | 95.4 | 80.7 |\n| Learnable (ours) | **76.27** | **57.1** | **15.1** | - | Learnable (ours) | **95.8** | **81.5** |\n\n> [1] Guangyi Chen, Weiran Yao, Xiangchen Song, Xinyue Li, Yongming Rao, and Kun Zhang. PLOT: Prompt learning with optimal transport for vision-language models. In The Eleventh International Conference on Learning Representations, 2023.\n> \n\n**Q4: On the missing introduction of perturbation-based methods and reversed legend**\n\nThank you for your careful reading and pointing out these issues. We will certainly add the introduction of RISE in Secs. 1 and 2 and fix the legend error in Figure 2 in the future version."}]

Author rebuttal:
## Global Responses with a PDF file

**Comments**:

Dear Reviewers,

We thank all the reviewers for their thorough summaries and valuable feedback. The reviewers appreciate that our LICO is novel and well-motivated (**rFY8**, **V7Pm**, **r2cD**) with good value of incorporating language prompts into explainable AI (**rFY8**, **r2cD**), the experiments are comprehensive (**rFY8, V7Pm, r2cD**) and demonstrate the effectiveness of LICO (**r2cD**), good performance (**tr3q, rFY8, r2cD**), and inference efficiency (**tr3q**), while the paper is well-organized and easy to follow(**rFY8**, **V7Pm**, **r2cD**).

We have posted detailed responses to each reviewer and deeply appreciate your further feedback on whether our responses adequately address your concerns. If you have any additional comments or questions, we will try our best to address them.

Per the Q4 of **V7Pm**, Q1 and Q2 of **rFY8**, the following **attached pdf file** provide more attention maps of complex images:

 - Figure 1:  Attention maps of single-class multi-object images.
 - Figure 2: Attention maps of multi-class multi-object images.

Best,

The authors

Meta-review: This paper proposes an approach for explaining the predictions of a neural network by relating the reasons for a prediction to language prompts.  Through the course of the reviewing process the authors and reviewers discussed the merits of the proposed approach, with all reviewers recommending acceptance.  The ACs find that there is not sufficient justification to overturn the unanimous recommendation of the reviewers, and remind the authors to use the comments and rebuttals to update their paper given the reviewer's feedback.
Decision: Accept (poster)