SlowGuess/NeurIPS_2023_OCR huggingface dataset:
Yiming Lei $^{1}$ , Zilong Li $^{1}$ , Yangyang Li $^{2}$ , Junping Zhang $^{1}$ , Hongming Shan $^{3,4*}$
$^{1}$  Shanghai Key Laboratory of Intelligent Information Processing, School of Computer Science, Fudan University
$^{2}$  Academy of Mathematics and Systems Science, Chinese Academy of Sciences
$^{3}$  Institute of Science and Technology for Brain-Inspired Intelligence and MOE Frontiers Center for Brain Science, Fudan University
$^{4}$  Shanghai Center for Brain Science and Brain-inspired Technology {ymlei, jpzhang, hmshan}@fudan.edu.cn, zilongli23@m.fudan.edu.cn, yyli@amss.ac.cn
Interpreting the decisions of deep learning models has been actively studied since the explosion of deep neural networks. One of the most convincing interpretation approaches is salience-based visual interpretation, such as Grad-CAM, where the generation of attention maps depends merely on categorical labels. Although existing interpretation methods can provide explainable decision clues, they often yield partial correspondence between image and saliency maps due to the limited discriminative information from one-hot labels. This paper develops a Language-Image COnsistency model for explainable image classification, termed LICO, by correlating learnable linguistic prompts with corresponding visual features in a coarse-to-fine manner. Specifically, we first establish a coarse global manifold structure alignment by minimizing the distance between the distributions of image and language features. We then achieve fine-grained saliency maps by applying optimal transport (OT) theory to assign local feature maps with class-specific prompts. Extensive experimental results on eight benchmark datasets demonstrate that the proposed LICO achieves a significant improvement in generating more explainable attention maps in conjunction with existing interpretation methods such as Grad-CAM. Remarkably, LICO improves the classification performance of existing models without introducing any computational overhead during inference. Source code is made available at https://github.com/ymLeiFDU/LICO.
Although deep neural networks (DNNs) have shown excellent performance in many fields, the lack of interpretability is still a barrier to landing in some high-stakes scenarios such as medical diagnosis, autonomous driving, etc. Therefore, the literature proposes various interpretation methods for DNNs and reveals the decision clues of DNNs to some extent.
Popular interpretation methods can be roughly categorized into two types: 1) gradient backpropagation-based, and 2) class activation mapping (CAM)-based. Both of them mainly take image classification as the pretext task and then generate explainable noisy gradients and saliency maps, respectively. As illustrated in Fig. 1(a), CAM-based methods often explore a better weighting scheme for integrating feature maps of a given input image. The gradient-based methods, in Fig. 1(b), also
Figure 1: Motivation of LICO. (a) Existing post-hoc CAM-based methods focus on generating better weighting schemes to obtain weighted-sum attention maps. (b) Gradient-based methods tend to back-propagate gradients from logits to input space. (c) Proposed LICO incorporates learnable prompts to enable image features to approximate semantic information in latent space.
start from the output logits while back-propagating gradients to the input space. However, they are all post-hoc approaches that investigate a DNN pre-trained on a specific dataset [1, 2, 3, 4, 5, 6, 7], and yield biased interpretations due to the limited semantic information that a set of one-hot labels can offer. In addition, one-hot labels often trigger overfitting of the pre-trained model so that the effectiveness of existing interpretation methods could be compromised. On the other hand, the latent feature space of such pre-trained models is unable to sense real semantic space reflecting crucial parts in images. Therefore, it may be futile to explore complex post-hoc techniques for interpreting a pre-trained model.
Inspired by advanced large vision-language models (VLMs) such as CLIP [8], which developed generalized image and text encoders through training on huge amounts of image-text pairs, in this paper, we assume that the large VLMs can encode real-world semantic knowledge through contrastive vision-language alignments, whereas the traditional models pre-trained on relatively small datasets, such as ImageNet-1k, are inferior in capturing the true semantic information.
Motivation. The DNN-based image classification framework generally consists of a convolutional neural network (CNN) for feature extraction and a linear layer acting as a classifier. Since feature representations are used as the input to the classifier, if the DNN is truncated from feature representations, it becomes a linear classifier that aims to classify feature representations linearly into discrete one-hot label space. More specifically, the feature representation lies in the learned manifolds of high-dimensional semantic space [9]. Unfortunately, training using cross-entropy loss with one-hot labels cannot guarantee that the manifolds of image features can reflect the distribution of real-world semantic knowledge, which hinders performance improvement of existing interpretation methods.
In this paper, we leverage language information from large VLMs to enhance current interpretation methods for achieving more explainable saliency maps while enabling promising classification performance improvements; see Fig. 1(c). First, we propose Language-Image-CONSistent (LICO) learning to facilitate the alignment between the manifolds of visual features and class-aware language information. To address the discrete nature of categorical classes, we construct a learnable prompt for each class and map all prompts into a continuous space using the CLIP text encoder, which is feasible to align manifolds of both image and text. Second, we impose each prompt token to correlate with certain feature maps. Considering that the feature maps are redundant to the final classification decision, we propose to encourage the context tokens to guide certain feature maps through distribution alignment using optimal transport (OT) theory.
Contributions. We summarize the main contributions of this paper as follows. (i) We propose a novel framework to enhance current interpretation methods by introducing language guidance from large VLMs. (ii) We model the discrete categorical class to a continuous space by constructing class-aware learnable prompts, hence, enabling consistent manifold matching between image features and text features. (iii) To ensure consistent local feature alignment, we utilize OT theory to reduce the distance between distributions of image and text. (iv) Extensive experimental results on eight classification datasets demonstrate the superiority of the proposed LICO against current interpretation methods in terms of quantitative and qualitative results. For practical applications, LICO does not introduce any computational overhead during inference while maintaining improved performance.
Figure 2: Framework of the proposed LICO. (a) Conventional classification pipeline of DNNs. (b) Language feature extraction with pre-trained text encoder. (c) Manifold matching among samples and optimal transport alignment between feature maps and prompt tokens within each sample.
Interpretation methods. Class activation mapping (CAM) is a simple approach to generating class-aware saliency maps for CNNs [1]. Inspired by CAM's effectiveness, its variants including Grad-CAM [2], Grad-CAM++ [3], Score-CAM [4], RISE [10], and Group-CAM [5], were proposed to enhance the precision and interpretability of CNNs. Gradient-based methods such as Guided Back-propagation [11], SmoothGrad [7], and Integrated Gradient (IG) [6] compute the gradients from output logits back to input space. However, they are prone to yielding attribution noise that is inconsistent with human intuition. Some advanced techniques have been developed to reduce attribution noise by utilizing improved path integration theory, such as adaptive path method [12] and important direction method [13]. Distinct from existing interpretation methods that focused on the post-processing of feature map weighting scheme and accurate gradients calculation based on a pre-trained model, our LICO aims to incorporate class-aware learnable prompts that reflect real-world semantic knowledge to guide latent image features. It is worth noting that LICO is compatible with current methods and can consistently improve their performance.
Prompt learning. Prompt learning stems from natural language processing (NLP), enabling large language models to directly model the prediction probability of raw text [14, 15, 16]. Following the proliferation of large VLMs, prompt learning began to surface in the realm of computer vision, significantly enhancing multi-modal tasks and text-guided image generation. To overcome the limitations of learning with fixed prompts used in CLIP, learnable prompts have been explored to provide more generalized performance on downstream tasks. CoOp firstly proposed a framework that constructs learnable prompts to align image and text in latent space [17]. Furthermore, to enable CoOp with a stronger capability to perceive the knowledge of new classes, the authors proposed CoCoOp in which the prompts are learned conditioned on image features [18]. PLOT correlated one given image with multiple prompts of its class by optimal transport, leading different prompts to reflect features with respect to different regions within this image [19]. In our LICO, we propose that learnable prompt tokens in one sentence should be registered to certain parts of an image, as shown in Fig. 1(c).
Optimal transport. Optimal transport (OT) is a typical metric for measuring discrepancy between two distributions, requiring solving a linear programming problem [20, 21]. Thanks to some fast alternatives, OT has been widely utilized in deep learning, including graph cross-domain alignment [22, 23], domain adaptation [24, 25, 26], optimal graph matching for vessel image registration [27], and multi-modal distribution alignment [28, 29]. Although KL-divergence can effectively measure the cost of which the predicted distribution approximates ground-truth distribution, given the normalized image and text distributions, it is intractable to guarantee they share a common metric space. Therefore, we propose to utilize OT to tackle fine-grained cross-modal alignment.
3.1 Overview of LICO
Fig. 2 presents the overview of LICO framework. Given a mini-batch of  $B$  training samples  $\{(\pmb{x}_i, y_i, t_i)\}_{i=1}^B$ , the input image  $\pmb{x}_i$  is of size  $H \times W \times C$ , where  $C$  is the number of channels,  $H$  and  $W$  are height and width, respectively.  $y_i \in \mathcal{C}$  denotes the class label, and  $t_i$  represents the text of corresponding label  $y_i$ . We denote the pre-trained CLIP text encoder as  $g_\phi$  while we fixed its parameters  $\phi$  during training. The image encoder is denoted as  $f_\theta$  that is to be optimized. The image encoder takes  $x_i$  as input and outputs feature maps  $F_i$  with  $N$  channels. Note that  $F_i$  is a tensor, which is then flattened to one-dimension for each channel; i.e.,  $F_i \in \mathbb{R}^{N \times d'}$ . For the language modeling branch, the text encoder  $g_\phi$  takes as input the constructed learnable prompt:  $t_i = [X_1, X_2, \dots, X_{M-1}, t_i]$ , where [...] denotes the concatenation,  $t_i$  is the text corresponding to label of the  $i$ -th sample,  $X_m$  are learnable context tokens, and  $M - 1$  is the number of context tokens. Through the pre-trained text encoder, the prompt vector  $t_i$  is mapped into a continuous  $d$ -dimension space; i.e.,  $d = 512$  in CLIP.
In order to measure the distances between image and language features in latent space, we append a mapping net  $h_{\psi}$ , a multilayer perceptron (MLP) with only one hidden layer, to map the  $d$ -dimension language features  $\pmb{t}_i \in \mathbb{R}^{M \times d}$  to the space of  $d'$ -dimension, i.e.,  $G_i \in \mathbb{R}^{M \times d'}$ , which is the same as that of image features  $\pmb{F}_i$ . The structure of  $h_{\psi}$  varies along different image encoders. We use  $h_{\psi}[a, b]$  to describe its structure, i.e., numbers of hidden units and output units are  $a$  and  $b$ , respectively. Note that in this paper, we do not intend to modify the structure of the existing image and text encoders so that we can obtain the feature maps and corresponding attention maps that are comparable to existing methods. Consequently, we utilize  $F_i$  and  $G_i$  to preserve the language-image-consistent structure in latent space using manifold matching (Section 3.2) and fine-grained feature-language alignment by optimal transport (Section 3.3).
3.2 Language-Image Manifold Matching
To preserve the global manifold structure in latent space, we first measure the relationships, i.e. the distances or similarities, among training images and corresponding class-aware prompts.
Although the class categories are discretely distributed that it is infeasible to capture their manifold, we can map them into a continuous space by constructing a prompt vector for each class, then mapping it to a latent space using a pre-trained text encoder. Therefore, we can enforce the image manifold [30] to approximate the language manifold.
In practice, we align the adjacent matrices of language and image features. Specifically, the language adjacent matrix is denoted as  $A_{B \times B}^{G}$ , and that of image is  $A_{B \times B}^{F}$ :
$$
\boldsymbol {A} _ {i, j} ^ {F} = \frac {\exp (- D (\boldsymbol {F} _ {i} , \boldsymbol {F} _ {j}) / \tau)}{\sum_ {s = 1} ^ {B} \exp (- D (\boldsymbol {F} _ {i} , \boldsymbol {F} _ {s}) / \tau)}, \quad \boldsymbol {A} _ {i, j} ^ {G} = \frac {\exp (- D (\boldsymbol {G} _ {i} , \boldsymbol {G} _ {j}) / \tau)}{\sum_ {s = 1} ^ {B} \exp (- D (\boldsymbol {G} _ {i} , \boldsymbol {G} _ {s}) / \tau)}, \tag {1}
$$
where  $D(\cdot, \cdot)$  calculates the distance between two images or two prompts, such as Euclidean distance,  $\tau$  is the temperature to be learned during training. Then, each image is formulated as a distribution  $A_{i,:}^{F}$  where each dimension denotes the distance from the  $i$ -th sample to others within a mini-batch. Similar to the class-wise prompts,  $A_{i,:}^{G}$  implies the relationships between the class prompt of the  $i$ -th sample and those of others.
Recall our assumption that the large amounts of image-text pairs used in large VLMs, such as CLIP, can lead to a generalized text encoder that well establishes a real-world semantic space. Therefore, for certain downstream tasks and datasets, we aim to introduce the knowledge of this semantic space to the latent space of the target image domain. Then, we propose a manifold matching loss  $\mathcal{L}_{\mathrm{MM}}$ , which enables the manifold of image features to approach that of prompt features using KL-divergence:
$$
\mathcal {L} _ {\mathrm {M M}} = \frac {1}{B} \sum_ {i = 1} ^ {B} \operatorname {K L} \left[ \boldsymbol {A} _ {i,:} ^ {G} \| \boldsymbol {A} _ {i,:} ^ {F} \right]. \tag {2}
$$
We note that we do not consider the inverse version  $\mathrm{KL}[A_{i,:}^{F}\| A_{i,:}^{G}]$  since LICO focuses on enabling image manifold to approach the manifold of language prompts.
3.3 Feature Distribution Alignment by Optimal Transport
While we strive for a coarse alignment of the global manifold structure, it is crucial to establish a correlation between prompt tokens and specific feature maps for each sample, which aids in mitigating the influence of redundant features on the generation of attention maps. Most importantly, the critical feature maps that are most related to the target class should possess the highest similarity with respect to the class token.
Unfortunately, it is challenging to determine or assign which of the  $N$  feature maps are correlated with certain prompt tokens. In this paper, we propose to align  $\pmb{G}_i = [g_1, g_2, \dots, g_{M-1}, g_{t_i}]^\top \in \mathbb{R}^{M \times d'}$  and  $\pmb{F}_i = [f_1, f_2, \dots, f_N]^\top \in \mathbb{R}^{N \times d'}$  for achieving consistency between feature maps and specific prompt tokens. In other words, different words in a sentence should correspond to parts of an image, and in contrast, partial regions in an image reflect the semantic information delivered by certain tokens. Therefore, it is intractable to measure this distance using KL-divergence since it is not a strict metric, i.e., it does not satisfy the property of triangle inequality. In LICO, we use optimal transport, which is widely used in measuring distances of two distributions, to align distributions of normalized visual features and prompt features.
For the given image feature maps  $\pmb{F}_i \in \mathbb{R}^{N \times d'}$  and a prompt tokens  $\pmb{G}_i \in \mathbb{R}^{M \times d'}$ , we construct two discrete distributions:
$$
\boldsymbol {\mu} = \sum_ {n = 1} ^ {N} u _ {n} \delta_ {\boldsymbol {f} _ {n}}, \quad \boldsymbol {v} = \sum_ {m = 1} ^ {M} v _ {m} \delta_ {\boldsymbol {g} _ {m}}, \tag {3}
$$
where  $\delta_{\pmb{f}_n}$  is a Dirac function centered at  $\pmb{f}_n$ , so as to  $\delta_{\pmb{g}_m}$ , and the weights  $\pmb{u} = \{u_n\}_{n=1}^N \in \Delta_N$  and  $\pmb{v} = \{v_m\}_{m=1}^M \in \Delta_M$ ,  $\Delta$  denotes the  $N$ - and  $M$ -dimensional probability simplex, i.e.,  $\sum_{n=1}^N u_n = 1$ ,  $\sum_{m=1}^M v_m = 1$ . Finally, the discrete OT distance for one sample is defined as follows:
$$
D _ {\mathrm {O T}} (\boldsymbol {\mu}, \boldsymbol {v}) = \inf  _ {\boldsymbol {\pi} \in \Pi (\boldsymbol {\mu}, \boldsymbol {v})} \mathbb {E} _ {(\boldsymbol {\mu}, \boldsymbol {v}) \sim \boldsymbol {\pi}} [ \boldsymbol {C} (\boldsymbol {f}, \boldsymbol {g}) ] = \min  _ {\boldsymbol {T} \in \Pi (\boldsymbol {\mu}, \boldsymbol {v})} \sum_ {n = 1} ^ {N} \sum_ {m = 1} ^ {M} \boldsymbol {T} _ {n, m} \cdot c (\boldsymbol {f} _ {n}, \boldsymbol {g} _ {m}), \tag {4}
$$
$$
\begin{array}{l l} s. t. & T \mathbf {1} _ {m} = \boldsymbol {\mu}, \quad T \mathbf {1} _ {n} = \boldsymbol {v}, \end{array}
$$
where  $C \in \mathbb{R}^{N \times M}$  represents the cost matrix in which each element  $c(\pmb{f}_n, \pmb{g}_m)$  denotes transportation cost between  $\pmb{f}_n$  and  $\pmb{g}_m$ .  $\pmb{T} \in \mathbb{R}^{N \times M}$  is the transport plan that is to be optimized and  $\Pi(\pmb{\mu}, \pmb{v})$  denotes the transportation polytope that contains all joint probabilities of  $\pmb{\mu}$  and  $\pmb{v}$ . In practice, solving the optimization problem in Eq. (4) often equips with a high computation cost. Thus we use the Sinkhorn algorithm, which is more computationally amenable, to solve an entropy-constrained problem [31]:
$$
D _ {\mathrm {O T}} (\boldsymbol {\mu}, \boldsymbol {v}) = \min  _ {\boldsymbol {T} \in \Pi (\boldsymbol {\mu}, \boldsymbol {v})} \sum_ {n = 1} ^ {N} \sum_ {m = 1} ^ {M} \boldsymbol {T} _ {n, m} \cdot c \left(\boldsymbol {f} _ {n}, \boldsymbol {g} _ {m}\right) - \lambda \mathbb {H} (\boldsymbol {T}), \quad s. t. \quad \boldsymbol {T} \mathbf {1} _ {m} = \boldsymbol {\mu}, \quad \boldsymbol {T} \mathbf {1} _ {n} = \boldsymbol {v}, \tag {5}
$$
where  $\lambda$  is Lagrange multiplier and  $\mathbb{H}(\boldsymbol{T}) = \sum_{n,m}\boldsymbol{T}_{n,m}\log \boldsymbol{T}_{n,m}$ . Then after a few iterations, we obtain the optimal solutions:
$$
\boldsymbol {T} ^ {*} = \operatorname {d i a g} \left(\boldsymbol {\mu} ^ {t}\right) \exp (- \boldsymbol {C} / \lambda) \operatorname {d i a g} (\boldsymbol {v} ^ {t}), \tag {6}
$$
where  $t$  is the iteration step.  $\pmb{\mu}^t$  and  $\pmb{v}^t$  are updated according to following rules:
$$
\boldsymbol {\mu} ^ {t} = \boldsymbol {\mu} / \left(\exp (- C / \lambda) \boldsymbol {v} ^ {t - 1}\right), \quad \boldsymbol {v} ^ {t} = \boldsymbol {v} / \left(\exp (- C / \lambda) ^ {\top} \boldsymbol {\mu} ^ {t}\right). \tag {7}
$$
Dynamic context (DC). To endow each image with diverse prompt tokens, we shuffle the learnable context tokens in each training iteration referring to the training procedure in Algorithm 1.
3.4 Final Objective Function
The final training loss function is as follows:
$$
\mathcal {L} = \mathcal {L} _ {\mathrm {C E}} + \alpha \mathcal {L} _ {\mathrm {M M}} + \beta \mathcal {L} _ {\mathrm {O T}}, \tag {8}
$$
where  $\mathcal{L}_{\mathrm{CE}}$  is the cross-entropy loss,  $\mathcal{L}_{\mathrm{OT}} = \frac{1}{B}\sum_{i=1}^{B}D_{\mathrm{OT}}, \alpha$  and  $\beta$  are hyperparameters for adjusting different terms. During the inference phase, we apply the trained image encoder and classifier to conduct conventional classification that yields the predicted probability of a given input image. Note that the text encoder and the MLP mapping do not affect the inference procedure. The detailed algorithm can be found in Algorithm 1.
Algorithm 1 Training Algorithm of LICO.
Require: Training set  $S$ , total epochs  $U$ , image encoder  $f_{\theta}$ , text encoder  $g_{\phi}$ , MLP  $h_{\psi}$ , learnable prompts  $t_i = [X_1, X_2, \ldots, X_{M-1}, t_i]$ .  
Return: Image encoder with optimal parameter  $\theta^*$ .  
1: for  $u = 1$  to  $U$  do  
2: Sample a mini-batch of  $(\{x_i, t_i, y_i\}_{i=1}^B)$  from  $S$ .  
3: Randomly shuffling  $X_m$  and  $t_i$ .  
4:  $F_i = f_{\theta}(x_i)$ ,  $G_i = h_{\psi}(g_{\phi}(t_i))$ .  
5: Calculate  $A_{B \times B}^G$ ,  $A_{B \times B}^F$ .  
6: Calculate  $\mathcal{L}_{\mathrm{MM}}$  according to Eq. (2).  
7: Optimal transport plan  $T^*$  by Eq. (6), then calculate  $\mathcal{L}_{\mathrm{OT}}$ .  
8: Classifier  $\longleftarrow F_i$   
9: Total loss:  $\mathcal{L} = \mathcal{L}_{\mathrm{CE}} + \alpha \mathcal{L}_{\mathrm{MM}} + \beta \mathcal{L}_{\mathrm{OT}}$ , update  $\theta$ ,  $\psi$ , and  $X_m$  by gradients:  $\frac{\partial \mathcal{L}}{\partial \theta}$ ,  $\frac{\partial \mathcal{L}}{\partial \psi}$ ,  $\frac{\partial \mathcal{L}}{\partial X}$ .  
10: end for
This paper focuses on image classification task and evaluates the proposed LICO on well-known datasets, including ImageNet-1k [32], CIFAR-10/100 [33], and SVHN [34]. We conduct the classification experiments under the setting of limited training data in which the splits of labeled data follow the previous works for fair comparison [35, 36]. Furthermore, we conduct fine-grained classification on typical benchmarks, including CUB-200 [37], FGVC-Aircraft [38], Stanford Cars-196 [39], VGG Flowers [40]. The evaluation is carried out on both the full dataset and few-shot settings, following the procedure of CGC [41].
We employ the ViT-B/32 trained by CLIP [8] as the text encoder and its parameters are fixed during training. The output dimension of this text encoder is 512 for each token. The image encoders in our experiments vary along different datasets and training settings. Specifically, for the ImageNet experiments, we utilize ResNet-50 as the image classifier [42]. In doing so, the convolutional layers preceding the final linear layer constitute the image encoder of LICO. For the CIFAR-10/100 and SVHN, we follow the experimental settings in [35, 36], the classification network is Wide ResNet (WRN) [43]. We further conduct the same experiments using another network, the PreAct-ResNet-18 (PARN-18) [44]. For fine-grained classifications on CUB-200, Standford-Cars, Aircraft, and VGG Flowers datasets, we applied the same settings used in CGC for fair comparison [41], where the image encoder is also the ResNet-50 that has been pre-trained on ImageNet-1k with CE loss. Note that for all the experiments of LICO, we only use the trained image encoder and the classifier during the inference phase. The text encoder, learnable prompt contexts, and MLP mapping net are dropped, thus, will not compromise the computational efficiency. Hyperparameters  $\alpha$  and  $\beta$  are set as 10 and 1, respectively.