LeGrad: An Explainability Method for Vision Transformers via Feature Formation Sensitivity

## Abstract

Abstract Vision Transformers (ViTs), with their ability to model long-range dependencies through self-attention mechanisms, have become a standard architecture in computer vision. However, the interpretability of these models remains a challenge.
To address this, we propose LeGrad, an explainability method specifically designed for ViTs.
LeGrad computes the gradient with respect to the attention maps of ViT layers, considering the gradient itself as the explainability signal.We aggregate the signal over all layers, combining the activations of the last as well as intermediate tokens to produce the merged explainability map.
This makes LeGrad a conceptually simple and an easy-to-implement tool for enhancing the transparency of ViTs.
We evaluate LeGrad in challenging segmentation, perturbation, and open-vocabulary settings, showcasing its versatility compared to other SotA explainability methods demonstrating its superior spatial fidelity and robustness to perturbations. 1 1 1 A demo and the code is available at https://github.com/WalBouss/LeGrad

## 1 Introduction

In the realm of computer vision, Vision Transformers (ViTs)[] have revolutionized the field with their ability to model long-range dependencies through self-attention mechanisms [, , , , ]. Explanability methods designed for convolutional or feed-forward neural networks are not directly applicable to ViTs due to their architectural requirements, like GradCAM’s [] reliance on convolutional layers and Layer-wise Relevance Propagation’s (LRP) [] specific layer-wise propagation rules.
While ViT-specific interpretability techniques exist, including adaptations of traditional methods [, , ], attention-based techniques [, , , ], and text-based explanations [, , ], the interpretability especially of large-scale architectures remains a challenge.

To address this, we propose a Layerwise Explainability method that considers the Gradient with respect to the attention maps, LeGrad. LeGrad is specifically designed for ViTs as leverages the self-attention mechanism intrinsic to ViTs to generate relevancy maps highlighting the most influential parts of an image for the model’s prediction.
Compared to other methods, LeGrad computes gradient with respect to the attention maps, as e.g. opposed to CheferCam [, ], which uses the gradient to weight the attention maps. This is done independently for each layer. The final explainability signal is then pooled over all layers of the ViT. Note that using a layerweise gradient, compared to other signals, allows to sum up over different layers without further need for normalization. To further improve the signal, the gradient is clipped by a ReLU function preventing negative gradients to impact positive activations.
The proposed approach is conceptually simple and versatile, as it only requires the gradient w.r.t. to the ViT’s attention maps. This facilitates its adoption across various applications and architecture, including larger ViTs such as ViT-BigG as well as attention pooling architectures [, ], enhancing the transparency of ViTs.

We evaluate the proposed method for various ViT backbones
on three challenging tasks: segmentation, open-vocabulary detection, and perturbation on ImageNet [, ] as well as OpenImagesV7 []. It shows that while current methods struggle especially with the diverse object categories in OpenImagesV7, LeGrad can achieve performance gains of $2\times$–$5\times$, reaching a score of $48.4$ p-mIoU on OpenImagesV7 using OpenCLIP-ViT-B/16. Furthermore, we demonstrate the applicability of LeGrad to very large models, such as the ViT-BigG/14 [] with 2.5 billion parameters. Finally, it also adapts well to different feature aggregation strategies employed by ViTs [, ], making it a versatile tool for explainability.

We summarize the contributions as follows:
(1) We propose LeGrad as a layerwise explainability method based on the gradient with respect to ViTs attention maps.
(2) As the layerwise explainability allows to easily pool over many layers, LeGrad scales to large architectures such as ViT-BigG/14 and is applicable to various feature aggregation methods.
(3) We evaluate LeGrad on various tasks and benchmarks, showing its improvement compared to other state-of-the-art explainability methods especially for large-scale open vocabulary settings.

## 2 Related Work

Gradient-Based Explanation Methods
Feature-attribution methods (also known as saliency methods) are a commonly used explanation technique that explain model decisions by assigning a score to each image pixel representing its importance to the model’s output.
Largely, these methods can be categorized into two groups [] — gradient-based methods that compute explanations based on the gradient of the model’s prediction with respect to each input pixel [, , , , , , ] and perturbation-based methods that measure pixel-importance by successively perturbing the input images and measuring the impact on the model’s output [, , , , ].
While both types of methods have been used successfully to identify reasoning, spurious correlation, and trustworthiness in traditional computer vision models [, ], gradient-based methods are often more computationally efficient since they only require a single backwards pass and simple to understand since they are a direct function of the model’s parameters and do not rely on additional models or image modifications.
However, many existing gradient-based methods were designed for convolutional and feed-forward model architectures, so it is non-trivial to directly apply them to ViTs since ViTs do not contain spatial feature maps and include complex interactions between patches induced by the self-attention mechanism.
Instead, we build on the simplicity and computational efficiency of gradient-based explanation methods to develop LeGrad, a gradient-based feature-attribution method specifically designed for ViT architectures.

While most gradient-based methods were designed prior to the widespread use of ViTs, researchers have recently made efforts to adapt existing methods or develop new ones specifically for transformers.
Chefer et al. [] extend LRP [] to transformers by integrating gradients within the self-attention layers.
However, this approach is computationally heavy and is inflexible to architecture changes as it requires specific implementation for each module of the network.
To circumvent that complexity, CheferCAM [] weights the attention by their gradient and aggregates it through the layer via matrix multiplication.
However, the use of gradients to weigh the attention heads’ importance makes this method class-specific.
LeGrad on the other hand is class-agnostic and computationally efficient, making it simple to use and more effective than these other methods (see Sec. 4 for a quantitative evaluation).

Attention-Based Explanation Methods
A separate line of research has proposed using attention maps, as opposed to gradients, as a way to explain for transformers’ decisions [, ].
One attention-based method, rollout [], traces the flow of importance through the transformer’s layers by linearly combining the attention maps via matrix multiplication.
Attention flow [] contextualizes the attention mechanism as a max-flow problem; however, it is computationally demanding and has not been extensively evaluated for vision tasks.
While these methods offer insights into the attention mechanism, they often neglect the non-linear interactions between attention heads and the subsequent layers.
Moreover, they may not adequately distinguish between positive and negative contributions to the final decision, leading to potentially misleading interpretations, as found in Chefer et al. [].
While LeGrad also focuses on the attention maps to generate its explanation, it uses the gradient w.r.t. to the attention maps, thereby assessing the sensitivity of the attention maps to a change in the patch tokens.
LeGrad does not attempt to track the flow of information throughout the transformer layers but, rather, focuses on assessing the sensitivity of the model prediction at each layer for every location of the image.

Text-based Explanations of Intermediate Representations
Research has also explored text-based methods for interpreting intermediate representations in vision models.
For instance, leveraging CLIP’s language-image space, researchers have provided text descriptions for active neuron regions [, ] and projected model features into text-based concept banks [].
In particular, TextSpan [] focuses on the explainability of CLIP-like models.
It refrains from using gradient computation by aggregating the intermediate features’ similarities along a given text direction, creating a relevancy map for a text query.
LeGrad advances this line of work by focusing on the sensitivity of feature representations within ViTs, offering a clear and concise method to generate relevancy maps that can be adapted to various feature aggregation strategies.

Evaluating Explanation Methods
As the popularity of feature-attribution based explanation methods has grown, so has research evaluating their ability to faithfully reflect the model’s decision making process [, , , , , , , , , ].
Faithfulness is an important criteria because explanation methods that highlight human-salient pixels may look correct even if the model relies on spurious correlations and give human interpreters unwarranted confidence in the model.
These evaluations can be categorized based on the aspect of faithfulness that they test [], such as the method’s sensitivity to meaningful changes to the input, model, or data [, , , , , , , ] and its ability to highlight a succinct and meaningful pixel region [, ].
We evaluate LeGrad on both of these types of tests, demonstrating that it is faithful to the model’s decision making process by testing that removing LeGrad’s important pixels impacts the model’s output and checking that LeGrad’s output corresponds to a concise object region.

Figure: Figure 2: Overview of LeGrad: Given a text prompt or a classifier $\mathcal{C}$, for each layer $l$, an activation $s^{l}$ is computed and used to compute the explainability of that layer. The layerwise explainability maps are then merged to produce LeGrad’s output.

## 3 Method

In this section, we first introduce ViT’s mechanics and the different feature aggregation mechanisms used for this architecture.
We then explain the details of LeGrad, starting by a single layer and then extending it to multiple layers.

### 3.1 Feature formation in ViTs

The ViT architecture is a sequence-based model that processes images by dividing them into a grid of $n$ patches. These patches are linearly embedded and concatenated with a class token $z_{0}^{0}=z^{0}_{[CLS]}\in\mathbb{R}^{d}$, which is designed to capture the global image representation for classification tasks. The input image $I$ is thus represented as a sequence of $n+1$ tokens $Z^{0}=\{z_{0}^{0},z_{1}^{0},\dots,z_{n}^{0}\}$, each of dimension $d$, with positional encodings added to retain spatial information.

The transformation of the initial sequence $Z^{0}\in\mathbb{R}^{(n+1)\times d}$ through the ViT involves $L$ layers, each performing a series of operations. Specifically, each layer $l$ applies multi-head self-attention (MSA) followed by a multilayer perceptron (MLP) block, both with residual connections:

$$ $\displaystyle\hat{Z}^{l}$ $\displaystyle=\text{MSA}^{l}(Z^{l-1})+Z^{l-1},$ (1) $\displaystyle Z^{l}$ $\displaystyle=\text{MLP}^{l}(\hat{Z}^{l})+\hat{Z}^{l}.$ $$

After $L$ layers, the image representation can be obtained via various strategies:

[CLS] token: The class token approach, as introduced in the original ViT paper [], uses the processed class token as the image embedding $\bar{z}_{[CLS]}=z^{L}_{0}$. This method relies on the transformer’s ability to aggregate information from the patch tokens into the class token during training.

Attentional Pooler: as used in [, , ], employs an additional multi-head attention layer with a learnable query token $q_{pool}\in\mathbb{R}^{d}$. This token interacts with the final layer patch tokens to produce the pooled representation $\bar{z}_{AttnPool}$:

$$ $\bar{z}_{AttnPool}=\text{softmax}\left(\frac{q_{pool}\cdot(W_{K}Z^{L})^{T}}{ \sqrt{d}}\right)(W_{V}Z^{L}),$ (2) $$

where $W_{K},W_{V}\in\mathbb{R}^{d\times d}$ are learnable projection matrices.

Independent of the feature aggregation strategy, it is important for an explainability method to account for the iterative nature of feature formation in ViTs.
The goal should be to capture the contributions of all layers for the final representation, providing insights into the model’s decision-making process.

### 3.2 Explainability Method: LeGrad

We denote the output tokens of each block $l$ of the ViT as $Z^{l}=\{z_{0}^{l},z_{1}^{l},\dots,z_{n}^{l}\}\in\mathbb{R}^{(n+1)\times d}$, where $d$ is the dimensionality of each token.

Consider a classifier $\mathcal{C}\in\mathbb{R}^{d\times C}$, where $C$ is the number of classes. This classifier can be learned during training for a supervised classification task (e.g., ImageNet) or can be formed from text embeddings of class descriptions for a zero-shot classifier, as in the case of vision-language models like CLIP, generating a prediction $\bar{y}$.
This prediction $\bar{y}$ is obtained by passing the aggregated feature representation of the ViT, noted $\bar{z}$, through the classifier $\mathcal{C}$ and selecting the class with the highest activation:

$$ $\bar{y}=\bar{z}\cdot\mathcal{C}\in\mathbb{R}^{C},$ (3) $$

The following is the description of how to obtain the explainability map for a single layer, using the last layer as an example and then how to generalize it to multiple layers. The overall method is visualized Figure 2.

Figure: Figure 3: LeGrad for a single layer.

Process for a Single Layer: To compute a 2D map that highlights the image regions most influential for the model’s prediction of a particular class, we focus on the activation with respect to the target class $\hat{c}$, denoted by $s=\bar{y}_{[\hat{c}]}$. The attention operation within a ViT is key to information sharing, and thus our method concentrates on this process.

We compute the gradient of the activation $s$ with respect to the attention map of the last layer, as shown in Figure 3, denoted as $\mathbf{A}^{L}\in\mathbb{R}^{h\times n\times n}$:

$$ $\nabla\mathbf{A}^{L}=\frac{\partial s}{\partial\mathbf{A}^{L}}\in\mathbb{R}^{h \times(n+1)\times(n+1)},$ (4) $$

where $h$ is the number of heads in the self-attention operation. Negative contributions are eliminated by applying a ReLU function, and the gradient is averaged across the patch and head dimensions:

$$ $\hat{E^{L}}(s)=\frac{1}{h\cdot(n+1)}\sum_{h}\sum_{i}\left(\nabla\mathbf{A}^{L} _{h,i,.}\right)^{+}\in\mathbb{R}^{n+1}.$ (5) $$

The final explainability map is obtained by removing the column corresponding to the [CLS] token to isolate the influence of the patch tokens, reshaping it into a 2D map, and applying min-max normalization:

$$ $E^{L}(s)=\text{norm}(\text{reshape}(\hat{E}^{L}(s)_{1:}))\in\mathbb{R}^{W \times H}.$ (6) $$

Process for Multiple Layers: Recognizing that information aggregation occurs over several layers, we extend the process to all layers. For each layer $l$, we calculate the activation score $s^{l}$ using the intermediate tokens $Z^{l}$ and derive the explainability map accordingly:

$$ $\hat{E}^{l}(s^{l})=\frac{1}{h\cdot(n+1)}\sum_{h}\sum_{i}\left(\nabla\mathbf{A} ^{l}_{h,i,.}\right)^{+}\in\mathbb{R}^{n+1}.$ (7) $$

Prior to normalization, we average the explainability maps from each layer:

$$ $\displaystyle\bar{\mathbf{E}}$ $\displaystyle=\frac{1}{L}\sum_{l}\hat{E}^{l}(s^{l})_{1:}$ (8) $\displaystyle\mathbf{E}$ $\displaystyle=\text{norm}(\text{reshape}(\bar{\mathbf{E}}))\in R^{W\times H}.$ $$

This approach queries each patch token at a given layer about its influence on the prediction at that stage.

Adaptation to Attentional Pooler: For ViTs using an attentional pooler (e.g. SigLIP []), a slight modification is made to compute the activation $s^{l}$ at each layer. We apply the attentional pooler module $Attn_{pool}$ to each intermediate representation $Z^{l}$ to obtain a pooled query $q^{l}\in\mathbb{R}^{d}$. The activation $s^{l}$ with respect to the desired class $c$ is then computed as $s^{l}=q^{l}\cdot\mathcal{C}_{:,c}\in\mathbb{R}$. Instead of considering the self-attention map, we use the attention map of the attentional pooler, denoted $\mathbf{A}_{pool}\in\mathbb{R}^{h\times 1\times n}$. Thus, for every layer $l$, $\nabla A^{l}=\frac{\partial s^{l}}{\partial A^{l}_{pool}}$.

## 4 Experiments

### 4.1 Evaluation Settings

Implementation details:
For the empirical evaluation of LeGrad, we utilized a suite of Vision Transformer models sourced from the OpenCLIP library [], trained on the LAION dataset []. The models employed in our experiments include ViT-B/16, ViT-L/14, ViT-H/14, and ViT-bigG/14.For SigLIP [], we use the ViT-B/16 with the "webli" weights. Each model processes input images at a resolution of $224\times 224$ pixels.
A key advantage of LeGrad is its hyperparameter-free nature, which obviates the need for hyperparameter tuning.
Furthermore, we do not retraining or fine-tune the vision-language models, thereby demonstrating their capabilities to localize queries by solely being trained on image-level data without any localization annotations during training or fine-tuning phases.

Object Segmentation: Following standard benchmarks [, , ] we evaluate the ability of explainability methods to accurately localize an object in the image. $\diamondsuit$Task: To do so, we generate image heatmaps based on the activation of the groundtruth class for models trained with a classifier or based on the the class descriptions "A photo of a [class]" for vision-language models. Subsequently, we apply a threshold to binarize these heatmaps (using a threshold of 0.5), thereby obtaining a foreground/background segmentation.$\diamondsuit$Metric: We assess the quality of this segmentation by computing the mIoU (mean Intersection over Union), pixel accuracy and the mAP (mean Average Precision) zero-shot segmentations produced by different explainability methods. This benchmark serves as a testbed for evaluating the spatial fidelity of the explainability method.
$\diamondsuit$Dataset: In our evaluation of heatmap-based explainability methods, we adhere to a standardized protocol and use the ImageNet-Segmentation dataset [], with $4,276$ images that provide segmentation annotations.

Open-vocabulary: For vision-language models, we extend our evaluation to encompass open-vocabulary scenarios by generating explainability maps for arbitrary text descriptions.
This allows us to assess the quality of explainability methods beyond the common classes found in ImageNet.
$\diamondsuit$Task: We generate a heatmap for each class object present in the image, binarize them (using a threshold of 0.5) and assess the localization accuracy.
$\diamondsuit$Dataset/Metric: We employ the OpenImageV7 dataset [], which offers annotations for a diverse array of images depicting a broad spectrum of objects and scenarios. Following [], our evaluation utilizes the point-wise annotations of the validation set, which contains 36,702 images labeled with 5,827 unique class labels. Each image is associated with both positive and negative point annotations for the objects present. In our analysis, we focus exclusively on the classes that are actually depicted in each image.

Perturbation-Based: To measure LeGrad’s ability to faithfully identify features important to the model, we employ a perturbation-based methodology. $\diamondsuit$Task: Given a classification dataset, we begin by generating explainability maps for every image using the different explainability methods. The analysis then consists of two complementary perturbation tests: positive and negative. In the positive perturbation test, image regions are occluded in descending order of their attributed relevance, as indicated by the explainability maps.
Conversely, the negative perturbation test occludes regions in ascending order of relevance (see the Annex for more details and visualizations of positive/negative perturbations).
$\diamondsuit$Metric: For both perturbation scenarios, we quantify the impact on the model’s accuracy by computing the area under the curve (AUC) for pixel erasure, which ranges from 0% to 90%. This metric provides insight into the relationship between the relevance of image regions and the model’s performance. The tests are applicable to both the predicted and ground-truth classes, with the expectation that class-specific methods will show improved performance in the latter. This dual perturbation approach enables a comprehensive evaluation of the network’s interpretability by highlighting the importance of specific image regions in the context of the model’s classification accuracy.
$\diamondsuit$Dataset: Following common practices, we use the ImageNet validation set, which contains $50K$ images and covers $1,000$ classes.

### 4.2 Comparison to State-of-the-Art

Segmentation Analysis:
We first evaluate LeGrad as well as other methods in the context of image segmentation using the ImageNet-segmentation dataset. As shown in Table 2, LeGrad achieved a mIoU of 58.7%, surpassing other SOTA explainability methods. Notably, it outperformed CheferCAM as a gradient-based method for ViTs, and TextSpan as a non-gradient-based method, indicating its robustness in capturing relevant image features for classification tasks.

Open-Vocabulary Performance:
We further evaluate the performance on the OpenImagesV7 dataset testing the capabilities of all methods in handling diverse object categories. Note that for GradCAM we searched over the layers and took the one that was performing the best. As shown in Table 2, LeGrad outperforms all other SOTA methods, with performance gains ranging from $2\times$ to $5\times$ compared to the second-best performing method. This can be seen as an indicator for LeGrad’s capacity for fine-grained recognition.

Perturbation-Based Evaluation:
We further performed the perturbation-based evaluation on the ImageNet-val datasets. As detailed in Table 3, LeGrad’s performance is here comparable to TextSpan for positive perturbations and slightly superior for negative perturbations. For all other methods, LeGrad outperformes both attention-based (e.g., "rollout" and "raw attention") and gradient-based methods (e.g., GradCAM, AttentionCAM, and CheferCAM) across various model sizes and for both predicted and ground truth classes, emphasizing its ability to identify and preserve critical image regions for accurate classification.

Performance on SigLIP:
We finally evaluate the adaptability b regarding the performance on SigLIP-B/16, a Vision-Language model employing an attentional pooler as shown in Table 4.
The results underscore the methods performance across both negative and positive perturbation-based benchmarks. Notably, in the open-vocabulary benchmark on OpenImagesV7, LeGrad achieved a p-mIoU of 25.4, significantly surpassing GradCAM’s 7.0 p-mIoU, the next best method.
These findings affirm the versatility of LeGrad, demonstrating its robust applicability to various pooling mechanisms within Vision Transformers.
Further details on the methodological adaptations of LeGrad and other evaluated methods for compatibility with SigLIP are provided in the annex.

**Table 1: Object Segmentation: method comparison on ImageNet-S using a CLIP(ViT-B/16) model.**
| Method | Pixel Acc.$\uparrow$ | mIoU$\uparrow$ | mAP$\uparrow$ |
| --- | --- | --- | --- |
| LRP | 52.81 | 33.57 | 54.37 |
| Partial-LRP | 61.49 | 40.71 | 72.29 |
| rollout | 60.63 | 40.64 | 74.47 |
| Raw attention | 65.67 | 43.83 | 76.05 |
| GradCAM | 70.27 | 44.50 | 70.30 |
| CheferCAM | 69.21 | 47.47 | 78.29 |
| TextSpan | 73.01 | 40.26 | 81.4 |
| LeGrad | 77.52 | 58.66 | 82.49 |

**Table 3: SOTA Perturbation Performance: Comparison of interpretability methods on the ImageNet-val using a different model size.**
|  |  | Negative |  | Positive |  |  |
| --- | --- | --- | --- | --- | --- | --- |
|  | Method | Predicted $\uparrow$ | Target $\uparrow$ |  | Predicted $\downarrow$ | Target $\downarrow$ |
| ViT-B/16 | rollout | 44.36 | 44.36 |  | 23.03 | 23.03 |
| Raw attention | 46.97 | 46.97 |  | 20.23 | 20.23 |  |
| GradCAM | 32.01 | 45.26 |  | 36.52 | 22.86 |  |
| AttentionCAM | 39.56 | 39.68 |  | 34.28 | 34.12 |  |
| CheferCAM | 47.91 | 49.28 |  | 18.66 | 17.70 |  |
| TextSpan | 50.92 | 52.81 |  | 15.10 | 14.26 |  |
| LeGrad | 50.24 | 52.27 |  | 15.06 | 13.97 |  |
| ViT-L/14 | rollout | 40.46 | 40.46 |  | 29.46 | 29.46 |
| Raw attention | 47.14 | 47.14 |  | 23.49 | 23.49 |  |
| GradCAM | 45.24 | 47.08 |  | 23.81 | 22.68 |  |
| AttentionCAM | 45.81 | 45.84 |  | 31.18 | 31.03 |  |
| CheferCAM | 49.69 | 50.37 |  | 20.67 | 20.14 |  |
| TextSpan | 53.17 | 54.42 |  | 16.77 | 16.12 |  |
| LeGrad | 53.11 | 54.48 |  | 15.98 | 15.23 |  |

**Table 4: SOTA comparison on SigLIP-B/16: Comparison of interpretability methods on perturbation-based tasks on ImageNet-val and open-vocabulary localization on OpenImagesV7.**
|  | ImageNet | OpenImagesV7 |  |  |  |
| --- | --- | --- | --- | --- | --- |
|  | Negative | Positive |  |  |  |
| Method | Predicted $\uparrow$ | Target $\uparrow$ | Predicted $\downarrow$ | Target $\downarrow$ | p-mIoU $\uparrow$ |
| rollout | 47.81 | 47.81 | 25.74 | 25.74 | 0.07 |
| Raw attention | 44.42 | 44.42 | 25.85 | 25.85 | 0.09 |
| GradCAM | 41.25 | 44.42 | 35.10 | 33.50 | 6.97 |
| AttentionCAM | 45.62 | 45.71 | 45.01 | 44.92 | 0.19 |
| CheferCAM | 47.12 | 49.13 | 22.35 | 21.15 | 1.94 |
| LeGrad | 50.08 | 51.67 | 18.48 | 17.55 | 25.40 |

### 4.3 Ablation

#### 4.3.1 Ablation Studies on Layer Contributions

Ablation studies are conducted to understand the impact of the number of layers considered in the computation of LeGrad’s explainability maps. We investigate this aspect across two distinct benchmarks: a perturbation-based evaluation using the ImageNet validation set and an open-vocabulary segmentation task on the OpenImagesV7 dataset. The experiments are performed on various model sizes from the openCLIP library, including ViT-B/16, ViT-L/14, and ViT-H/14.

Perturbation-Based Evaluation:
In the perturbation-based evaluation, see Figure 4(a), we employ a negative perturbation test as described in Section 4.1, using the ground-truth class for reference. The results indicate that the ViT-B/16 model’s performance is optimal when fewer layers are included in the explainability map computation. Conversely, larger models such as ViT-L/14 and ViT-H/14 show improved performance with the inclusion of more layers. This suggests that in larger models, the aggregation of information into the [CLS] token is distributed across a greater number of layers, necessitating a more comprehensive layer-wise analysis for accurate explainability.

Open-Vocabulary Segmentation:
For the open-vocabulary segmentation task, Figure 4(b), all models demonstrate enhanced performance with the inclusion of additional layers in the explainability map computation. The optimal number of layers varies with the model size, with larger models requiring a greater number of layers. This finding aligns with existing literature [], which suggests that the information aggregation process in ViTs is more distributed in larger architectures. However, it is also observed that beyond a certain number of layers, the performance gains plateau, indicating that the inclusion of additional layers does not further enhance the explainability map’s quality.

These ablation studies underscore the importance of layer integration in the construction of explainability maps for Vision Transformers.
They further show that while the inclusion of more layers can be beneficial, especially for larger models, there exists an optimal range beyond which no additional interpretability is gained. Indeed, we note in Figure 4 that including the last $40\%$ of the ViT layers in LeGrad’s explainability computation is a good trade-off. This can be helpful for the practical application of LeGrad, as it allows for a more targeted and computationally efficient approach to generating explainability maps without compromising on their quality.

Figure: (a) AUC for Negative perturbation on ImageNet-val for different layers used for LeGrad

#### 4.3.2 Ablations of ReLU and Layer Aggregation

We finally scrutinize the design choices underpinning LeGrad in Table 5. Specifically, we investigate the effect of discarding negative gradients before aggregating layer-specific explainability maps vai ReLU, as well as the implications of leveraging intermediate feature tokens $Z^{l}$ to compute gradients for each respective layer. We use the framework of the perturbation benchmark delineated in Section 4.1 and both ViT-L/14 and ViT-H/14 models. The results indicate that the omission of either component induces decline in performance, thereby affirming the role these elements play in the architecture of the method.

**Table 5: Ablation study: "ReLU" corresponds to whether or not negative gradients are set to 0. "All layers" corresponds to whether or not the intermediate tokens are used to compute the gradient for every layer or if only the features from the last layer are used. Numbers are the AUC score for the perturbation base benchmark using the target class to compute the explainability map.**
|  | L/14 | H/14 |  |  |  |
| --- | --- | --- | --- | --- | --- |
| ReLU | All Layers | Negative$\uparrow$ | Positive$\downarrow$ | Negative$\uparrow$ | Positive$\downarrow$ |
| ✗ | ✗ | 47.81 | 20.80 | 57.57 | 21.73 |
| ✓ | ✗ | 49.32 | 19.95 | 59.55 | 19.50 |
| ✗ | ✓ | 52.01 | 16.80 | 60.28 | 18.26 |
| ✓ | ✓ | 54.48 | 15.23 | 61.72 | 18.26 |

### 4.4 Qualitative analysis

#### 4.4.1 Ablation Studies on Layer Contributions

To elucidate the interpretability of Vision Transformers (ViTs) through our proposed LeGrad method, we extend our examination beyond quantitative ablation studies to a qualitative analysis. This analysis scrutinizes the localization maps generated by LeGrad for the last ten layers of various ViT models, as well as the final map that merge these individual layer contributions. The experiments are performed on various model sizes from the openCLIP library, including ViT-B/16, ViT-L/14, and ViT-H/14.

The qualitative results, illustrated in Figure 5, reveal that the localization of the prompt is not confined to a single layer but is distributed across multiple layers. This observation is consistent with the findings from the ablation studies in Section 4.3, which underscore the necessity of incorporating multiple layers into the explainability framework to capture the full scope of the model’s decision-making process.

A notable distinction is visible in the behavior of smaller models, such as ViT-B/16, with that of larger counterparts like ViT-L/14 and ViT-H/14. In the case of ViT-B/16, the explainability signal predominantly emanates from the final layers, particularly the last two. Conversely, in larger models, earlier layers also substantially influence the merged explainability map. For instance, in Figure 5, layers 8 and 9 of ViT-H/14 appear to distinctly focus on the heads of the cats, while other layers’ contributions are more diffused. This observation not only corroborates the utility of incorporating multiple layers into the explainability analysis but also suggests a more distributed information aggregation process into the [CLS] token in larger models, as posited in the literature [].

The qualitative analysis thus reinforces the hypothesis that the mechanism by which ViTs integrate information across layers is size-dependent, with larger models engaging a broader range of layers in the synthesis of the global feature representation. This is important for the practical application of LeGrad, as it informs the selection of layers to be included in the explainability map computation, ensuring both computational efficiency and the retention of fidelity.

Figure: Figure 5: Qualitative analysis of the impact of each layer for different model sizes using "a photo of a cat" as prompt. In smaller models the explainability signal predominantly emanates from the final layers while in larger models lower layers also contribute to the explainability map.

#### 4.4.2 Qualitative Comparison to SOTA

Here, we present a qualitative analysis of the explainability maps generated by LeGrad in comparison to other state-of-the-art (SOTA) methods. The visual results are depicted in Figure 6, which includes a diverse set of explainability approaches such as gradient-based methods (e.g., CheferCAM, GradCAM), attention-based methods (e.g., Raw Attention weights visualization, Rollout), and methods that integrate intermediate visual representations with text prompts (e.g., TextSpan).

Our observations indicate that raw attention visualizations tend to highlight a few specific pixels with high intensity, often associated with the background rather than the object of interest. This pattern, consistent with findings in the literature [, ], suggests that certain tokens disproportionately capture attention weights. Consequently, methods that rely on raw attention weights to construct explainability maps, such as CheferCAM, exhibit similar artifacts. For instance, in the localization of "Wheel" (Figure 6, row 2), the wheel is marginally accentuated amidst a predominance of noisy, irrelevant locations.

In contrast, LeGrad is resilient to such distractions by leveraging gradients with respect to attention maps. The presence of uniform noisy activations across different prompts results in minimal gradients for these regions, effectively filtering them out from the final heatmaps. This characteristic enables LeGrad to produce more focused and relevant visual explanations.

The presence of noisy activations is also observed in methods like TextSpan and GradCAM, which utilize intermediate feature representations. These artifacts tend to surface in the same spatial regions as those highlighted by raw attention weights, suggesting a potential link to tokens with inflated attention. While the underlying cause of this behavior demands further investigation, it is beyond the scope of the current work.
Eventually, Figure 6 showcases LeGrad’s capability to accurately localize a wide range of objects and concepts, from commonplace items like "baskets" to abstract notions such as "numbers" and activities like "swimming." The method’s robustness and versatility underscore its effectiveness in providing interpretable insights into the decision-making processes of vision transformers.

Figure: Figure 6: SOTA Qualitative Comparison: visual comparison of different explainability methods on images from OpenImagesV7.

## 5 Conclusion

In this work, we proposed LeGrad as an explainability method that highlights the decision-making process of Vision Transformers (ViTs) across all layers various feature aggregation strategies. We validated the method’s effectiveness in generating interpretable visual explanations that align with the model’s reasoning. Our approach offers a granular view of feature formation sensitivity, providing insights into the contribution of individual layers and attention mechanisms.
The evaluation across object segmentation, perturbation-based metrics, and open-vocabulary scenarios underscores LeGrad’s versatility and fidelity in different contexts. By facilitating a deeper understanding of ViTs LeGrad paves the way for more transparent and interpretable foundation models.

## 7 Overview

In this supplementary material, we first provide in section 8 a link to a demo of our novel method, LeGrad, showcasing its practical application. Detailed information regarding the implementation, including pre-trained weights and baseline methodologies for comparative evaluation, is presented in section 9. An expanded evaluation of LeGrad’s performance, specifically on Vision Transformers (ViT) trained with the ImageNet dataset, is documented in section 10. Section 10.2 offers a visual representation of the perturbation benchmarks utilized to assess the efficacy of various explainability approaches. Additional qualitative examples are provided in section 10.3 to further illustrate the capabilities of our method. Lastly, section 11 includes a disclaimer addressing the use of personal and human subjects data within our research.

## 8 Demo

An image is worth a $1000$ words, we, therefore, provide a demo at:
[https://huggingface.co/spaces/WalidBouss/LeGrad-Demo](https://huggingface.co/spaces/WalidBouss/LeGrad-Demo).

## 9 Implementation details

### 9.1 Pretrained weights

The experiments conducted in our study leverage a suite of models with varying capacities, including ViT-B/16, ViT-L/14, ViT-H/14, and ViT-bigG/14. These models are initialized with pretrained weights from the OpenCLIP library respectively identified by: "laion2b_s34b_b88k", "laion2b_s32b_b82k", "laion2b_s32b_b79k", and "laion2b_s39b_b160k". For the SigLIP method, we utilize the ViT-B/16 model equipped with the "webl" weights.

### 9.2 Detailed Description of Baselines

In this section, we provide a concise overview of the baseline methods against which our proposed approach is compared.

GradCAM: While originally designed for convolutional neural networks (CNNs), GradCAM can be adapted for Vision Transformers (ViTs) by treating the tokens as activations. To compute the GradCAM explainability map for a given activation $s$, we calculate the gradient of $s$ with respect to the token dimensions. The gradients are aggregated across all tokens and serve as weights to quantify the contribution of each token dimension. Formally, for intermediate tokens $Z^{l}=\{z_{0}^{l},z_{1}^{l},\dots,z_{n}^{l}\}\in\mathbb{R}^{(n+1)\times d}$, the GradCAM map $E_{GradCAM}$ is defined as:

$$ $\displaystyle w$ $\displaystyle=\frac{1}{n}\sum_{i=0}^{n}\frac{\partial s}{\partial z_{i}^{l}} \quad\in\mathbb{R}^{d\times 1\times 1}$ (9) $\displaystyle\hat{E}_{GradCAM}$ $\displaystyle=\left(\frac{1}{d}\sum_{k=1}^{d}w_{d}*Z^{l}_{1:,d}\right)^{+} \quad\in\mathbb{R}^{n}$ $\displaystyle E_{GradCAM}$ $\displaystyle=\text{norm}(\text{resize}(\hat{E}_{GradCAM}))\quad\in\mathbb{R}^ {W\times H},$ $$

with $d$ representing the token dimension, $*$ denoting element-wise multiplication, and the superscript $+$ indicating the ReLU operation. We empirically determined that applying GradCAM to layer 8 of ViT-B/16 yields optimal results.

AttentionCAM: This method extends the principles of GradCAM to ViTs by utilizing the attention mechanism within the transformer’s architecture. AttentionCAM leverages the gradient signal to weight the attention maps in the self-attention layers. Specifically, for the last block’s self-attention maps $\mathbf{A}^{L}$, the AttentionCAM map $E_{AttnCAM}$ is computed as:

$$ $\begin{aligned} \nabla\mathbf{A}^{L}&=\frac{\partial s}{\partial\mathbf{A}^{L} }\in\mathbb{R}^{h\times(n+1)\times(n+1)}\\ w&=\frac{1}{n^{2}}\sum_{i,j}\nabla\mathbf{A}^{L}_{:,i,j}\in\mathbb{R}^{h}\\ \hat{E}_{AttnCAM}&=\sum_{p}^{h}\left(w_{p}*A^{L}_{p,:,:}\right)\in\mathbb{R}^{ (n+1)\times(n+1)}\\ E_{AttnCAM}&=\text{norm}(\text{resize}(\hat{E}_{AttnCAM})_{0,1:})\end{aligned},$ (10) $$

where $h$ denotes the number of heads in the self-attention mechanism.

Raw Attention: This baseline considers the attention maps from the last layer, focusing on the weights associated with the [CLS] token. The attention heads are averaged and the resulting explainability map is normalized. The Raw Attention map $E_{Attn}$ is formalized as:

$$ $\displaystyle\hat{E}_{Attn}$ $\displaystyle=\mathbf{A}^{L}_{:,0,1:}\in\mathbb{R}^{h\times 1\times n}$ (11) $\displaystyle E_{Attn}$ $\displaystyle=\text{norm}(\text{resize}(\frac{1}{h}\sum_{k=1}^{h}(\hat{E}_{ Attn})_{k}))\quad\in\mathbb{R}^{W\times H}$ $$

These baselines provide a comprehensive set of comparative measures to evaluate the efficacy of our proposed method in the context of explainability for ViTs.

### 9.3 Adaptation of Baseline Methods to Attentional Pooler

In the main manuscript, we introduced our novel method, LeGrad, and its application to Vision Transformers (ViTs) with attentional poolers. Here, we provide supplementary details on how LeGrad and other baseline methods were adapted ViTs employing attentional poolers:

CheferCAM:
Following the original paper [] that introduces CheferCAM, we considered the attentional pooler as an instance of a "decoder transformer" and applied the relevancy update rules described in equation (10) of that paper [], (following the example of DETR). Since the attentional pooler has no skip connection we adapted the relevancy update rule not to consider the skip connection.

AttentionCAM:
For AttentionCAM, instead of using the attention maps of the last layer, we use the attention maps of the attentional pooler. We found this variant to work better.

Raw Attention:
Similarly, the Raw Attention baseline was adjusted by substituting the attention maps from the last self-attention layer with those from the attentional pooler.

Other Baselines:
For the remaining baseline methods, no alterations were necessary. These methods were inherently compatible with the attentional pooler, and thus could be applied directly without any further adaptation.

The adaptations described above ensure that each baseline method is appropriately tailored to the ViTs with attentional poolers, allowing for a fair comparison with our proposed LeGrad method.

### 9.4 Mitigation of sensitivity to irrelevant regions:

We observe that for all evaluated explainability methods, SigLIP displays high activations in image regions corresponding to the background. These activations appeared to be invariant to the input, regardless of the gradient computation’s basis, these regions were consistently highlighted. To address this issue, we computed the explainability map for a non-informative prompt, specifically "a photo of". We then leveraged this map to suppress the irrelevant activations.

Namely, for an activation $s$ under examination, we nullify any location where the activation exceeds a predefined threshold (set at 0.8) in the map generated for the dummy prompt. Formally, let $E^{s}$ denote the explainability map for activation $s$, and $E^{empty}$ represent the map for the prompt "a photo of". The correction procedure is defined as follows:

$$ $E^{s}_{E^{empty}>th}=0,$ (12) $$

where $th=0.8$. This method effectively addresses the issue without resorting to external data regarding the image content.

## 10 Additional results

### 10.1 Image Classification

In this section, we extend our evaluation of the proposed LeGrad method to Vision Transformers (ViTs) that have been trained on the ImageNet dataset for the task of image classification. The results of this evaluation are presented in Table 7, also providing a comparison with other state-of-the-art interpretability methods.

It shows that LeGrad achieves superior performance on the perturbation-based benchmark, particularly in scenarios involving positive perturbations.

Another observation is that even elementary interpretability approaches, such as examining the raw attention maps from the final attention layer of the ViT, demonstrate competitive results. In fact, these basic methods surpass more complex ones like GradCAM (achieving an AUC of 53.1 versus 43.0 for negative perturbations).

**Table 7: SOTA comparison on ViT-B/16: Comparison of interpretability methods on perturbation-based tasks on ImageNet-val for a ViT trained on ImageNet.**
|  | Negative | Positive |  |  |
| --- | --- | --- | --- | --- |
| Method | Predicted $\uparrow$ | Target $\uparrow$ | Predicted $\downarrow$ | Target $\downarrow$ |
| rollout [] | 53.10 | 53.10 | 20.06 | 20.06 |
| Raw attention | 45.55 | 45.55 | 24.01 | 24.01 |
| GradCAM [] | 43.17 | 42.97 | 26.89 | 26.99 |
| AttentionCAM [] | 41.53 | 42.03 | 33.54 | 34.05 |
| Trans. Attrib. [] | 54.19 | 55.09 | 17.01 | 16.36 |
| Partial-LRP [] | 50.28 | 50.29 | 19.82 | 19.80 |
| CheferCAM [] | 54.68 | 55.70 | 17.30 | 16.75 |
| LeGrad | 54.72 | 56.43 | 15.20 | 14.13 |

### 10.2 Pertubation example

Figure 8 illustrates the perturbation-based benchmark of Section 4.1 in the main paper. Given the explainability map generated by the interpretatability method, for the negative (respectively positive) we progressively remove the most important (respectively the least important) part of the image. We then look at the decline in the model accuracy.

Figure: Figure 8: Example of positive/negative perturbations: illustration of positive and negative perturbations used in the perturbation-based benchmark. (Top row): positive perturbation. Bottom row: negative perturbations

### 10.3 Qualitative examples

Finally, Figure 9 provides additional qualitative comparisons with state-of-the-art explainability methods, illustrating the efficacy of the proposed approach.

Figure: Figure 9: Qualitative Comparison to SOTA: visual comparison of different explainability methods on images from OpenImagesV7

## 11 Personal and human subjects data

We acknowledge the use of datasets such as ImageNet and OpenImagesV7, which contain images sourced from the internet, potentially without the consent of the individuals depicted. We also recognize that the VL models used in this study were trained on the LAION-2B dataset, which may include sensitive content. We emphasize the importance of ethical considerations in the use of such data.

## Peer Reviews

confidence: 3
contribution: 2
initial_rating: 5
presentation: 3
questions: NA
soundness: 3
strengths: - LeGrad effectively utilizes a layer-wise approach to explainability, allowing it to aggregate gradients from multiple layers of the Vision Transformer. This comprehensive analysis provides a more nuanced understanding of how different layers contribute to the model's predictions, enhancing interpretability.  

- By focusing on gradients with respect to attention maps, LeGrad captures the sensitivity of feature representations, enabling the generation of relevancy maps that highlight the most influential parts of an image.

- LeGrad demonstrates strong performance across various challenging tasks, such as object segmentation and open-vocabulary detection, achieving comparable results compared to other state-of-the-art explainability methods.
summary: This paper introduces LeGrad, a new explainability method tailored for Vision Transformers (ViTs), which leverages gradients with respect to attention maps to generate relevancy maps that highlight the most influential parts of an input image for model predictions. By conducting a layer-wise analysis, LeGrad aggregates information across multiple layers, revealing the contributions of individual layers to the model's decision-making process. The method is evaluated across various tasks, including object segmentation and open-vocabulary detection, demonstrating good performance and more focused visual explanations compared to existing state-of-the-art methods like GradCAM and CheferCAM.
weaknesses: - Missing references [1,2,3]. 

- The idea of the proposed method sounds very normal. It is very common and has been well studied to use gradient and attention to localize the object in Transformer-based architecture. Talking about text-image alignment, [4] also shows a more precise segmentation results than the proposed method.

- Figure 4 shows typo in the x-axis. The x-axis means the "% of accumulated layer", so it should be 10, 20, ..., 100 rather than 0.1, 0.2, ..., 1.0.



References:  

[1] IA-RED^2: Interpretability-Aware Redundancy Reduction for Vision Transformers. 

[2]  Emerging properties in self-supervised vision transformers.  

[3] Exploring Visual Explanations for Contrastive Language-Image Pre-training. 

[4] Open-Vocabulary Panoptic Segmentation with Text-to-Image Diffusion Models

---

confidence: 5
contribution: 2
initial_rating: 5
presentation: 2
questions: The visualization results (Figures 1, 5, 7, 11) indicate that LeGrad may produce a noisy background in the explanation maps, a problem studied in previous work [1, 2]. Is there any analysis on this issue? A possible reason is that the gradient signal is noisy in background areas and thus needs calibration. Can the vector norm methods in [1, 2] help improve the clearness in the background produced by LeGrad?

[1] Token Transformation Matters: Towards Faithful Post-hoc Explanation for Vision Transformer. CVPR
2024.

[2] Attention is Not Only a Weight: Analyzing Transformers with Vector Norms. EMNLP 2020.
soundness: 3
strengths: 1. Unlike previous works, this paper extends the scope of ViT explainability to include new models like OpenCLIP ViTs and new settings like open vocabulary.

2. this paper is well-written and easy to follow.

3. comprehensive experiments. Tables 1 and 2 show that LeGrad significantly outperforms the baseline methods.
summary: This paper studies the explainability of Vision Transformers (ViTs). The authors propose a new gradient based method, LeGrad, to produce explanation maps given pre-trained ViTs and specific inputs. LeGrad aggregates the gradient signal and combines the activations of the last and intermediate tokens to produce the merged explanation maps. Experiments show that the proposed LeGrad achieves better results in segmentation, perturbation, and open-vocabulary settings.
weaknesses: 1. the technical contribution of LeGrad is not very significant. The idea of using gradient as an explanation signal and aggregating information across layers [1, 2] has been well-studied and widely applied in the literature on ViT explainability, making this work incremental.

2. The method sums up the gradients across layers instead of performing matrix multiplication. This is different from the attention flow model in [1, 2, 3]. A detailed analysis of the motivation for summing up the signals would largely strengthen this paper.

3. Similar to weakness 2. The proposed LeGrad method mainly focuses on the gradient itself as an explanation signal instead of the activation. However, the reason behind this design has not been clearly explained.
[1] Generic Attention-model Explainability for Interpreting Bi-Modal and Encoder-Decoder Transformers. ICCV 2021.
[2] Transformer Interpretability Beyond Attention Visualization. CVPR 2021.

---

confidence: 4
contribution: 2
initial_rating: 3
presentation: 3
questions: See the weaknesses.
soundness: 2
strengths: Empirical results exhibit improved quality of saliency, which is validated across numerous benchmarks. I would also like to mention that detailed technical description such as how Grad-CAM can be implemented in ViT with respect to token axis may be valuable to several practitioners.
summary: This manuscript proposes LeGrad, a saliency method that visualizes the inner behavior of a vision transformer. After obtaining average over output tokens and mapping layer, LeGrad computes gradient of score with respect to attention map. The accumulation of gradients now becomes the saliency map that LeGrad provides.
weaknesses: Honestly speaking, I think that this manuscript provides not much significant advancement from existing variants of Grad-CAM or existing visualization methods used in ResNets. The summation over gradient with respect to score seems a minor variant of Grad-CAM and highly similar to those existing researches. I would say the novelty of the proposed method lies in detailed engineering such as computation over the token axis and reshaping operations to be compatible with ViTs. A new idea would be something like obtaining saliency maps over multiple layers and averaging them; though it may yield improved quality, it is essentially engineering, not the scientific finding. In summary, I think that LeGrad exhibits little advancement over existing saliency methods, and the proposed method seems mere engineering, not a scientific finding. I wonder whether the LeGrad exhibits improved theoretical properties or was just validated through empirical experiments.

Although the mapping model C can be easily implemented for vision-language models like CLIP, the standard ViT requires the mapping model C through additional classifier layers that are fine-tuned after appending them to ViT. In other words, the proposed method cannot be immediately applied to standard ViT and requires additional fine-tuning, which makes it difficult to be deployed immediately. Indeed, ViT with GAP is rarely used; standard ViT uses an output score obtained from the class token without average operation. This issue also raises limitations for the proposed method.

Minor comments: LayerNorm is omitted in Eq. 1.

---

confidence: 4
contribution: 3
initial_rating: 5
presentation: 1
questions: 1) Why are the results of CheferCam are not correlate with the results stated in the paper of Chefer et al.? for the negative they declare a result of 55.04 on the target (also on imagenet-val dataset) which I could'nt find in your table.
2) I didn't understand from the written if the C mapping is learned once only on top of the last layer and applied to all others? or you have L such mappings, one per layer? Moreover, if learned once only on ImageNet, have you used the same one on other analyzed datasets (i.e. audio)?
3) Explainability methods are preferable when they have some interpretability. How one can use your method in order to better understand the performance of ViT? for example - which layers/heads contributed the most? how can we use this method in order to debug misclassified samples (e.g. in case of classification).

Overall I think that the direction of analyzing each layer separately is a good one, with a potential to improve explainability and even interpretability. Although I think that the paper is lack of intuitions behind some choices, make it difficult for follow-up works. Moreover, the writing level of the paper is somewhat low and hard to follow.
soundness: 3
strengths: 1) The idea of dividing the explainability into a per-layer impact is sound and reasonable, potentially offering better insights into the information flow throughout the entire architecture.
2) The method is tested across a wide range of ViT adaptations and on classic classification as well as open-vocabulary settings, demonstrating its versatility.
3) The method compared in a versatile applications, especially audio to image tracking.
summary: This paper addresses the problem of explainable AI specifically designed for Vision Transformers. The authors propose using the derivative of the attention layer at each individual layer. They suggest treating each layer independently by applying a classification head that takes a function of the output from each layer and aggregates the impact across all heads and layers.
weaknesses: 1) The writing level is not up to standard, and this affects the clarity and coherence of the paper. The lack of a main chain of thought makes it difficult to follow.

1.a) Typos: influcened -> influenced (line 47), explanability -> explainability (line 49)

1.b) English mistakes: .. the explainability *(of)* those architectures remains a challenge (line 55), 
       This facilitates its adoption across various applications and architecture*(s)* (line 67)

1.c) Too long sentences, lack of appropriate punctuation (lines 69-72)

2) The research gap the paper aims to address is unclear. The introduction mentions that "the explainability of those architectures remains a challenge" but doesn’t fully explain why. LeGrad is then introduced as differ from CheferCam, but the main advances LeGrad provides over other methods are not clarified. Why are these specific advances? why are they needed and useful?

3) Lack of intuition behind architectural choices of the approach. For example:  Why using min-max normalization? why do you average on heads and layers in Eq. 5 (it is common to average heads, but not necessarily for layers)? why do you average patches together with CLS for the mapping in line 190 (I familiar with classification ViTs working either on CLS or on average of patch tokens but not the combination of both). All of these make it difficult to follow-up works to grasp the innovative delta and extend it. Moreover, the ablation is not cover parts of the method. Therefore, this is not clear what part of the method is mostly boost the performance.