CNN Interpretability with Multivector Tucker Saliency Maps for Self-Supervised Models
Authors: Aymene Mohammed Bouayed
Samuel Deslauriers-gauthier
Adrian IACOVELLI
David Naccache
Venue: Accepted by TMLR ()

Abstract: Interpreting the decisions of Convolutional Neural Networks (CNNs) is essential for understanding their behavior, yet it remains a significant challenge, particularly for self-supervised models. Most existing methods for generating saliency maps rely on reference labels, restricting their use to supervised tasks. EigenCAM is the only notable label-independent alternative, leveraging Singular Value Decomposition to generate saliency maps applicable across CNN models, but it does not fully exploit the tensorial structure of feature maps. In this work, we introduce the Tucker Saliency Map (TSM) method, which applies Tucker tensor decomposition to better capture the inherent structure of feature maps, producing more accurate singular vectors and values. These are used to generate high-fidelity saliency maps, effectively highlighting objects of interest in the input. We further extend EigenCAM and TSM into multivector variants—Multivec-EigenCAM and Multivector Tucker Saliency Maps (MTSM)—which utilize all singular vectors and values, further improving saliency map quality. Quantitative evaluations on supervised classification models demonstrate that TSM, Multivec-EigenCAM, and MTSM achieve competitive performance with label-dependent methods. Moreover, TSM enhances interpretability by approximately $50\%$ over EigenCAM for both supervised and self-supervised models. Multivec-EigenCAM and MTSM further advance state-of-the-art interpretability performance on self-supervised models, with MTSM achieving the best results.

Published in Transactions on Machine Learning Research (02/2025)
CNN Interpretability with Multivector Tucker Saliency Maps for Self-Supervised Models
Aymene Mohammed Bouayed DIÊNS, ENS, CNRS, PSL University, Paris, Fronce Be- Ys Research, France
symene.oouaeddeen.
Samuel
Gauthier
Centre Inria d'Université Côte d' d'Azur, Nice, Prance
Adrian laccovelli
adrian..acoovelliiteeee
Be- Bs Research, France
David Naccache
david. monddaanccachee@gaai.ccom
DIÉNS, ENS, CNRS, PSL University, Paris, Prance
Reviewed on OpenReview: https: // openreview. net/forunt forum? 3605
Abstract
Interpreting the decisions of Convolutional Neural Networks (CNNs) is essential for under- standing their behavior, yet it remains a significant. challenge, particularly for self-supervised models. Most existing methods for generating saliency maps rely on reference labels, re- stricting their use to supervised tasks. EigenCAM is the only notable iabeednnde alternative, leveraging Singular Value Decomposition to generate saliency maps applicable across CNN models, but it does not fully exploit the tensorial structure of feature maps. In this work, introduce the Tucker Saliency Map (TSM) method, which applies Tucker tensor decomposition to better capture the inherent structure of feature maps, producing more accurate singular vectors and values. These are used to generate high-fidelity saliency maps, effectively highlighting objects of interest in the input. We further extend EigenCAM and TSM into multivector variants Cuultivee-FigenAAAe and Multivector Tucker Saliency Maps (MTSM) which utilize all singular vectors and values, further improving saliency map quality. Quantitative evaluations supervised classification models demonstrate that TSM, Multivee-EigenCANe and MTSM achieve competitive performance with label-dependent methods. Moreover, TSM enhances interpretability by approximately 50% over EigenCAM for both supervised and self-supervised models. Multivee-LigenCAM and MTSM further advance state-of-the-art interpretability performance on self-supervised models, with MTSM achieving the best results.
1 Introduction
Convolutional Neural Networks (CNNs) demonstrate exceptional performance across various computer vision applications (Redmon et al., 2016; Krizhevsky al., 2017; Liu et al., 2021). These networks can be trained in supervised manner to tackle classification regression tasks (He et al., 2016) in self-supervised manner for image segmentation (Walsh et al., 2022; Kirillov et al., 2023), face recognition (Boutros et al., 2022), and learning image embeddings (Bardes et al., 2022). However, their inherent black-box nature poses significant challenges, partieularly in medicine (Cruciani et al., 2021; Bouayed et. al., 2022) and biometrics (Kim & Cho, 2021), where interpretability is crucial for trust and adoption.
Numerous works have attempted to address the challenge of interpreting CNN decisions, proposing diverse methods that can be broadly categorized into feature attribution methods and Class Activation Map (CAM)

---

Published in Transactions on Machine Learning Research (02/2025)
methods. Feature attribution methods involve either estimating locally interpretable model (Ribeiro et al., 2016) or propagating information from the network's output to its input using gradients (Springenberg et al., 2015), contribution scores (Bach et al., 2015; Shrikumar et al., 2017), or Shapley values (Lundberg & Lee, 2017). However, these methods often come with significant computational overhead, even when using approximations (Aas el al., 2021). In contrast, CAM methods (Zhou et al., 2016; Chattopadhay et al., 2018) perform a weighted sum of the feature map tensors, a multidimensionall array which preserves spatial relationships, and avoid backward propagation, yet they require a reference class to infer the weights (Chattopadhay et al., 2018; Wang et al., 2020; Desai & Ramaswamy, 2020; Schulz et al., 2020; Zhang et al., 2024).
Despite these advancements, a major limitation of current interpretability methods is their predominant focus on supervised classification models. Most existing techniques rely heavily on reference labels to generate saliency maps, rendering them label-dependent and restricting their applicability to supervised tasks. The only exception EigenCAM (Muhammad & Yeasin, 2020), which offers a habel-independent approach and can be applied to both supervised and sell-supervised models. However, EigenCAM has notable drawbacks: its use of matricization' disrupts the spatial relationships within the feature maps, particularly in the height and width dimensions. This limitation lcads to less accurate estimations of singular vectors and negatively impacts the quality of the resulting saliency maps.
In this work, we aim to propose a general label-independent interpretability method for CNNs that addresses the limitations of existing approaches, particularly EigenCAM. Specifically, we seek to answer the question: ' "Can tensor algebra provide a better estimation of the weight vector?". We answer the question positively through the introduction of the Tucker Saliency Map (TSM) method. TSM applies Tucker tensor decom- position (Kolda & Bader, 2009; Cheng et al., 2023) directly to the feature map tensor, preserving spatial relationships and resulting in different singular vectors. This enhancement allows for improved saliency maps that capture more variance in the first singular vector.
Furthermore, we extend both CigenCAM and TSM into multivector variants: Multivec-EigenCAle and Multivector Tucker Saliency Maps (MTSM), which leverage all singular vectors and values to create richer and more accurate saliency maps than their respective base methods.
We evaluate proposed methods in both label-dependent and label-independent settings. In the label- dependent context, TSM, Aallivee-FFgenCAMM and MTSM demonstrate competitive performance against six state-of-the-art label-dependent CAM methods while showing substantial improvements over EigenCAM. For the label-independent setting, we introduce a comprehensive evaluation framework tailored for self-supervised models, addressing a significant gap in the literature. This framework assesses the impact of saliency maps on downstream classification performance, the reproducibility of generated embeddings, and their similarity to ground truth segmentation maps. Our results consistently show that TSM outperforms EigenCAM,, establishing a new state-of-the-art in self-supervised CNN interpretability. Miieive and MTSM further enhance these results, particularly with MTSM leading in performance.
We outline our contributions as follows:
We introduce Tucker Saliency Map (TSM) for label-independent CNN interpretability, leveraging Tucker tensor decomposition to enhance saliency map quality.
We develop CAA and Multivector Tucker Saliency Maps (MTSM) to utilize all singular vectors for richer saliency maps.
We introduce a new framework for assessing saliency maps in self-supervised models, focusing on classification performance, embedding reproducibility and alignement with segmentation masks.
We evaluate the proposed methods in both label-dependent and label-independent settings, showing competitive performance against state-of-the-an methods. Also, we establish TSM a new state-of- the-art in selfssupervised interpretability, with MTSM furthering the state-of-the-art performance.
The remained of this paper is organized as follows: in Section 2 we introduce essential background definitions and notations. Section 3 reviews related work addressing various interpretability methods for CNN models.
Matricization the operation of transforming tensor into matrix (Kolda & Bader, 2009).
2

---

Published in Transactions on Machine Learning Research (02/2025)
The mechanics of our proposed methods are detailed in Sections 4 and 5. We rigorously test and verify the efficacy of our methods across diverse set of supervised and self-supervised CNN models in Section 6. Finally, we conclude and outline potential avenues for future exploration in Section 7.
2
Background
2.1 Class Activation Maps
Class Activation Maps (CAMs) (Zhou et al., 2016) are local interpretability methods for CNN based models. They help visualize the regions of the input image that significantly contribute to the CNN's prediction by analyzing the intermediate results. Formally, CAMs are defined as the sum of the feature maps resulting from the last convolutional layer of a CNN F E R.C.H.W, weighted by a vector w E RC as follows:
CAM = Σ W.F.
(1)
As observed from Equation 1, the weight vector w is the only unknown to determine?. Various works propose different methods to infer (Chattopadhay et al., 2018; Wang et al., 2020; Muhammad & Yeasin, 2020; Zhang el al., 2024). In Section 3.2, we provide an overview of the different CAM based methods with a focus on the EigenCAM method (Muhammad & Yeasin, 2020) which utilizes the SVD (Trefethen & Bau, 1997).
2.2 Matrix Decomposition and SVD
Matrix decomposition involves factorizing a matrix M m, into a product of matrices to simplify compu- tations and reveal underlying structures (Trefethen & Bau, 1997). Singular Value Decomposition (SVD) (Tro- fethen & Bau, 1997) is prominent matrix decomposition method, which expresses matrix M as Matrices U E mmm. and V E R consist of singular vectors and form a basis for the rows and columns of M, respectively. The matrix Σ E m,. is a diagonal matrix containing singular values on the diagonal, representing the information and variance encoded per direction of the new basis (Trefethen & Bau, 1997). However, the SVD exclusively applicable to matrices therefore can not be applicable higher dimensional data in the form of tensors without the need to undergo matricization operation. This ultimately results in loss of relationship information between the elements of flattened modes. Consequently, tensor algebra introduces multitude of decompositions which operate directly in the tensor space hence solving the relationship disruption problem.
UEVT,
2.3 Tensors and Tucker Decomposition
A tensor E R".."....." an algebraic object that can be thought of multi-dimensional array that general- izes matrices to higher dimensions, allowing for more complex data representations. Consequently, a multitude of operations matrices can be extended to tensors. One such extension is tensor decomposition (Kolda & Bader, 2009). Among the most well known tensor decompositions we find the CANDECOMP SADECCOMPPARAAALC (CP) decomposition (Kiers, 2000; Bove, 2010) and the Tucker tensor decomposition (Sheehan & Saad, 2007; Cheng et al., 2023). As noticed by Kolda & Bader (2009), CP decomposes a tensor as a sum of rank-one tensors with no additional constraints, whereas the Tucker tensor decomposition is a higher-order form of principal component analysis. Hence, the Tucker decomposition provides sorted singular vectors according singular values opposed to the CP decomposition. This property is of importance to the modeling of our proposed TSM methods and motivates our choice of the Tucker decomposition over the CP docomposition any other decomposition (See Section 4).
Focusing on the Tucker tensor decomposition, it decomposes a tensor T as
T C x ( X2 (2) X3 X (k),
(2)
2 Additional operations be applied the weighted sum depending on the CAM method. Among such operations the ReLU function and/or the min-max normalization.
3

---

Published in Transactions on Machine Learning Research (02/2025)
Here, C E R=... is the core tensor containing singular values calculated by taking the Frobenius norm along each mode i. The symbol x; denotes matrix multiplication along the 8th mode, and A(),... A are unitary matrices representing the newly calculated basis, akin to the matrices U and V in SVD.
The estimation of the values the core tensor C and the basis {4 can be achieved through different algorithms, notably, the Higher Order Orthogonal Iteration (HOOT) algorithm (Sheehan & Saad, 2007) or the Higher-Order SVD (HOSVD) (De Lathauwer et al., 2000) algorithm. In this work, we harness the HOOL algorithm, as it is known to be more accurate than HOSVD for Tucker Decomposition. HOOI uses HOSVD initialization step increasing the chances of having a unique solution which is then optimized in order to minimize the the Frobenius form between the tensor T and its decompasition (Kolda & Bader, 2009).
3
Related work
3.1 Feature attribution methods
Feature attribution methods are interpretability techniques which assign a weight to each feature of the input, such as a pixel in an image (Lundberg & Lee, 2017). One such feature attribution method is the Guided Backpropagation method (Springenberg et al., 2015), which propagates positive gradients of the classífication loss function the input. However, to address the gradient saturation and the stochastic nature of gradients, methods like Layer-wise Relevance Propagation (BaP) et al., 2015) and DeepLFTT (Shrikumar et al., 2017) were proposed. These methods calculate relevance score for the outpul of each neuron j as a weighted sum of all the neurons it is connected to. This operation is then back pack-propagated till the input, resulting in heatmap showcasing the importance of each region to the network's output (Montavon et al., 2019). However, Lundberg & Lee (2017) notices that the latter methods require domain-specific knowledge, which is dataset-specific, and therefore introduces Deep SHapley Additive ex exPlanations (Deep SHAP) (Lundberg & Lee, 2017). Deep SHAP uses Shapley values (Ichiishi, 1983) to calculate the contribution of each feature in the network's imput. Despite its strong thcoretical backing, Deep SHAP's computational complexity poses challenges for large datasets, making it less practical for real-time applications, even with approximation methods (Aas et al., 2021). Moreover, all the stated methods highlights certain pixels which does not produce spatially lumman-iteeppretbblle coherent maps. LIME (Ribeiro et al., 2016) solves the latter problem and highlights contiguous regions of the input. This is achieved by perturbing input data and create a locally interpretable model around the predicted classification, relying on the model's output class.
All feature attribution methods are inherently label-dependent, as they calculate saliency maps based on specified target class in different ways. The reliance of feature attribution methods on label information prevents their application to self-supervised models, where no labels are available. In contrast, our proposed methods, TSM and MTSM, operate in a label-independent manner and produce visually interpretable saliency maps by highlighting larger, contiguous regions of the input. Consequently, in our experiments we exclusively evaluate the performance of feature attribution methods on classification models.
3.2 Class activation map methods
Building on feature attribution techniques, Class Activation Maps (CAMs) (Zhou et al., 2016) offer a visual interpretation of model predictions, particularly in computer vision tasks. They have also found application in different domains, such anomaly detection (Kimura et al., 2020), disease identification and localization (Khan et al., 2019; Bouayed et al., 2022). CAM methods can be broadly categorized into label-dependent and label-independent methods. Below is a brief description of each family of CAMs.
3.2.1 Label-dependent class activation maps
Label-dependent class activation maps rely on the label information to generate saliency maps. The label information is combined with the output of the network to estimate a loss and backpropagate the loss information to infer the weight vector w. This information can be in the form of gradicnts as in GradCAM (Selvaraju et al., 2017) and XGradCAM (Fu et al., 2020), or the impact of each feature map on the classification of the input. image in the class specified label as in ScoreCAM (Wang et al., 2020) and
4

---

Published in Transactions on Machine Learning Research (02/2025)
AblationCAM (Desai & Ramaswamy, 2020). Other methods perform optimization procedure to minimize the model's 's training loss w.r.t. the specified label namely Information Bottleneck Attribution (Schulz et al., 2020) and Opti-CAM (Zhang et al., 2024).
These methods primarily target classification models, providing interpretations in the form of a saliency map for given reference label. Additionally, the most widely used label depend methods are gradient methods such as GradCAM (Selvaraju et al., 2017). However, since gradients indicate the direction of the steepest decent and the loss function is non-linear, the communicated information by the gradient changes rapidly. This can be seen noise which prevents the accurate estimation of the weight vector and the saliency map.
Recent work (Ramprasaath R. Selvaraju, 2021; Shu et al., 2023) use the gradient of the contrastive loss to calculate the GradCAM based saliency map of a self-superviser model. However, the accurate estimation of the contrastive loss heavily depends on the number of negative samples. Therefore, it is required to have a large batch size as in (Chen et al., 2020a) or a queue storing previous data points seen during the training as in (Chen et al., 2020b; Caron et al., 2020). Moreover, data augmentation techniques are essential to the generation of negative samples and some data augmentation transformations allow for more informative contrastive loss than others (See Figure 5 in (Chen et al., 2020a)). However, in the absence of large batch size (in inference for instance), loss estimation will degrade and the gradient of the loss would be even noisier. Consequently, using this gradient in a GradCAM-like method would result in poor-quality saliency maps.
In view of these constraints, using gradient based CAM methods for the interpretability of self-supervised CNN models would results in saliency maps which depend on the careful tunning of multiple factors. As a result, in our experiments on label-dependent methods we only consider classification models.
3.2.2 Label-independent class activation maps
Label-independent class activation maps rely on the decomposition of the feature map tensor to infer the weight vector W. To the best of our knowledge, EigenCAM (Muhammad & Yeasin, 2020) is the only method in this family. EigenCAM flattenes the feature map tensor into a matrix, and after applying the SVD (Trefethen & Bau, 1997), the first right singular vector is used as the weight vector. Despite EigenCAM's good performance, the matricization step disrupts spatial relationships between dimensions, captures limited amount of variance in the first singular vector and fails to utilize the full potential of tensor algebra, leading to a less accurate weight vector and lower quality saliency map. To address these shortcomings, in the following Section 4 we propose the TSM method, which leverages Tucker tensor decomposition for more accurate weight vector estimation and improved saliency maps. Moreover, since both EigenCAM and TSM rely only on one singular vector, in Scction we propose an extension of both methods to harness all the singular vectors and singular values.
4 Tucker Saliency Maps
The Tucker Saliency Map (TSM) method is a label-independent local interpretability approach. It involves generating saliency maps in the form of heat maps highlighting significant regions in the input images of CNN related to the task at hand. To accomplish this, firstly the feature map tensor outputted by convolutional layer is retrieved. Then, weighted sum of this tensor is performed along the channels mode with the weight vector weighting each feature map estimated through the Tucker tensor decomposition. Consequently, it can be noticed that TSM is applicable to both supervised and self-supervised CNN models as it does not require information regarding the loss but only requires access to the feature map tensor. Figure provides an overview of the proposed method, visually summarizing its key components and process.
In detail, given a feature map tensor J R.C... retreived as the output of convolutional layer, we firstly perform its Tucker tensor decomposition
F C-CxAA" X2 ( X3 A())
With C being the core tensor, A0) E RC.CC. A "H and A w.W are the matrices containing the orthonormal singular vectors for each dimension. The choice of Tucker tensor decomposition among all tensor
5

---

Published in Transactions on Machine Learning Research (02/2025)
A
A(3)
CNN
Output
F
(3
Feature map tensor
Tucker decompusition
norm 2i
E
TSM
MISM
Figure 1: Proposed TSM and MTSM methods. Firstly, the feature map tensor F extracted from the output of convolutional layer (prefcrably the last convolutional layer, it has the widest field of view). Then, the Tucker tensor decomposition (Sheehan & Saad, 2007) is performed on F. This decomposition results in a core tensor C and three matrices containing orthonormal singular vectors for each dimension, notably A A(2), and A The matrix A and the singular values are appropriately used to calculate the TSM (See Equation 4) and the MTSM (See Equation 6). Finally, the absolute value function is applied per element to the resulting matrix and the values are renormalized to the range [0; 1].
decomposition methods is strategic, as it accurately identifies the direction of highest variance, a feature crucial for weighting feature maps effectively (Kolda & Bader, 2009).
Secondly, inspired by the work of Muhammad & Yeasin (2020), we consider the first singular vector i.c. the one associated the largest singular value, A( of the matrix ( as the weight vector w E RCC This can be seen as projecting the feature map tensor on the direction with the highest variance. As result, the obtained saliency maps highlighting more of the regions important the model's output hence more accurate local interpretability. Formally, calculate the saliency map follows
C, C,A()), A(3) TTucker(F)
w A
(4)
TSM norm
Wi F,
The application of min-max normalization norm(.) ensures that the saliency map's values are bounded within the [] range. The absolute value function | is employed instead of ReLU to focus on the activation magnitude, disregarding the sign, considering the possibility negative value multiplication in subsequent layers of the feature map F.
The main difference between TSM and EigenCAM resides in the decomposition ion methods. EigenCAM performs SVD decomposition after matricization operation of the feature map tensor mat(-)
U, VT VT = -VV((ma((F))
Vi.
(5)
The matricization operation F ultimately mixes the elements of the first and second mode (height and width) so to obtain matrix of size x. Such operation would have negative impact the relationships between the elements of the tensor. Therefore, performing dccomposition on such matrix results in less accurate estimation of the singular vectors and values for each mode which leads to less representative saliency maps. However, in TSM, we adopt a more general and more suitable neighborhood information preserving dccomposition notably the Tucker tensor decomposition. This decomposition allows for better approximation of the first singular vector capturing the most variance. This can be observed in Figure 2 where the amount of variance encoded by the first singular vectors using the Tucker decomposition

---

Published in Transactions on Machine Learning Research (02/2025)
0.45
decompodttion SVD decompostion
0.30
0.25
6.10
ViRegl. Resnet
Vistegl Convieut
Barlowaaiind
M2GO
SWAI
Figure 2: Boxplot comparing the distribution of the first singular values divided by the sum of all singular values per tensor in both SVD decomposition and Tucker decomposition. These singular values are obtained from the decomposition of all the feature map tensors produced by five different self-supervised models on the Pascal VOC (Everingham et al., 2010) validation dataset.
is higher compared to the SVD decomposition most tested models. Additionally, the use of the absolute value in the calculation of the saliency map puts forward more information hence more informative and accurate saliency maps. Nevertheless, both EigenCAM and TSM utilize only the singular vector associated with the largest singular value. In the following section, we propose a generalization of both methods to utilize all singular values and singular vectors.
5 Multivector EigenCAM and Multivector Tucker Saliency Maps
Figure 3 represents the distribution the first five singular values in both SVD decomposition and Tucker decomposition divided by the sum of all singular values per tensor. From this figure we notice that for both the SVD and Tucker decompositions singular values other than the first one encapsulate a significant amount of variance. Consequently, FigenCAM and TSM only convey a portion of the total information encoded in the feature map tensor. To this end, in this section we introduce Multivector EigenCAM and Multivector Tucker Saliency Maps (Mullivee- Eigen CAM and MTSM) interprelability methods. These methods generate a saliency map per singular vector in the SVD or the Tucker tensor decomposition. Then, a weighted sum of the generated saliency maps is performed. The weights of each saliency map are inferred based on the singular values such as, given a, the singular values associated with the i-th singular vector (extracted either from the SVD or the Tucker decompasition), the weight are given by d where on is the largest singular value. di We provide an overview of the proposed MTSM method in Figure 1.
Formally, given feature map tensor F E R.HHW. operator D(-) encapsulating the SVD or Tucker tensor decomposition and returning the ordered singular values R² and their corresponding singular vectors for the channels dimension V EERR R
(6)
Mutivec-* * norm
Σ VasFi
This formulation of the Mutivec-* can be seen as first generating the CigenCAM or TSM, since the weight of the first saliency map is one, than adding additional information to it which is generated using the rest of the singular vectors. Consequently, the produced saliency maps are richer and more precise as demonstrated quantitatively and qualitatively in Section
The differentiating point between uni-vector methods to its multi-vector ones in performance would depend the amount of variance encoded by the singular vectors other than the first one. If these vectors do not.
7

---

Published in Transactions on Machine Learning Research (02/2025)
Tycker docompposoon SVD decomoozttin
Tacker doco pcompssoon SVD decomposition
0.25
0.70
6.2
0.30
+|+
0.05
Turrer
sxp dp
Turker di
Tester di
500 dy
Tucher di
Tutter de
d
Tucrerd do
SVB
Tecter dt
SVD d:
di
SVD dz
Turter di
Tocker
SVD
VicRegL ConvNext (Bardes al., 2022)
(b) Moco V2 (Chen et al. 2020b
Figure 3: Boxplot representing the distribution of the first five singular values in both SVD dccomposition and Tucker decomposition divided by the sum of all singular values per lensor. These singular values are obtained from the decomposition of all the feature map tensors produced by the Moco V2 (Chen et al., 2020b) and VicRegL ConvNext (Bardes et al., 2022) models on the Pascal VOC validation dataset.
encode large amount of variance the performance does not improve significantly (See the Tucker tensor decomposition in Figure 3(a) and the quantitative results in Table 2). However, if they encode large amount of information, harnessing these vectors would result in significant performance improvement (See Figure 3(b)). Consequently, multi-vector saliency map methods in the worst scenario output the same result as the uni-vector hods. Moreover, as SVD based saliency methods do not estimate the most accurate singular vectors, we notice a significant amount of variance encoded by the rest of the singular vectors in Figure Therefore, EigenCAM's performance would greatly improve when opting for the multi-vector formulation. We quantilatively and qualitatively confirm these remarks in the following Section 6.
6 Experiments
In this section we start by presenting the metrics use to evaluate interpretability methods based on saliency maps both supervised classification models and self-supervised representation learning models. Then, we present, discuss and compare the experimental results of our proposed methods compared to methods in the literature. Moreover, in Appendix A we present Python code for the implementation of our proposed methods and in Appendix B we lay out our experimental setup. Appendices F., and F put forward additional qualitative results for the different tested supervised and self-supervised CNN models.
6.1 Evaluation metrics
6.1.1
Supervised learning metrics
To evaluate our proposed methods on supervised models, we opt for the two conventionally used metrics, the Average Drop and Average Increase proposed in the work of Chattopadhay et al. (2018) as they align with human interpretation and favor saliency maps with contiguous regions.
Average Drop (AD) This metrics quantifics the answer to the question 'Did we hide in the image information which is relevant to the target class ?*. To do so, the AD calculates the mean drop in classification confidence for the target class via the following formula:
off 100.
AD(%) N
1
Average Increase (AI)This metric measures if the saliency map has removed unnecessary informa- tion from the input image, hence removing noise and improving the model's confidence in the target.
8

---

Published in Transactions on Machine Learning Research (02/2025)
class. This is done via the following formula
AI(%)
Ipgcof 100.
(8)
In the previous formulae, we denote by N the size of the dataset and by e the ground truth label of the input pf (respectively, of) represents the probability of classifying the image (respectively, the image masked by the inferred saliency map) in class We note that the AD and AT metrics are to be minimized and maximized respectively.
6.1.2 Self-supervised learning metrics
Since, to the best of our knowledge, no framework exists for the evaluation of saliency map based interpretability methods on self-supervieed CNN models, we propose one. The proposed framework relies on four metrics; notably the Average Drop (AD), the Average Increase (AI) (Chatlopadhay et. al., 2018), the Mean Squared Error (MSE), and the mean Intersection over Union (mloU) metric (Jaccard, 1901).
Average Drop (AD) and Average Increase (AI) (Chattopadhay et al., 2018) To calculate the AD and AI, we estimate the saliency map from the sclf-supervised model, then evaluate the quality of the salieney map using pretrained supervised classification model and the formulae described in the previous Section 6.1.1. We detail the pretrained classification models used for each self-supervised model in Appendix B.4.
Mean Squared Error Since self-supervised models output an encoding for each input image, we calculate the mcan squared error between the encoding 2, of the image and the encoding & of the image masked with the inferred saliency map. If the saliency map able to isolate the important regions in the image which encoded, there should be difference between the encodings. This metric is to be minimized and we formulate it follows
N MSF
1 N
Σ
-1a--11 E-13-
Mean Intersection over Union (Jaccard, 1901) Under the hypothesis that the saliency map represents a segmentation highlighting the most important parts in an image, we calculate the mean Intersection over Union segmentation metric using the ground truth segmented images in the Pascal VOC dataset (Everingham et al., 2010). Formally, we calculate this metric using the following formula
[1 if saliency map T 0 clse
BBS mloU(B, = SU
B-
B
(10)
where B is the pixel-wise binarized saliency map according to threshold T, S is the ground truth binary sogmentation mask and |B n S] (respectively, [B U SS) represents the area of intersection (re- spectively, union) between the saliency map and the ground truth segmentation S. Moreover, to be thorough, we study the impact of the performance of the different proposed interpretability methods as a function of the threshold parameter T on a multitude of models. The obtained results are reported in Appendix C. We note that this metrics to be maximizod.
6.2 Results and discussion on supervised classification models
We evaluate and compare the TSM, MTSM and Mullivee-EigenCAM method to a multitude of saliency map extraction methods on the pretrained VGG16 (Simonyan & Zisserman, 2014), Resnet50 (He et al., 2016) and ConvNext (Liu et al., 2022) models. For the evaluation, we calculate the AD and AI on the 50, 000 validation images of the ImageNet dataset. The obtained quantitative and qualitative results are presented in Table and Figure 4 respectively.

---

Published in Transactions on Machine Learning Research (02/2025)
Table 1: Saliency map methods comparison on the validation set of the ImageNet dataset across different pretrained classification models. The best values are highlighted in bold, while the second-best values are underlined, Cyan-colored rows denote label-independent saliency map methods.
Resnet50
ConvNext
VGG16
Method
AD
(T)%
AD(%) +
AT(%) f
AD(%)
AT(%) 1
GradCAM
13.01
44.52
32.49
14.67
17.58
10.32
+
13.61
42.50
80.32
9.72
19.12
35.20
HiResCAM
13.01
44.52
32.49
16.66
19.18
37.59
XGradCAM
13.01
44.52
32.49
16.66
16.00
41.64
AblationCAM
13.21
43.05
51.74
10.99
17.66
38.72
OptiCAM
26.45
27.77
19.60
26.77
15.59
40.48
DeepLIFT
95.37
2.50
93.44
1.75
95.34
2.47
DeepSHAP
95.36
2.49
93.43
1.75
95.33
2.47
LIME
57.73
12.71
56.23
10.92
62.33
11.47
EigenCAM TSM
38.51
21.41
36.24
9.69
47.32
17.11
16.86
37.27
30.85
9.36
26.74
27.50
Multivec- EigenCAM
22.69
30.43
37.93
9.92
25.64
27.50
MTSM
16.27
37.24
31.78
9.16
22.02
30.65
From Table 1, we notice that CAM methods' performance superior to that of feature attribution methods as they highlight larger contiguous regions important the classification task. Moreover, when comparing label-dependent CAM methods to our proposed label-independent methods, we notice that the latter methods achieve competitive performance especially, the ConvNext model where note TSM's second best result the AD metrie. However, label-independen methods' performance is still lower than state-of-the-art label- dependent methods on classification models as the latter utilize more information. A fairer comparison of our proposed methods is with label-independent methods notably EigenCAM. In this setting, TSM demonstrates approximately 50% improvement in performance across most models, measured by the AD and Al metrics. For example, on the VGG16 model, TSM achieves 26.96% Average Drop compared to the 47.32% obtained by the EigenCAM method, representing a 56.97% improvement. Additionally, Multivec-EigenCAM and MTSM improve upon the performance of EigenCAM and TSM respectively. The magnitude of the difference in performance between uni-vector and multi-vector methods is governed by the amount of variance encoded in the singular vectors following the first one as discussed in Section 5. We pereeive a larger performance jump going from EigenCAM to Multivee-Eigen compared to going from TSM to MTSM. This goes back to the more accurate estimation of the singular vectors by Tucker tensor decomposition and capturing of most the variance the first singular vector. Therefore, even though MTSM results in a smaller performance improvement gap, it significantly improves upon the performance of Multivec- Jultivee-EigenCAM. EigenCAM. We further emphasize the importance of using the Tucker tensor decomposition instead of the SVD that two out of the three tested models the uni-vector TSM achieves significantly better performance than Sallivee-EigenAAMM.
notice
Lobel depeudde mothods
Label Independent
Input
GovRCAM
AllationAM
OptiCAM
LIMR
FiyeCAM
TSM
AITSM
Figure 4: Qualitative comparison of saliency map-based methods on the Resnet50 model.
10

---

Published in Transactions on Machine Learning Research (02/2025)
Figure 4 presents qualitative comparison between sample of label-dependent and independent meth- ods (Additional figures are presented in Appendix E). This figure backs our quantitative results where we notice that TSM and MTSM yield saliency maps similar to the most performant label-dependent method notably GradCAM. However, the highlighted regions by TSM and MTSM can be relevant to the image class or not, showcasing the label-independent nature of the methods. This particularly noticeable in the second image, where TSM and MTSM effectively highlights the bridge and the river instead of just the bridge as the label-dependent methods. Furthermore, compared to EigenCAM and Multive--EigenCAM TSM and MTSM highlight more of the regions in the image and with higher confidence (Notice the mosque and dog images in Figure 4). Finally, comparing uni-vector and multi-vector, we perceive significant improvement of Multiveo-EigenAAM over EigenCAM with more highlighted regions. Nevertheless, between TSM and MTSM little difference can be observed which joins our observation and discussion of the quanlitative results.
6.3 Results and discussion self sule-vupe models
In this section, present the quantitative and qualitative results obtained by the methods propose. We analyse, discuss and compare them only to the results obtained by the EigenCAM method (Muhammad & Ycasin, 2020) as it is the only principled method which can be applied in this context In this section, we calculate the AD, AT and MSE metrics on the 50, 000 validation images of the ImageNet dat (Deng et al., 2009) whereas the mloU metric calculated on the 449 validation images of the Pascal VOC dataset (Everingham et al., 2010). The obtained quantitative and qualitative results are presented Table and Figures 5, and
Table 2: Comparison between TSM, EigenCAM, MTSM and Multivee-FigenCAAN on self-supervised models trained on the ImageNet dataset. The reported mloU values are computed with a threshold of 0.5 applied to the saliency maps. The best values are highlighted in bold.
Method
EigenCAM MSEx 10
TSM
AD(%) 4
A[(%)
J mloU(9) f
AD(%) J
AI(%)
10-EX10-2
mloU(%) f
Moco V2
38.42
32.56
0.25
23.13
20.98
44.60
0.16
28.82
SWAV
31.94
37.49
12.22
26.35
14.61
47.23
5.55
39.97
Barlow Twins
41.72
31.02
8.73
21.06
21.41
44.69
5.95
27.91
VidRegL Resnet50
42.15
30.80
75.50
21.28
20.30
45.36
48.20
28.47
VieRegl. ConxNext
40.71
9.93
95.54
22.65
17.42
13.26
25.57
36.69
Method
Multiver- CiggenCAM
MTSM MSEx 10-3
AD(%) 4
A1(%) f MSEx10-3
mloU(%) 1
AD(%) 4
AI(%)
mloU(%) 1
Moco V2
19.24
45.49
0.15
27.75
16.38
46.96
0.12
33.57
SWAV
16.40
44.81
6,00
37.54
13.89
46.59
4.63
41.33
Barlow Twins
18.96
43.84
5.26
32.35
16.58
46.42
4.18
36.15
VicRegL Resnet50
18.10
45.59
42.37
33.19
16.30
46.67
35.98
35.38
VieRegL ConvNext
17.10
13.85
31.49
26.61
17.87
13.51
26.84
37.00
From Table 2, we observe the superior performance of Tucker decomposition based saliency map methods (i.e.
TSM and MTSM) compared to the SVD based saliency map methods (i.c. EigenCAM and Multivec- EigenCAMM. Additionally, similar to our observation on the supervised classification model, TSM acheives a 50% improvement in performance on all metrics compared to EigenCAM. This is attributed to the benefits of using the most suitable mathematical tools to manipulate tensors notably the Tucker tensor decomposition. Also, the use the absolute value in the TSM and MTSM methods brings forward negative contributions which are interest. Consequently, through the used metrics we can conclude that compared to FigenCAM, TSM is able to
Retain more of crucial information in the image which is relevant to image classification.
Produce saliency maps aligned with segmentation masks.
Leebel-deneddent methods like GradCAM, which we in the previous section, difficult adapt this setting principled manner and can represent an interesting future work (See discussion Section 3.2.2).
11

---

Published in Transactions on Machine Learning Research (02/2025)
Conccal notably less imput regions vital to the self-supervised model.
Additionally, notice that the multi-vector methods improve the performance of TSM. Also, Multivec- EigenCAM has a significant performance jump compared to EigenCAM opposed to the performance gap between TSM and MTSM. This can also be explained by our previous argument on the supervised classification modols which links back to the amount of variance encoded by the first singular voctors. However, for the VicRegL ConvNext model we notice a slight drop in performance going from TSM to MTSM. The reason for this is illustrated in Figure 3(a). We can see that a large portion the variance captured by the first singular vector around 30% 40%. Consequently, the rest of the singular values do not bring significant information. Combined with the weight assigned to these vectors, their impact on the final saliency map is minimal at best adds noise to it.
Input image
EigenCAM
TSM
Multive EigenCAM
MISM
Figure 5: Qualitative comparison of saliency maps extracted using the EigenCAM versus TSM on the SWAV model applied to the ImageNet dataset.
Iuput inage
Musked image
EngenCAM
TSM
Multivee EigenCAM
MISM
Figure 6: Qualitative comparison of saliency maps extracted using the EigenCAM versus TSM on the SWAV model, juxtaposed with the segmentation mask from the Pascal VOC dataset. The showcased segmentations based on the saliency maps employ a threshold of 0.5.
Figures and 6 qualitatively confirm the quantitative results of Table 2. These figures illustrate superior coverage of important objects in the saliency maps generated by TSM compared to those produced by EigenCAM. For instance, we notice TSM providing better coverage of distinctive features in the dog and the mosque images. Lastly, the visual reprosentations in Figure 6 underscore the qualitative findings reflected in the mloU metric of Table Notably, EigenCAM (respectively Multivec-EigenCAAM highlights fewer regions of interest than TSM (respectively MTSM), leading to a lower mloU. Moreover, in Figure 5, EigenCAM presents a substancial difference the resulting saliency map compared latter produces saliency maps inline with those of TSM and MTSM. Consequently, EigenCAM is not able capture all the necessary information and presents a very limited view of what. the self-supervised model is
Eigeiiee whereas the
12

---

Published in Transactions on Machine Learning Research (02/2025)
focusing Therefore, when using EigenCAM interpretability method the conclusion that one could arrive to might not be the correct one.
Lastly, we compare the saliency maps produced through supervised classification models in Figure 4 and the ones produced through self-supervised models in Figure 5. We notice that all the label-indeenndent saliency map methods are not class specific but the learning task has an impact on the produced map. This is particularly evident the first and third images.
Table 3: Execution time comparison between label- inbel-independent methods in seconds to produce a single saliency map for a single input image. In orange we outline the difference between the SVD methods and Tucker decomposition methods.
Model
EigenCAM
TSM
Mullivee-EigennANN
MTSM
Resnet50
1.64
3.68 (2.2x)
1.90
4.05 (2.1x)
ConvNext
1.19
1.65 (1.4)
1.14
1.73 (15x)
VGGI6
1.10
1.21 (1.1x)
1.12
1.22 (1.1x)
Moco V2
1.13
2.64 (2.3x)
1.12
2.85 (2.5x)
SWAV
1.15
2.35 (2.0x)
1.15
2.56 (2.2x)
Barlow Twins
1.17
3.32 (2.8x)
1.17
3.62 (3.1x)
VicRegT. Resnet50
1.22
2.48
(2.1x)
1.19
2.53 (2.1x)
VicRegL ConvNext
0.99
1.23
(1.2x)
0.99
1.26 (1.3))
From Table 3 we notice that Tucker decomposition methods even though they output high quality saliency
maps they come with an increased computational cost compared to SVD based methods. The former methods' mean processing time is approximately two times slower. However, going from uni-vector methods to multivector methods come with a negligible runtime increase.
7 Conclusion
In this work, we introduce the TSM method, a label-independent CNN interpretability method harnessing tensor algebra notably Tucker tensor decomposition to produce high quality and fidelity saliency maps of CNN's input regions. TSM is principled method, not restricted supervised classification CNN models and yet achieves competitive quantitative and qualitative results compared to label-dependent methods. Moreover, compared to the only CNN label-independent method namely EigenCAM, TSM allows for more accurate estimation of singular vectors and values resulting in more accurate saliency maps. Furthermore, we extend the EigenCAM and TSM method to the Mettivee-LEignCCAM and MTSM which take profit of all the singular vectors and values further improving the interpretability of supervised and self-supervised CNN models. We Lest all the label-independent methods on various supervised classification CNN models via established metrics and on self-supervised CNN models through an evaluation procedure we propose. When comparing CAgenCAM to TSM, results indicate 50% improvement in quantitative results on all tested metrics and models. base methods with varying extents depending on the amount of variance encoded by each singular vector. Our qualitative results showcase that EigenCAM can be misleading when used as an interpretability method. This is because highlights fewer regions of the imput in the saliency map compared to our proposed methods which are more reliable since they highlight all the significant regions.
and MTSM further improve the performance of their
It worth noting that TSM and MTSM come with increased computational costs, requiring approxi- mately 2x more processing time than EigenCAM and Multivec-EigenCCin respectively due to the Tucker tensor decomposition. Despite this complexity, the significant improvements in interpretability and the meth- ods' offectiveness in interpreting self-supervised models justify the additional computational burden. Future work will focus on optimization techniques to reduce execution time, ensuring that these methods remain efficient. Moreover, in this work we have tailored the TSM and MTSM methods to particularly interpret CNN models. However, light of their statc-of-tho-art performance for self-supervised interpretability on CNN models, an interesting perspective work is explore innovative ways to utilize tensor algebra to better interpret other neural network architectures such transformers.
13

---

Published in Transactions on Machine Learning Research (02/2025)
Acknowledgements
The authors extend their gratitude to Stéphane Ayache and Rachid Deriche for their invaluable input and constructive leedback on the content of this paper. This work received access the High-Performance Computing (HPC) resources of MesoPSL, financed by the Region Ile-de-France and the Equip@Meso project (reference ANR-10-EQPX-29-00 of the Investissmmnns d' d'vvenir program supervised by the Agence nationale pour la recherche.
References
Kjersti Aas, Martin Jullum, and Anders Løland. Explaining individual predictions when features are dependent: More accurate approximations to shapley values. Artificial Intelligence, 298:103302, 03 2021.
doi: 201 103502..
Scbastian Bach, Alexander Binder, Grégoire Montavon, Frederick Klauschen, Klaus-Robert Müller, and Wojeiech Samek. On pixel-wise explanations for non-linear classifier decisions by layer-wise relevance propagation. PLOS ONE, 10(7):1-46, 07 2015. doi: 00.187//ourmalLLooe.001000 01300440 URL https: /doi. org/10. org/10.1371/ jounaa pone. 0130140.
Adrien Bardes, Jean Ponce, and Yann LeCun. VICRegL: Self-supervised learning of local visual features. In S. Koyejo, S. Mohamed, A. Agarwal, D. Belgrave, K. Cho, and A. Oh (eds.), Advances in Newral Informalion Processing Systems, volume 35, pp. 8799 8810. Curran Associales, Inc., 2022.
Aymene Mohammed Bouayed, Samuel Sastauriers-Gaathie Mauro Zucchelli, and Rachid Deriche. CNN and diffusion MRT's 4th degree rotational invariants for Alzheimer's disease identification. In Nuno Moniz, Paula Branco, Luís Torgo, Nathalic Japkowicz, Michal Wozniak, and Shuo Wang (eds.), Proceedings of the Fourth International Workshop on Learning with Imbalanced Domains: Theory and Applications, volume 183 of Proceedings of Machine Learning Research, pp. 163- 174- PMLR, 23 Sep 2022.
Fadi Boutros, Marco Huber, Patrick Siebke, Tim Rieber, and Naser Damer. SFace: Privacy-friendly and accurate face recognition using synthetic data. In IEEF International Joint Conference on Biometrics, IJCB 2022, Abu Dhabi, United Arab Emirates, October 10-13, 2022, pp. 11. IEEE, 2022. doi: 10.1109/ IJCB54206. 001B544006.022.1111 2022.10007961.. URL https: //tp://ddiiioaa org./..ooo/ooooooo www.oogg////./... 1tw.11119IIIB///0 2022. 10007961.
Ginseppe Bove. Methods for the analysis of skew-symmetry asymmetric multidimensional scaling. In Hermann Locarek-Junge and Claus Weihs (eds.), Classifiention as a Tool for Research, pp. 271 1288, Berlin, Heidelberg, 2010. Springer Berlin Heidelberg. ISBN 978-3-642-10745-0,
Mathilde Caron, Ishan Misra, Julien Mairal, Priya Goyal, Piotr Bojanowski, and Armand Joulin. Unsupervised lcarning of visual fcatures by contrasting cluster assignments. In H. Larochelle, M. Ranzato, R. Hadsell, M.F. Balean, and H. Lin (eds.), Advances in Neural Information Processing Systems, volume 33, pp- 9912 9912-9924. Curran Associates, Inc., 2020.
Aditya Chattopadhay, Anirban Sarkar, Prantik Howlader, and Vineeth N Balasubramanian. Grad-CAM++: Generalized gradient-based visual explanations for deep convolutional networks. In 2018 IEFE Winter Conference on Applications of Computer Vision (WACV), pp. 839, 2018. 00097.
doi: 10.1109/ 10.1109/WACV WACV.2015
Ting Chen, Simon Kornblith, Mohammad Norousi, and Geoffrey Hinton. A simple framework for contrastive learning of visual representations. In Hal Daumé III and Aarti Singh (eds.). Proceedings of the 37th International Conference on Machine Learning, volume 119 of Proceedings of Machine Learning Research, pp. 1597 1607. PMLR, 13 18 Jul 2020a. URL
chen20j mlr, www.ppreessvvieee htts///1////cc/nn html.
Xinlei Chen, Haoqi Fan, Ross Girshick, and Kaiming He. Improved baselines with momentum contrastive learning. arXiv preprint arxiv:2003. 0at.200030,297, 2020b.
Lei Cheng, Zhongtao Chen, and Yik-Chung Wu. Tensor Decomposition: Basics, Algorithms, and Recent Advances, pp. 1-14. Springer International Publishing, 2023. doi: 10.1007/997--3-03--2228---- 1.
14

---

Published in Transactions on Machine Learning Research (02/2025)
Federica Cruciani, Lorenza Brusini, Mauro Zucchelli, Gustavo Retuci Pinheiro, Francesco Setti, Ilaria Boscolo Galazzo, Rachid Deriche, Leticia Rittner, Massimiliano Calabrese, and Gloria Menegaz. Explainable 3D-CNN for Multiple Scierosis patients stratification. In ICPR 2020 25th International Conference on Pattern Recognition Workshops and Challenges, pp. 103-114, 2021. doi: 070000077777830--677777 URL https://hal. www.sncceeeeee00/99/7771
8.
Lieven De Lathauwer, Bart De Moor, and Joos Vandewalle. A multilinear singular value decomposition. SIAM Journal on Matrir Analysis and Applications, 21 (4):12233-127))) 2000. doi:
Jia Deng, Wei Dong, Richard Socher, Li-Jia Li, Kai Li, and Li Fei-Fei. ImageNet: A large-scale hierarchical image database. In 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248 255, 2009. doi: 10. 10.1109/CCT CCVPR.2009.506666
Saurabh Desai and Harish G. Ramaswany. Ablation-CAM: Visual explanations for deep convolutional network via gradient- free localization. In 2020 IEEE Winter Conference on Applications of Computer Vision (WACV), pp. 972-980, 2020. doi: 10. WWACV45572.20000000
M. Everingham, L. Van Gool, C. K. I. Williams, J. Winn, and A. Zisserman. The pascal visual object classes (VOC) challenge. International Journal of Computer Vision, 88(2):303 338, June 2010.
Ruigang Fu, Qingyong Hu, Xiaohu Dong, Yulan Guo, Yinghui Gao, and Biao Li. Axiom-based Grad-CAM: Towards accurale visualization and explanation of CNNs. Brilish Machine Vision Conference, 2020.
Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 770-778, 2016.
T. Tchiishi. Game Theory for Economic Analysis. Economic Theory, Econometrics, and Mathematical Economics. Elsevier Science, 1983. ISBN 9780123701800. URL https://book:: google fr/books?id= ZFm7AAAATAAJ.
Paul Jaccard. Etude de la distribution florale dans une portion des Alpes du Jura. Bulletin de la Sociele Vaudoise des Sciences Naturelles, 37:547-579, 01 1901. doi: 10.5169 151699/eeals26665 266450.
Jacob Gildenblat et al. Pytorch library for CAM methods. gracch-grr-camm. 2021.
http:://github..comjaaaccc
Naimul Mefraz Khan, Nabila Abraham, and Marcia Hon. Transfer learning with intelligent training data selection for prediction of Alzheimer's discase. IEEE Access, 7:72726-72735, 2019. doi: 10.1109/ACCESS. 2019.20 292048..
Henk A. L. Kiers. Towards standardized notation and terminology in multiway analysis. Journal of Chemometrics, 14(3):105-122 122, 2000.
Eunji Kim and Sungzoon Cho. Exposing fake faces through deep neural networks combining content and trace feature extractors. IEEE Access, 9:123493-123503, 2021. doi: 10.1109/ACCESS 2021.3110899.
Daiki Kimura, Subhajit Chaudhury, Minori Narita, Asim Munawar, and Ryuki Tachibana. Adversarial discriminative attention for robust anomaly detection. In 2020 IEEE Winter Conference on Applications of Computer Vision (WACV), pp. 2161-2170, 2020. doi: 10.1109 10.1109/WACTT /WW/////C222222200000000
Alexander Kirillov, Eric Mintun, Nikhila Ravi, Hanzi Mao, Chloe Rolland, Laura Gustafson, Tete Xiao, Spencer Whitehead, Alexander C. Berg, Wan-Yen Lo, Piotr Dollar, and Ross Girshick. Segment anything. In Proceedings of the IEEE/CVF International Conference Computer Vision (ICCV), pp. 4015-4020 October 2023.
Narine Kokhlikyan, Vivek Miglani, Miguel Martin, Edward Wang, Bilal Alsallakh, Jonathan Reynolds, Alexander Melnikov, Natalia Kliushkina, Carlos Araya, Siqi Yan, and Orion Richardson. Captum: A unified and generic model interpretability library for pytorch, 2020.
15

---

Published in Transactions on Machine Learning Research (02/2025)
Tamara G. Kolda and Brett W. Bader. Tensor decompositions and applications. SIAM Review, 51(3):455-50 2009. doi: 10.11370707011113
Alex Krizhevsky, Iyya, Sutskever, and Geoffrey E. Hinton. ImageNet classification with deep convolutional neural networks. Commun. ACM, 60(6):84-90, May 2017. ISSN 0001-0782. doi: 10.1145/3063366. URL
https://doi. http///oi.orr/////// ot./ 1115301.1155/3363666
Ze Liu, Yutong Lin, Yue Cao, Han Hu, Yixuan Wei, Zheng Zhang, Stephen Lin, and Baining Guo. Swin transformer: Hierarchical vision transformer using shifted windows. In Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), pp. 10012-10022, October 2021.
Zhuang Liu, Hanzi Mao, Chao- Yuan Wu, Christoph Feichtenhofer, Trevor Darrell, and Saining Xie. A ConvNct for the 2020s. In Proceedings of the IEEE/CVF Conference Computer Vision and Pattern Recognition (CVPR), pp. 11976 11986, 2022.
Scott M. Lundberg and Su-In Lee. A unified approach to interpreting model predictions. In I. Guyon, U. Von Luxburg, S. Bengio, H. Wallach, R. Fergus, Vishwanathan, and R. Garnett (eds.), Proceedings of the 31st International Conference on Advances Neural Information Processing Systems, NeurIPS'17 17, pp. 4768 4777, Red Hook, NY, USA, 2017. Curran Associates Inc. ISBN 9781510860964.
Grégoire Montavon, Alexander Binder, Sebastian Lapuschkin, Wojeiech Samek, and Klaus Robert Müller. Layer-wiss relevance propagation: An overview. Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), pp. 193-209, 09 2019. doi: 10.1007/978-3-030 28954-6 10.
Mohammed Bany Muhammad and Mohammed Yeasin. Eigen-CAM: Class activation map using principal components. In 2020 Internalional Joinl Conference on Newral Nelworks (IJCNN), pp. 1-7, 2020. doi:
Justin Johnson Nikhil Naik Ramprasaath R. Selvaraju, Karan Desai. Casting your model: Learning to localize improves self-supervised reprosentations. In CVPR, 2021.
Joseph Redmon, Santosh Divvala, Ross Girshick, and Ali Farhadi. You Only Look Once: Unified, real-time object detection. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 779-788, June 2016.
Marco Tulio Ribeiro, Sameer Singh, and Carlos Guestrin, "why should i trust you?": Explaining the predictions of any classifier. In Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, KDD '16, pp. 1135-1144, New York, NY, USA, 2016. Association for Computing Machinery. ISBN 9781450342322. doi: 10. 10.1145/2833672228877777 URL https://doi.....
10. 1t.1145/2939977 2939778.
Karl Schulz, Leon Sixt, Federico Tombari, and Tim Landgraf. Restricting the flow: Information bottlenecks for attribution. In International Conference on Learning Representations, 2020. URL https: https://openrevviet
Ramprasaath R. Selvaraju, Michael Cogswell, Abhishek Das, Ramakrishna Vedantam, Devi Parikh, and Dhruv Batra. Grad-CAM: Visual explanations from deep networks via gradient-based localization. Proceedings of the IEEE International Conference on Computer Vision (ICCV), pp. 618-626, Oct 2017.
Bernard Sheehan and Yousef Saad. Higher Order Orthogonal Iteration of Tensors (HOOI) and its relation PCA and GLRAM. In Proceedings of the Seventh SIAM International Conference on Data Mining, pp. 355-365, 2007.
doi: 10.1137/1.9781611977777
Avanti Shrikumar, Peyton Greenside, and Anshul Kundaje. Learning important features through propagating activation differences. In Doina Procup and Yee Whye Tch (ods.), Proceedings of the 34th International Conference on Machine Learning, volume 70 of Proceedings of Machine Learning Research, pp. 3145 3153. PMLR, 06-11 11 Aug 2017.
16

---

Published in Transactions on Machine Learning Research (02/2025)
Yangyang Shu, Anton van den Hengel, and Lingqiao Liu. Learning common rationale to improve self- supervised representation for fine-grained visual recognition problems. In Proceedings of the TEEE/CV Conference on Computer Vision and Pottern Recognition, pp. 11392 11401, 2023.
Karen Simonyan and Andrew Zisserman. Very deep convolutional networks for large-scale image recognition.
arXiv preprint (xXtc:1409.1550 2014.
Jost Tobias Springenberg, Alexey Dosovitakiy, Thomas Brox, and Martin Riedmiller. Striving for simplicity:
The all convolutional net. arXiv preprint 1rXiv:1412.6800 2015.
L.N. Trefethen and D. Bau. Numerical Lineor Algebra. Other Titles in Applied Mathematics. Society for Industrial and Applied Mathemalics, 1997. ISBN 9780898713619. URL https://boole https///ooooooooggl..eeeeo go/ boots?iddoj-LuuzjWbBB.
Jason Walsh, Alice Othmani, Mayank Jain, and Soumyabrata Dev. Using U-Net network for efficient brain tumor segmentation in MRI images. Healthcure Analytics, 2:100098, 2022. ISSN 2772-4425, doi: https:
(/10i.or//10.0016//.eeaalll 2tt.thh.222.0000000000 URL http://pw/w..ccccceaeecccoe/oo/r $2772442522000429.
Haofan Wang, Zifan Wang, Mengnan Du, Fan Yang, Zijian Zhang, Sirui Ding, Piotr Mardziel, and Xia Hu. Score-CAM Score-weighted visual explanations for convolutional neural networks. In Proceedings of the IEEE/CVE Conference on Computer Vision and Pattern Recognition (CVPR) Workshops, pp. 111 119, June 2020.
Jure Zbontar, Li Jing, Ishan Misra, Yann LeCun, and Stephane Deny. Barlow Twins: Self-supervised learning via redundancy reduction. In Marina Meila and Tong Zhang (eds.), Proceedings of the 38th International Conference on Machine Learning, volume 139 of Proceedings of Machine Learning Research, pp. 12310-12320. PMLR, 18-24 Jul 2021.
Hanwei Zhang, Felipe Torres, Ronan Sicre, Yannis Avrithis, and Stephane Ayache. Opti-cam: Optimizing saliency maps for interpretability. Computer Vision and Image Understanding, 248:104101, 2024. ISSN 1077-3142. doi: https://doi.org//
Bolei Zhou, Aditya Khosla, Agala Lapedriza, Aude Oliva, and Antonio Torralba. Learning deep features for discriminative localization. In Proceedings of the IEEE Conference Computer Vision and Pottern Recognition (CVPR), pp. 2921-2929, June 2016.
17

---

Published in Transactions on Machine Learning Research (02/2025)
Supplementary Material
A Code
Hereafter, present the implementation of the TSM, Multivee-EigenCAAM en MTSM methods proposed in this work using the Pylorrh library for Class Activation Map methods, referenced in Jacob Gildenblat et al. (2021).
import torch import numpy as up
from gytorch_grad_cam base_cam inport BaseCAM
from tensorly decoopostion import tucker
class (BasecAM)
def __init__ (self, model, target_ layers, cue_cuda=aalse, scchae_transform--1
super (TSM, self)... _nt.____i____iiee (model, target_layers, use_cuda, reshape_transtie transforn,
usss_gradients=Falss)
def iacam_naaeeeeeee tepso_t target_layer, target_ activation, batch, grads, eigen_snonth):
feature_mape terch, arrm_nuunpapppete (activattoon_aatthh
[tucker (feature, map. numpy (), yank-list map map. shape)) [1] [0] for
feature_map in Seature_map3
fron_nunpy numpy (np. array (VT))
from
VT[:,0] unsqueeze (-1)) unsqueeze (-1)
(feature maps VT) sum (dim-1)
return 8. abs () numpy ()
import torch import numpy up from pytorch_grad_cet base_cam inport BaseCAN
class MultiVecEigenCan (BaseCAM): def __init__ (self, model, target_layers, use cuda--aaal,,, reshape, transform=None))
initrt....nit_..... (nodel-nodel, target_ lageet.ayyrr-ttteet..areee reshape_ transform-
reshape_transfore usss_graddeete==aalee))
(self, iaput_tensor, target_layer, target_category activation_ get_cam_inee (sel,,
grads, eigen smoothh)o
size (activation_batel) shape (0), activation_batee shape [-2], activation_ batch. shape
])
feature_nape torch wwrm_frrompppppppp feature_mape feature_mant sateee_mpp.va.. viev(
feature_naps shape [0], feature sacuure_aps__ee shape [1], -1) permute (0, 2, 1)
feature_naps feature_maps feature_maps.. mean (dim=1, keepdin=True)
sigma, VT torch linalg. avd (feature_maps, full sigms sigma.real sigma sigma/sigms max (dim=1, xeepdin-True) values
in range (sigma shape [1]):
tmp VTI:, 1]. unsqueeze (-1)
projection torch (aaaaffeeuueeaageee paauuurreepeeeeeeeee_____ tmp) - (size)
weight sigmali, 1, None, None]
projection projection weight
cams, append (projection abs () numpy()) ())
cam = np array (cams)
return cam mean (0)
18

---

Published in Transactions on Machine Learning Research (02/2025)
import
import numpy as np
from attorch_grad_com came_cam inport BaвeCAN
fron tensorly. decomposition import tucker
class NTSM (BaseCCM):
def __init__ (self, model, target_layen use_cuda=False, "eaaape.transtorn-kkooo
auper (MTSK, self).... _intt__init___.. (model model, targeeleeeteeteet_eeeee reshape reshape.transform transform
reehapee.tranatin uses_ sree_grrdenttssaales
def get_ image (self, (sel, input_tensor, target_layer, cargor_category, activation.batch, grads, eigen_smooth):
feature_maps = torch. frrm__umpy (attivation_ VT
singular_valuee []
for feature_me map feature_naps:
cetture_map.uuaeee aunpy
core, factors tucker rank=list (t.
TT.apend singular_valuee append (np. linall.nnnnn. norm
norm (core, axis=(-2,-1) ord='fro'))
torch. prom_numpy i array (VT))
singular_valle values torch. from from_auppy (np.arra
sanulaarvvalues /= Lingular_valuse max (dim=1, keepdin=True) values
saliency_mape []
for i in range (singular_vaaaieeeee values. shape [1]):
tmp VT[:,1] unsqueeze (-1) (nn.uuuueoo (-1)
projection = (feature_mag
tap) mean (din=1)
seight singular.values unsqueeze (-1) unsqueeze (-1)
projection projection weight
saliency_maps append detach () abs O) numpy ())
ap. apray (saliency_mapp
np. upp.transposeG.. axes-(1 (1,0,2,3)
s.mean return
B Experimental setup
B.1 Selected layers per model for the CAM inference
In Table 4, outline the chosen target layers for each examined model. These layers serve as the source from which extract the feature map tensor for computing the saliency map.
Table 4: Designated layers for saliency map calculation across all tested models and CAM methods in this paper.
Model
Layer
Resnet50 ConvNext
(He al., 2016)
model.layer44-11
(Liu et al., 2022)
featurre
VGG16
(Simonyan & Zisserman, 2014)
model.featured
Moco V2
(Chen et al., 2020b)
model.1ayer4[-11
SWAV
(Caron et al., 2020)
model.1ayer4[--1
Barlow Twins
(Zbontar et al., 2021)
model.1ayer4[-11
VieRegL 5e
(Bardes et al., 2022)
model.1ayer4[-1.
VicRegL Conv
(Bardes et al., 2022)
stagesstages [3] [2] dwconv
19

---

Published in Transactions on Machine Learning Research (02/2025)
B.2 Datasets
We evaluate the different saliency map calculation methods in this work the ImageNet 2012 (Deng el al., 2009) and Pascal VOC (Everingham et al., 2010) datasets. All images of these datasets have been rescaled to a size of 224 x 224, and the values of the pixels have been normalized to match the preprocessing of the used pre-trained CNNs.
ImageNet 2012 For this dataset, we use the 50, 000 validation images of the ImageNet ILSVRC 2012 dataset (Deng et al., 2009). We mainly use this dataset to calculate the Average Drop, Average Increase and Mean Squared Error metrics.
Pascal VOC Owing to the availability of the segmentation masks on the Pascal VOC 2012 challenge dataset (Everingham et al., 2010), we harness this dataset to calculate the mean Intersection over Union metric (Jaccard, 1901).
B.3 Used libraries
The implementation of the different CAM based saliency map methods is done via the Py'Torch library for CAM methods (Jacob Gildenblat et al., 2021), the DeepLIFT and DeepSHAP methods are implemented using the Captum Python library (Kokhlikyan et al., 2020) and the LIME method is imported from the official implementation in (Ribeiro al., 2016).
B.4 Used classification pretrained models
In Table 5, we provide a list of pretrained supervised classification models utilized for computing the Average Drop and Average Incrcase metrics on the self-supervised models. The primary rationale behind selecting these classification models is their alignment with the backbone model employed in the self-supervised models.
Table 5: Utilized supervised classification models to calculate the metries of Average Drop and Average Increase on the self-supervised modols.
Self-supervised model
Classification model
Moco V2
(Chen et al., 2020b)
Resnet50
(He et al., 2016)
SWAV
(Caron et al., 2020)
Resnet50
(He et al., 2016)
Barlow Twins
(Zbontar et al., 2021)
Resnet50
(He et al., 2016)
VicRegL Resnet50
(Bardes et al., 2022)
Resnet50
(He et al., 2016)
VicRegL ConvNext.
(Bardes et al., 2022)
ConvNext
(Liu et al., 2022)
C Hyperparameter study for the mloU metric on self-supervised models
In Figure 7, illustrate our investigation into the threshold hyperparameler employed to binarize saliency maps into segmentation masks and its impact on the mloU metric. We systematically vary the binarization threshold from 0.4 to 0.9 with a step size of 0.1 and observe the corresponding mloU values. We exclude threshold values lower than 0.4 as we deem them irrelevant for segmentation. From the observations in Figure 7, becomes evident that, irrespective of the pretrained self-supervised model, TSM and MTSM yield more accurate approximation of the segmentation mask than EigenCAM or Multivee-EigenCAA.
20

---

Published in Transactions on Machine Learning Research (02/2025)
EigenCAM
Moco V2 TSM
-X- SWAV EigenCAM
Barlow ElgenCAM
Barlow Twins TSM
-
VicRegL Resnet50 ElgencAM
VicRegi. Resnet50 TSM
VicRteLL EigenCAM
VicRegl. ConvNext
miuu
20
10
0.4
0.6
0.7
0.9
Threshold
Single
40
30
mieu
Multivec-EigenCAM
Moco V2 MTSM
-X- SWAV Multivec-EigenCAM.
SWAY MTSM
Multivec-EignCAMM
Barlow Twins MTSM
VicRegL Resnet50 Multivec-EigenCAM
VicRegL Resnet50 MTSM
VicRegL Multivee-EigenCAM
VicRegL ConvNext
0.4
0.6
0.7
0.8
0.9
Threshold
(b) Multi-vector
Figure 7: On the exploration of the threshold hyperparameter for Class Activation Map (CAM) binarization and its influence on the mloU metric, solid lines in the plot depict the mloU values obtained using Tucker decomposition based CAM methods, while dashed lines represent the mIoU values obtained using SVD basod CAM methods.
21

---

Published in Transactions on Machine Learning Research (02/2025)
D Additional self-supervised models' saliency maps
From the first three images in the Figurcs 8, 9, 10, 11, 12, 13, 14 and 15 we notice that when we have one class the image with multiple subjects all the saliency map methods are able to identify the objects in the input image. However, when we have two classes, as in the forth image with the dog and the cat at the top and bottom of the image, TSM outperforms EigenCAM as is able to locate the cat in the image which CigenCAM fails to do. Furthermore, all the multivector methods are able to identify all the objects in the image.
Input image
CAgenCAMM
TSM
Mlutiive--igeenAAN
MTSM
Figure Qualitative comparison of the saliency maps extracted using the EigenCAM method versus the TSM, Multivec-Figee FitiiveeeEEggCCAMMM and MTSM methods the VicRegL Resnet50 model (Bardes et al., 2022) applied to the ImageNet dataset (Deng et al., 2009).
22

---

Published in Transactions on Machine Learning Research (02/2025)
Input image
AigenCAM
TSM
Huitivee-NigenCAAM
MTSM
Figure 9: Qualitative comparison of the saliency maps extracted using the FigenCAM method versus the TSM, Multivec- CigeeeCA and MTSM methods on the VicRegL CouyNext model (Bardes et al., 2022) applied the ImageNet dataset (Deng al., 2009).
23

---

Published in Transactions on Machine Learning Research (02/2025)
Input image
CigenCAM
TSM
Hlalivec--iigeCCae
MTSM
Figure 10: Qualitative comparison of the saliency maps extracted using the FigenCAM method versus the TSM, Multivec-Eigge CigereCAM and MTSM methods on the Barlow Twins model (Zbontar et al., 2021) applied to the ImageNet dataset (Deng et al., 2009).
24

---

Published in Transactions on Machine Learning Research (02/2025)
Input image
AigenCAM
TSM
Multivee-IieenAAM
MTSM
Figure 11: Qualitative comparison of the saliency maps extracted using the EigenCAM method versus the TSM, Multivee- Caaevee-EigenCAMM and MTSM methods on the Moco V2 model (Chen et al., 2020b) applied to the ImageNet dataset (Deng al., 2009).
25

---

Published in Transactions on Machine Learning Research (02/2025)
Input image
AigenCAM
TSM
Multivee-
MTSM
Figure 12: Qualitative comparison of the saliency maps extracted using the EigenCAM method versus the TSM, Muulivee-EEigeCAAe and MTSM methods on the SWAV model (Caron et al., 2020) applied to the ImageNet dataset (Deng al., 2009).
26

---

Published in Transactions on Machine Learning Research (02/2025)
E
Additional supervised models' saliency maps
Laput image
GradCAM
GrdCAMMe-+
HAM
NGmACAMI
CAlatioCAM
OpUCAM
DeeLIFFT T
DOP
LIME
KigoCAM
TSM
EigenCAM
MTSM
Figure 13: Qualitative comparison of saliency maps calculation methods on the supervised classification Resnet50 model (He et al., 2016) applied to the ImageNet dataset (Deng et al, 2009).
27

---

Published in Transactions on Machine Learning Research (02/2025)
Eaput imane
GraCCAM
GrdCAMM++
HAM
NGmaCAM
CAlatioCAM
OpUCAM
DeeLLIFT
DOP
LIME
KigoCAM
TSM
EigenCAM
MTSM
Figure 14: Qualitative comparison of saliency maps calculation methods on the supervised classification ConvNext model (Liu et al., 2022) applied to the ImageNet dataset (Deng et al., 2009).
28

---

Published in Transactions on Machine Learning Research (02/2025)
Iaput image
GraCCAM
GadCAMe++
HAMRoWCAM
NGmaCAM
CAlatooCAM
OpUCAM
DeeLLIFT
DOP
LIME
KigoCAM
TSM
EigenCMM
MTSM
Figure 15: Qualitative comparison of saliency maps calculation methods the supervised classification
VGG16 model (Simonyan & Zisserman, 2014) applied to the ImageNet dataset (Deng et al., 2009).
29

---

Published in Transactions on Machine Learning Research (02/2025)
F
Self- suff-vupeeiied models image segmentation based on CAMs
Input inage
Mosked image
EigenCAM
TSM
Multives-EigenCAae EigenCAM
MTSM
Figure 16: Qualitative comparison of the binarized saliency maps extracted using the EigenCAM method versus the TSM, Multivec-FigenCAN and MTSM methods the VicRegL Resnet50 model (Bardes et al., 2022), juxtaposed with the segmentation mask from the Pascal VOC dataset (Everingham et al., 2010). The binarization is performed using 0.5 threshold.
Input image
Masked image
EigenCAM
TSM
Muleivee-EigenCAM
MTSM
Figure 17: Qualitative comparison of the binarized saliency map extracted using the EigenCAM method versus the TSM, Aultivec-EigenCAN and MTSM methods on the VicRegL ConyNext model (Bardes et al., 2022), juxtaposed with the segmentation mask from the Pascal VOC dataset. (Everingham et al., 2010). The binarization is performed using 0.5 threshold.
30

---

Published in Transactions on Machine Learning Research (02/2025)
Input image
Masked image
EAgenCAM
TSM
Multiveo-Eipee EigenCAM
MTSM
Figure 18: Qualitative comparison of the binarized saliency map extracted using the EigenCAM method versus theTSM, Multivee-EEigeCAle and MTSM methods on the Barlow Twins model (Zbontar al., 2021), juxtaposed with the segmentation mask from the Pascal VOC dataset (Everingham et al., 2010). The binarization is performed using 0.5 threshold.
Input image
Masked image
EAgenCAM
TSM
Multivec-Eigee EigenCAM
MTSM
Figure 19: Qualitative comparison of the binarized saliency map extracted using the EigenCAM method versus the TSM, Multivec- Multivee-EigenCAM and MTSM methods on the Moco V2 model (Chen al., 2020b), juxtaposed with the segmentation mask from the Pascal VOC dataset. (Everingham et. al., 2010). The binarization is performed using 0.5 threshold.
31

---

Published in Transactions on Machine Learning Research (02/2025)
Input image
Masked image
EAgenCAM
TSM
Multiveo- Multiveo-EigenCAe EigenCAM
MTSM
Figure 20: Qualitative comparison of the binarized saliency map extracted using the EigenCAM method versus the TSM, Sualtivee-EigeCAAA and MTSM methods on the SWAV model (Caron et al., 2020), juxtaposed with the segmentation mask from the Pascal VOC dataset (Everingham et al., 2010). The binarization is performed using 0.5 threshold.
32


Reviews:
[{"review_id": "eTY4bz6lwR", "reviewer": "Reviewer_P129", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strengths:\n\n1. The method can create higher-quality saliency maps\n\n2. MTSM is competitive with label-dependent methods on supervised models.\n\n\n\nWeaknesses\n\n1. The method seems to be hard to generalize to other architectures like transformers.\n\n2. The computational cost should be discussed in detail like tables or Figures.", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "The paper Introduced Tucker Saliency Map (TSM), which uses Tucker tensor decomposition to better analyze CNN feature maps' structure compared to existing methods. It can generate more accurate singular vectors and values and create higher-quality saliency maps. The improved MTSM achieves competitive performance with label-dependent methods.", "claims_and_evidence_yn": "Yes", "claims_explanation": "", "audience_yn": "Yes", "audience_explanation": "", "requested_changes": "I think the paper is interesting and does not need much adjustment. Except that the computational cost should be discussed. And the clear limitation on architecture should be explicitly stated.", "broader_impact_concerns": "N/A", "additional_comments": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "SeerAZGOks", "reviewer": "Reviewer_z2n1", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strengths:\n- This work is very well introduced and the paper is easy to read. I really appreciate that the authors took the time to write a good background section as well as the relevant related work.\n- In addition of leveraging the Tucker decomposition instead of SVD, the author developed a multivector version for both of them.\n- The experimental section with both use cases in supervised and self-supervised settings is very relevant. Especially since the authors are leveraging different architectures (Resnet/ConvNext/VGG) and different SSL methods (Moco, SWAV, BT, VICReg)\n- It's also great that the authors leverage different metrics such as AD, AI, MSE and mIoU to confirm their findings.\n- The authors also provide a lot of visualization between different methods and also a good ablation over the threshold value for mIoU.\n- The code is in the appendix as well as the experimental details.\n- The authors are correctly addressing the limitations of their work by acknowledging that their method has an increased computational cost.\n\nWeaknesses:\n- Most of the models used have been pre-trained on ImageNet which is single object classification (and for which in many case, the object of interest will be close to the center). I am wondering wether this method would be able to find multiple objects in an image. Like all the visualizations we can see in the paper have the object of interest that is in the center of the images. What happen if you have a photo of two cats and each one of them is close to the image border?", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "This paper focuses on features map interpretability in convolutional neural networks. Instead of relying of labels, the authors introduced an unsupervised interpretability method that leverage Tucker decomposition of the features map to be able to capture the direction of higher variances which can be used to highlight objects of interest in images. This Tucker Saliency Map (TSM) is presented as an alternative to another unsupervised interpretability method called EigenCAM which rely on SVD. The authors argue that the Tucker decomposition is better than SVD to approximate the first singular vector with the higher variance. The second contribution of the authors is to introduce a multivector version of EigenCAM and TSM. Instead of using a single singular vector, they compute one saliency map by singular vectors and then perform a weighted sum based on the singular values across the saliency maps. The authors test their methods with supervised and self-supervised models on the following metrics: AD (Average Drop), AI (Average Increase), MSE (Mean-Squared Error), mIoU (Mean Intersection over Union). On both supervised and self-supervised models, the authors show that TSM is better than EigenCAM (same for their multivector version). For supervised training, the performances are still the best for supervised methods such as GradCam. The authors claim to have the state of the art for self-supervised explainability.", "claims_and_evidence_yn": "Yes", "claims_explanation": "", "audience_yn": "Yes", "audience_explanation": "", "requested_changes": "This is a good paper which I think will be of interest to the TMLR's audience. The claim and evidence seems also to be strong enough. I do not have any strong request except maybe some visualizations in which the object of interest is not in the center and what happen when there are multiple objects from a given class. Same question on having visualization on what happen if there is two different classes in the same image (like a dog + cat).\n\nI would also advise against using images of people for visualization in the paper if the authors did not get explicit consent from them.", "broader_impact_concerns": "Do not have any Broader Impact Concerns.", "additional_comments": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "z10F1oAXFz", "reviewer": "Reviewer_94v7", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "## Strengths\n\nThe method is clearly laid out and the background section is useful to understand the method. The advantages of the TSM and MTSM methods are clearly explained and the experimental data shows improvements. Further, the authors evaluate their method also in the context of supervised learning, which is helpful to understand the strengths and weaknesses of the proposed methods.\n\n## Weaknesses\n\nThe paper has no major weaknesses. As the authors acknowledge, the new TSM and MTSM methods are most effective in the case of unsupervised learning, but this is expected and discussed.\n\nThere are, however, some errors that I believe should be addressed; please see the requested changes section below.", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "The paper proposes a method for improving interpretability (note: the authors use 'explainability') of CNNs using a technique from Tensor calculus, i.e., multi-dimensional extensions of matrix decompositions. This extension is especially useful in the context of unsupervised learning (e.g., clustering) as it is class-independent. The authors extend their method, as well as the method EigenCAM from literature into a multivector version, which uses a weighted sum of vectors from the SVD/Tucker decomposition to capture more of the variance and produce more useful saliency maps.", "claims_and_evidence_yn": "Yes", "claims_explanation": "", "audience_yn": "Yes", "audience_explanation": "", "requested_changes": "1. In Table 1, some of the numbers don't agree with the existing literature (see the GradCam++ paper, https://arxiv.org/pdf/1710.11063), table 10. Could you please explain why the results are different?\n1. Please strongly consider replacing \"explainability\" with \"interpretability\" everywhere (see, for instance, Section 2.1 of Linardatos, P.; Papastefanopoulos, V.; Kotsiantis, S. Explainable AI: A Review of Machine Learning Interpretability Methods. Entropy 2021)\n2. I agree with the authors' basic premise that the majority of saliency attribution methods require a reference class and thus are unsuitable for, e.g., clustering algorithms. However, please note that these methods can be used with reference to any class, not necessarily the correct one, and so the labels are not required. I would ask the authors to update their papers to reflect this distinction. (e.g., in the last sentence of page 1 - \"Most existing techniques rely heavily on ground truth labels to generate saliency maps\".\n3. It would be helpful if the authors explained right away that by \"tensors\" they simply mean 'multidimensional arrays', since Tensors also refer to the actual multilinear map (rather than just its numerical representation)\n4. (Minor) - on the bottom of page 4, please justify why \"gradient information is inherently noisy\" and \"such loss depends heavily on the batch size\"\n5. For Figures 4, 5, and 6, please also provide the raw image, as it is hard (even on an ipad) to see what's under the saliency maps.", "broader_impact_concerns": "None", "additional_comments": "", "extra_scores": {}, "extra_text": {}}]

Author rebuttal:

Meta-review: Altogether, the reviewers are all positive about this work. In their final recommendations, they acknowledge that the work is interesting and provides a clear contribution to the field of CNN interpretability.
Decision: Accept (poster)