Reproducibility Study of "Languange-Image COnsistency"
Authors: Konrad Szewczyk
Patrik Bartak
Mikhail Vlasenko
Fanmin Shi
Venue: Accepted by TMLR ()

Abstract: This report aims to verify the findings and expand upon the evaluation and training methods from the paper LICO: Explainable Models with Language-Image COnsistency. The main claims from the original paper are that LICO (i) enhances interpretability by producing more explainable saliency maps in conjunction with a post-hoc explainability method and (ii) improves image classification performance without computational overhead during inference. We have reproduced the key experiments conducted by Lei et al.; however, the obtained results do not support the original claims. Additionally, we identify a limitation in the paper’s evaluation method, which favors non-robust models, and propose robust experimental setups for more comprehensive quantitative analysis. Furthermore, we undertake additional studies on LICO’s training methodology to enhance its interpretability. Our code is available at https://github.com/konradszewczyk/lico-reproduction.

Mikhail Vlasenko Published in Transactions on Machine Learning Research (10/2024)
Reproducibility Study of "Languange-Image COnsistency"
Patrik Bartak!
patrik. hatriiabrrraak@sttudee........ uvo.nl
Informatics Institute, University of Amsterdam
Konrad Szewezyk!
Informatics Institute, University of Amsterdam
mikhail, Shakaatt.daeenk@@@ttddeenuu....
Informatics Institute, University of Amsterdam
Fanmin Shi
fonmmin..h@@@tuudd nl
Informatics Institute, University of Amsterdam
'These authors contributed equally to this work.
Reviewed on OpenReview: https: openreviee......... net/ forum? id- PuRTs
Abstract
This report aims verify the findings and expand upon the evaluation and training meth- ods from the paper LICO: Explainable Models with Language-Image COnsistency. The main claims from the original paper are that LICO (i) enhances interpretability by produc- ing more explainable saliency maps in conjunction with a post-hoc explainability method and (ii) improves image classification performance without computational overhead during inference. We have reproduced the key experiments conducted by Lei et al.} however, the obtained results do not support the original claims. Additionally, we identify a limitation in the paper's evaluation method, which favors non-robust models, and propose robust exper- imental setups for more comprehensive quantitative analysis. Furthermore, undertake additional studies LICO's training methodology to enhance its interpretability. Our code is available at https: //pp::///ithbb.. http////ooooooeeees/o/oooooooooa
1 Introduction
Despite deep neural networks showing state-of-theeart performance in numerous computer vision tasks Chai et al. 2021), their application in safety-critical tasks like medical diagnosis is limited due to their black-box nature, making it hard to understand how decisions are made. While contemporary post-hoc explanation methods like Grad-CAM (Selvaraju et al. 2019 or RISE (Petsiuk et al. 2018) can increase the transparency in these systems, they cannot be used to improve the decision-making process underneath to make it more consistent with human intuition.
Previous model- based methods proposed improving the models by enforcing consistency constraints the post-hoc explanations generated during training (Pillai et al. 2022 Pillai & Pirsiavash 2021). In the reviewed study, Lei ct al. (2023) propose Language-Iman COnsistency (LICO), training framework for enhancing model interpretability. Unlike the previous methods, LICO does not utilize post-hoc explanations during training. Instead, the loss enforces consistency between the manifold visual features and the manifold of class-aware semantic information from text encoder of a pretrained vision-language model (VLM), such as CLIP (Radford al., 2021). By doing so, the authors claim to achieve both improved interpretability and classification performance.

---

Published in Transactions on Machine Learning Research (10/2024)
In recent years, there has been growing concern about the reproducibility of AI research (aker 2016 Hernández & Colom 2023 where it is becoming increasingly difficult to reproduce and validate the findings of the papers, which undermines trust in the scientific community and leads to the waste of resources spent on pursuing unreproducible results. The causes of this crisis include incomplete documentation of the experimental procedures or introduced methods (Semmelrock et al. 2023) as well as the need for significant computational resources (Nature Computational Science Editorial Board 2021). For this reason, we attempt to verify whether the LICO authors' claims can be reliably reproduced based on the paper. Our analysis is extended with additional experiments evaluating the interpretability of the trained models and modifications to LICO. The following are the main contributions:
We reproduce the key experiments conducted by Lei et al. to find out which of their claims can be reliably reproduced by following the paper. Furthermore, we provide information about the computational costs required during the reproduction.
Reproducibility Study:
Improved Codebase: The code delivered by authors is incomplete and cannot be executed with- out non-trivial work. We provide our re-implementation and documentation of the LICO method, together with the experimental setup demonstrating its interpretability and classification perfor- mance.
Extended Evaluation: We provide justification that the Insertion and Deletion tests used by Lei et al. are not indicative of the model's interpretability. Instead, we propose evaluation setup for qualitative analysis of alignment between saliency maps and human expectations (priors).
LICO Extensions: We conduct additional experiments the language knowledge extraction component of LICO by (i) introducing trainable prompts specific to each class and (ii) positioning the class label tokens before the trainable prompts with the goal of improving the method.
Our work is structured as follows. Section 2 presents the claims of the paper we attempt to replicate. In section 3 we introduce the method proposed by Lei et al.] and describe the experimental sctup used. Then, covers the results of both replicated and extended evaluation conducted by us. The report concludes with section where we discuss our findings about the reproducibility of LICO.
section 4
2 Scope of Reproducibility
A central idea in model-based methods comparable with LICO is enforcing consistency with an additional human prior during training (Han et al. 2021 Pillai & Pirsiavash 2021 Pillai et al. 2022 In LICO, this prior takes the form of semantic information encoded within a pretrained text encoder, which should allow for training more interpretable models without compromising the quality of classification. In this work, we will focus on the following claims by Lei et al.
Enhanced Interpretability: By matching the manifolds of visual and text embeddings, the se- mantic information from the pretrained text encoder can guide the image classification model focus distinguishing features of objects. As result, the explanations for model decisions are more consistent with human expectations, which is reflected in both quantitative and qualitative analyses.
2. Improved Classification Performance: As opposed to the previous methods increasing the inter- pretability of image models, LICO docs not bring a deterioration to the classification performance and, in some cases, may even result in increased accuracy, when compared with baseline models trained without LICO.
3. Necessity of both Manifold Matching and Optimal Transport Loss: LICO introduces a training method that supplements the traditional cross-entropy loss with two additional loss com- ponents (described in scction 3.1). both with goal of bringing the manifolds of text and visual features closer to each other. Although these two components serve similar functions, the mutual presence of both is beneficial the quality of the resulting models.
2

---

Published in Transactions on Machine Learning Research (10/2024)
In addition to validating the claims brought in the original paper, we analyze the impact of the LICO method on the training times of the model. Moreover, we extend the interpretability study by proposing an enhanced experimental setup consisting of three quantitative tests.
3 Methodology
This section describes the approach taken to reproduce the work. We start with an overview of the LICO algorithm, and then continue by describing the datasets and hyperparameters used. We then discuss the experimental setup and code specifically, which parts of the authors' implementation were missing or incomplete, the parts we re-implemented based on the paper, and the resources that we used. Lastly, we introduce the quantitative metrics that we use for the evaluation.
3.1 The LICO Algorithm
LICO is a training framework for supervised image classification models that utilises a pre-trained text encoder (e.g. from CLIP models) to guide the training process of the image classifier such as ResNet- 18. This achieved by supplementing the the standard Cross-Entropy (CE) loss used to train the model by adding Manifold Matching (MM) and Optimal Transport (OT) loss components, weighted by a and B parameters respectively. A general description of both is provided in the following paragraphs.
The idea behind LICO is presented in fig. | with the most important concepts to consider being image manifold and language prompt manifold. The image manifold is defined by feature maps fe(æ;) from the produced by CNN within the image classifier, and the language prompt manifold is defined by projected text embeddings (t)((()))) obtained by passing a prompt through a pretrained Vision-Language Model (VT.M) and trainable MLP (which matches the dimensionality of the visual feature maps and the text embeddings). This prompt consists of the ground truth class label and several trainable tokens.
The MM and OT losses aim to align these two manifolds, such that the image manifold becomes enhanced with the semantic structure allegedly present in the language prompt manifold due to the multi-modal training of the VLM from which the embeddings are extracted. The goal of MM loss is to provide coarse alignments between the two manifolds based on the similaritics between pairs of samples within each mini- batch, while OT loss responsible for establishing a correlation between text tokens and feature maps for individual data points.
Other than images and their ground truth class labels, additional information is needed to make use of LICO. After training, everything except the image model is discarded. This means that model trained using LICO only differs from a non-LICO model by its parameters, and LICO therefore docs not increase the computational cost at inference.
3.2 Datasets
We train and evaluate the presented models on two image classification datasets. Following the original experiments, we use CIFAR R-100 (Krizhevsky et al. 2009), which provides 50000 training and 10000 vali- dation images divided into 100 classes. Additionally, we use ImageNet-SSo (Gao et al. 2022), consisting of 64431 training images and 752 validation images with segmentation masks and bounding box information that we use for extended evaluation. Both datasets are balanced, containing approximately the same amount of samples of cach class. The preprocessing performed during training described in appendix B
The selection of these datasets, and not full ImageNet-1k, has been dictated mostly by the computational limitations have encountered during our work. However, believe those to be sufficient to show whether the main claims of the LICO paper hold. As the original authors have used CIFAR-100 in their work, we should be able to see the same trends present in our results as they have. However, we acknowledge the fact that limited resolution of images within CIFAR- 100 may limit the ability of the method to connect the visual cues with semantic information from VLM. For this reason, we have chosen to conduct additional experiments using ImageNet- dataset, a subset of the original ImageNet- 1k limited 50 classes. The higher resolution with images remove the possibility of LICO's impact being limited by the lack of distinguishable features in
3

---

Published in Transactions on Machine Learning Research (10/2024)
classification
Language
Extraction
Classifee
Ccs
96
She
fo (az,
Image:
Feature Extraction
Classthcatio
N
Prompt: E
(t.
Optimal Tramport Aligument
3 Parameter updated
10) Parametes fiozen
fo Image encoder Text encoder
Cot
MLP
Language Prompt Manifold
Csm
Image Manifold
Cor
OT
Figure Framework of the LICO method. (a) Conventional classification pipeline of DNNs. (b) Language feature extraction with pro-trained text encoder. (c) Manifold matching among samples and optimal trans- port aligument between feature maps and prompt tokens within each sample. Figure and caption adapted from Lei et al. 2023
blurry images of CIFAR-100. An additional benefit of this datasets is the presence of segmentation masks and bounding boxes for objects, which we utilise in our extended analysis in the Section 4.2.1
3.3 Hyperparameters
We the original values for the hyperparameters that specified by Lei al. [2023]: SGD optimizer with learning rate 0.03, momentum 0.9, weight decay 0.0001, and cosine rate decay schedule ie. 7 1/1 cos IGk where V/ denotes the initial learning rate, k is the index of training step, and K is the total amount of training steps. The LICO-specific parameters are also used unchanged: = 10, 3 = 1, and the hidden dimension the text projection MLP is 512. We use 100 epochs for all tested datasets and the ResNet-18 architecture unless otherwise stated.
2akk
3.4 Experimental Setup and Code
Some code for LICO has been made publicly available by the authors via GitHub However, as of the time of writing, the published codebase does not include a significant portion of the experimental and method-related code. While the currently available README file provides some guidance, additional details would greatly decrease the effort required for an accurate reproduction of the original experiments. Implementations of explainability methods, such as GradCam and the metrics used for evaluation, like insertion and deletion score, were not provided in the available codebase. Because of that, use the CGC GitHub repository for implementations of GradCam, insertion score, and deletion score.
Consequently, we re-implemented most of the method based on the paper, except for computing the OT loss, which is present in the authors' codebase. In our work, we follow the paper as closely as possible. Our codebase includes the complete LICO method, scripts for experiments, thorough documentation, and instructions how reproduce the results of our extended evaluation. To reduce the amount of code needed for the implementation, and to increase readability, we use the PyTTrrc Lightning framework (Falcon & The PyTorch Lightning team 2019
Our code diverges from the formulas by Lei et al. in the way the temperature parameter of the MM loss is implemented. Inspired by the implementation of CLIP (Radford et al. 2021), we multiply by the exponent of the trainable value instead. The range of valid values in this case the same, but the multiplicative factor is naturally in the interval (0, oo), avoiding a possible division by zero or negative temperature. Further in line with CLIP, we bound the trainable parameter to Jo, (o100)).. Not doing so allows trivial solution for the MM loss: minimize the temperature -o and achieve festure-independent uniform softmax output.
htape:///gthuub.cmm//leeeeeeeeee
https://github. wtt.:la..ooomD//Doiiiiioooooo
4

---

Published in Transactions on Machine Learning Research (10/2024)
Furthermore, the division by unconstrained trainable parameter risks introducing the division by zero negative number. These potential issues were not explicitly discussed by the authors, and we addressed them by multiplying by a logarithm.
Moreover, the original paper does not provide specific details about the normalization methods applied to text and visual features in the calculation of MM and OT losses. Our initial experiments indicated that the training process with unnormalized feature vectors led to instability due to the high magnitudes of gradients. Subsequently, we observed that the magnitude of the OT loss component scales with the L2-norm of the feature vectors. To compensate for that, conduct L2-normalisation of text and visual features before the calculation of MM and OT losses, which increases the stability of the training.
In line with the authors, we use top-1 and lop-5 accuracy metrics to assess the classification performance. We also use Insertion and Deletion scores [Petsiuk et al. 2018) for quantitative analysis of model interpretability, However, in section 3.4.1 we suggest that this metric may not be the most appropriate for mcasuring the quality of explanations in model-based methods, and we propose an alternative evaluation setup.
3.4.1 Quantitative Analysis of Salience Maps
First, we show that the metrics used in the original paper, insertion and deletion scores, are applied in a context for which they were not originally intended. Then, we propose an extended experimental setup to quantitatively assess the interpretabllity of LICO and analyze the consistency of its explanations with the human prior. Our approach involves the use of three metrics: Salience Equivariance Similarity (SES), Segmentation Content Heatmap (SCH), and the Multi Object Salience Uniformäty (MOSU).
Insertion and deletion: Initially proposed by Petsiuk et al., these metrics were introduced for the as- sessment of post-hoe interpretablity methods that produce saliency maps, not the models. The metrics work as follows: given a picture, a saliency map for the target class is extracted from the model. Then, for insertion, the blurred version of the image taken, and the parts that have high saliency values are gradually unblurred. At every step, the output probability of the model for the target class is recorded, The value of the insertion metric is the area under the curve (AUC) of the graph with share unblurred (the percentage of pixels within the image being unblurred in the order of decreasing salience) the x-axis and output probability of the target class as the y-axis. The deletion acts similarly, but instead, the evaluation starts with a fully complete image, and the high-saliency regions are greyed out. The insertion needs to be maximized, while deletion minimized.
As such, a good metric score for a combination of a model and an explanation method merely demonstrates that the explanation method has done well. As demonstrated in appendix [A] even a model that focuses on small regions and lacks robustness can perform well on these metrics, highlighting their limitations in this context.: removing a tiny part completely confuses the model, and adding that part immediately makes the model predict the correct class.
The LICO method aims to change the model itself in order to make it more interpretable. Therefore, the comparison between the insertion and deletion scores of a baseline model and a model trained with LICO is not sufficient to claim that one model is more interpretable than the other.
Salience Equivariance Similarity: One intuitively desirable property of saliency maps is for them to be near-equivariant under certain transformations. This idea has resulted in the previously mentioned CGC method, created by Pillai et al. (2022), which uses contrastive loss in order to encourage saliency maps that are both equivariant (under translation and cropping), and distinct from saliency maps of other images.
Bocause LICO docs not explicitly encourage equivariance, yet claims to outperform CGC, it is of interest to investigate whether CGC retains an advantage in this respect. We compute the SES metric, used by (Pillai et al. 2022). as follows for an image 1:
SES(T) = (c((((())
(1)
With - denoting cosine similarity, s(-) being a post-hoc saliency map method (in our case GradCam), and t(-) being some spatial transformation (in our case translation and cropping). is important to note that for spatial transformations, this metric can be trivially maximized by simply returning uniform saliency
5

---

Published in Transactions on Machine Learning Research (10/2024)
Salience Equivarlance Similarity
Segmentation Content Heatmap
Augrent
GruOCAM
Sum
0.71
Buatca
sum
Rugrass
Coshne Siilriiy
0.52
Maltoaadd
Eng Hy My
SES(T) = sc((()(().
SCH(H,
Ens Hu
Mutli-Object Salience Uniformity
BraICAM
Isum(
sum(
[sum(
summ
Marca
Psalience
suml
sumn
sum(
sum(
(Vaaiteee:::::: Parca)
DRreVeres Cealience)
Figure 2: graphical overview the evaluation methods our evaluation sctup: SES, SCH and MOSU. It should be noted that for the same of clarity, the computation MOSU metric presented in this figure is limited two instances of objects in the images; however, all sums included in the formula are over all instance of the objects present in the image.
map for any image. It must therefore be considered in combination with other metries, such as the one we discuss next.
Segmentation Content Heatmap: One of the obstacles in producing models consistent with human intuition the reliance image classification models on spurious features that are associated with the target object, but are not causally related to it. Hence, our second evaluation concerns the intuition that the features used by the model classify the object ought lie within the boundarics of the object. Using specific variant of the Content Heatmap (CH) metric proposed by Pillai & Pirsiavashi (2021). h, consider dataset where segmentation masks for the main object are provided, and take the weighted share of the saliency heatmap that lies within the segmented class. More precisely, we compute the SCH for some image as:
Eis Hij Mú Ens Hig
SCH(H, M)
(2)
Where is matrix with salience values H. E [0, 1] for each pixel, and M a matrix representing a binary segmentation mask obtained from the dataset, with My E 10, being if the pixel belongs to the target object, and otherwise. Such metric shows how much importance the model assigns to the target object. It does not penalize if only part of the object is required perform the classification, but discourages attending to features that do not identify the class in way consistent with human expectations.

---

Published in Transactions on Machine Learning Research (10/2024)
Multi Object Salience Uniformity: The last component of the proposed experimental setup provides quantitative evaluation of the performance of the model when multiple objects of the same class are visible in the image. In such a case, a human would expect the explanation to cover all of the instances. However, the model could casily make the correct prediction based only on a single occurrence of an object. We propose to consider the distribution of salience within the bounding boxes of instances of a given class for data points where multiple instances are present. We would expect it to be proportional to the area of a bounding box encompassing the particular instance. To measure the distance between distributions, MOSU uses Beilbacck-Leibble divergence:
MOSU(Farea: Vealience) ("orea
Usalience)
Where Vsatience IR denotes the probability vector corresponding to the distribution of salience lying within the bounding box encompassing each of the n instances of the class in an image with its elements equal to Usalience [i] Baliencetn/ooundiggBoeel with SalieneennBuundngBooii] being sum of salience within the bounding box of the object i. In similar manner, Varea E IR can be defined as vector such that Varea [1] = Σ Boundinggox Arvsell where corresponds to each of object instances in the image. To our knowledge, MOSU is a novel metric. Since this measure does not lake into account salience altributed to regions outside of objects, it should be used in tandem with the SCH metric or its variant utilizing bounding box information Piilla et al. 2022
SalleneelaBouudingBoor
3.5 Computational Requirements
Aiming to provide a more robust evaluation, we decided to run the experiments with multiple seeds, rather than once like the original authors do. To accommodate that, we needed to focus on datasets smaller than the full ImageNet-1k due to limited computational resources. For that reason, we have decided to limit our reproduction to CIFAR- 100 and ImageNet-Sap datasets as described in 3.2 This allowed us reduce the hardware requirements of our reproduction compared to Lei et al. (0023). For the experiments, use 2 machines with the following GPUs: NVIDIA Gelorce RTX 4090 (Machine 1), and NVIDIA A100-SXM4- 40GB (Machine 2).
The time taken per training epoch in different configurations is provided fully in table 1 observe that the LICO and CGC take significantly more time and memory to train. We also notice that the time increase depends on the model size and batch size. The table does not provide standard deviations as they are all near 0.
We generally
Table 1: Seconds per epoch for RResNet-18/ReeNetteee Evaluated Machine 1 at 16-bit precision.
Training method CIFAR100, batch 64
ImageNettS-50,, batch 64
ImageNet-S-500. batch 128
Baseline
23/68
42/87
43/91
LICO
08/173
127 02/223
102/13
CGC
107/245
133/316
135
4 Results
4.1 Results reproducing original paper
Claim 1. Enhanced Interpretability
The experiments in Table 2 based on Insertion and Deletion metrics used by Lei et al. show deterioration over the bascline, which is not in line with the authors's claims. However, due reasons described in section section 3.4.1 this may not be sufficient evidence to judge the interpretability of models; hence, the extended quantitative evaluation is conducted in section 4.2.1
Insertion and Deletion Tests:
324 GB VRAM is not enough this experiment at fp16 precision. Same experiment with the baseline takes 6.9 GB of VRAM.
7

---

Published in Transactions on Machine Learning Research (10/2024)
Table 2: Insertion and deletion on ImageNet-SSp using ResNet-18. '4' indicates standard deviation.
Method
Inscrtion (f)
Deletion (4)
Combined Insertion Deletion (1)
CE loss (baseline)
.727 .003
.243 .006
.484 .008
MM loss
.725 + .003
.248 + .011
.477 + .011
+ OT loss
.707 + .004
.258 + .004
.447 .008
+ MM OT (LICO)
.706 + .012
.263 + .008
.443 .019
Visual Analysis of Saliency Maps: To qualitatively investigate the interpretability of LICO, saliency maps were also analyzed visually. Several images were sampled randomly from the CIFAR-100 and ImageNet- S50 datasets, The samples can be seen in fig. 3 Ideally, we would like to see that GradCam saliency maps of LICO models are less often focused on spurious features. However, the opposite can gencrally be observed, indicating that the saliency maps tend to be less interpretable. We also noted that saliency concentrates the extreme edges of images for LICO models notably more often than for the baseline. This can be seen in fig. 3 for classes "snail", 'bottle', and "kuvasz'.
"snail"
"table"
"boy"
"bottle"
"kuvasz"
"goldfinch"
"siamese cat"
"ladybug"
Baseline
Boseline
LCO
LICO
(a)
Examples of saliency maps on CIFAR-100
(b) Examples of saliency maps on ImageNet-Sea
Figure 3: Examples of GradCam saliency maps for baseline ResNet-18 and ResNet-18 + LICO sampled from CIFAR-100 and TmageNet-Sso part of the qualitative evaluation.
Claim 2. Improved Classification Performance
Although LICO trained CIFAR-100 outperforms CGC model on top-1 and top-5 accuracy, it does not improve upon a baseline ResNet-18 as seen in Table 3 Likewise, the experiments conducted on ImageNet- Se0 dataset in Table do not support the claim that the classification accuracy of bascline models can be improved by training with LICO.
Table 3: Classification accuracy and SES
CIFAR-100 using ResNet-18
Method
Top-1
Top-5
SES
ResNet-18
.7431
.9320
.447
CGC
.7250
.9153
.949
LICO
.7309
.9285
.485
Claim 3. Necessity of both MM and OT Loss
Interpretabillit:: Experiments on ImageNet-Sga using Insortion and Deletion metrics Table show that training with MM loss achieves a Combined score that slightly worse compared to ResNet-19 baseline, whereas OT loss brings significant deterioration to this metric. Combining the two losses results in further
8

---

Published in Transactions on Machine Learning Research (10/2024)
decrease and suggests the lack of positive interaction between MM and OT losses, contrary to the claims of Lei et al. This will be further investigated in section 4.2.1
Classification Performance: Table shows that full LICO obtains lower top-1 accuracy on ImageNet-Sso compared to model trained using only OT or only MM loss. should be noted that while in terms of top-5 accuracy LICO slightly outperforms a model trained with only MM loss or only OT loss, it still worse than the baseline. Overall, these results do not support the claims of LICO authors, who argue that using MM and OT losses jointly is beneficial to the result.
4.2 Results Beyond the Original Paper
4.2.1
Extended Evaluation Setup for Interpretability
For the extended quantitative analysis of LICO's interpretability, we follow an experimental setup that has been described in detail in section 3.4.1
Salience Equivariance Similarity: LICO trained on CIFAR-100 shows a small improvement over the baseline in terms of SES metric as seen in Table 3 Tt should be noted that CGC (Pillai et al. 2022) vastly outperforms both LICO and the baseline, which should be expected given that a similar objective is used during its training.
During the experiments on TmageNet-Sso,, the full LICO achieves lower SES than the baseline as shown in Table 4 Since the SES metric measures the robustness of post-hoc explanations produced for the model to different image transforms, this suggests that saliency maps for LICO models are less stable given basic geometric transformation of the input compared to the baseline ResNet-18 model. However, it should be noted that models trained using only one of the OT and MM losses slightly outperform the bascline, which shows the negative effect of combining two losses.
The results for the experiments on ImageNet-Sso in Table 4 show that the explanations generated for bascline models tend to be more localized and contained within the boundaries of the relevant objects compared to LICO models measured by SCH. Although the models trained in the ablation study on loss components perform worse than the baseline on SCH metric, they again outperform LICO models.
Segmentation Content Heatmap:
Multi Object Salience Uniformity: The results for MOSU in Table show that the baseline models are better at distributing the salience between all relevant instances of a considered class proportionally to their size compared to the LICO model. The full LICO also achieves worse results than the model trained using only MM or only OT loss.
As shown by these three metrics, the explanations produced for LICO-trained models are less consistent with human intuition compared to a ResNet-18 baseline as well as models trained using only one of the additional loss components. This further confirms that there is little support for Claim 1 regarding improved interpretability of LICO models and Claim 3 regarding the beneficial effect of using MM and OT losses jointly.
Table 4: Accuracy and interpretability of loss ablation on ResNet-18 with LICO on ImageNel-Sso
Accuracy
Interpretabilitty
Method
Top-1
Top-5
SES (t)
SCH (t)
MOSU (L)
CE loss (Basclinc)
.8218 + .0132
.9525 + .0045
.914 - .001
.459 + .002
.216 + .013
+ MM loss
8227 + .0049
.9468 + .0039
.916 + .002
.427 + .116
.233 + .013
+ OT loss
.8187 + .0035
.9494 + .0038
.917 + .001
.411 + .004
.229 + .032
+ MM & OT (LICO)
.8160 + .0027
.9503 + .0023
.901 + .002
.396 + .014
.253 + .024
4.2.2 LICO Extensions for Improved Interpretability
In our attempt to reproduce the experiments, we noticed that the text features for the context tokens encoded by CLIP were the same for all classes, which limits the chances of identifying meaningful features in aligned visual feature maps. This happens due to the CL.PP's text transformer encoding the text features based only
9

---

Published in Transactions on Machine Learning Research (10/2024)
the previous tokens by using masked self-attention (Vaswani et al 2017 In such setup, the context tokens cannot encode information useful to specific class as the encoder cannot attend to class labels that are appended after them. Identifying this as a potential limitation of LICO, we propose two extensions targeted at resolving this problem.
Cless-Specific Trainable Context: Using class-specific trainable context tokens preceding the class labels will allow contexts that only have to fit one specific class. As a result, better text-image alignment can be achieved for all classes. The experiments ImageNet-Sgo in Table 5 show that this improves both the classification performance and interpretability of the trained models over the vanilla LICO method.
Front Placement of Class Label: Since the text encoder of CLIP attends only to the previous tokens, we propose changing the position of class labels to the front of the prompt. This should allow for producing text features of the context tokens that would encode aware semantic information. Although this leads the deterioration in classification performance over LICO, there is a significant improvement in terms of model interpretability as seen in Table 5
Table 5: Accuracy and Interpretability Prompt Variants on ResNet-18 with LICO and ImageNet-Ssu
Accuracy
Interpretability
Method
Top-1
Top-5
SES
SCH (1)
MOSU (1)
LICO
.8160 + .0027
.9503 + .0023
.901
.002
.396 + .014
.253
.024
LICO + class-specific conlext
.8191
.9574
.908
.418
.219
LICO + class label at the front
.8045
.9441
.905
.419
.206
5
Discussion
In this work, we were unable to reproduce the claims made by Lei et al. (2023). We conducted several experiments, some of which replicated the original authors' evaluations, while others expanded upon the evaluation methods and the LICO model itself.
The first claim of enhanced interpretability was not reproduced. Qualitatively, produced saliency maps on ImageNet-Ssg tend to cover spurious features more often than for the baseline. For CIFAR-100, this even more the case. In some instances, LICO resulted in artifacts appearing around the borders of the saliency maps, which were not present in maps for bascline models. Quantitatively, LICO saliency maps result in insertion, deletion, and combined scores that are not significantly better than baseline scores.
Likewise, we did not observe significant improvements in classification performance compared to the baseline as stated the second claim we strove to reproduce. For both CIFAR-100 and ImageNct-Sso. LICO models result in lower or similar top-1 and lop-5 accuracy than the baseline model.
Finally, we were also unable to confirm the third claim regarding the necessity of both MM and OT loss, Ablating individual loss components did not lead to decrcase in either classification performance or model interpretability for any of the evaluations. In most cases, it rather showed improvement over the full LICO approach.
Our extended analysis highlighted additional areas where the performance of LICO may be limited. The saliency maps generated for LICO models appear to be less equivariant to translations and crops compared the baseline and also appear cover more of the regions outside the boundaries of the target objects. Moreover, the explanations for LICO are worse at highlighting all instances of the relevant class compared to the baseline model.
Furthermore, we observed improved accuracy and interpretability when incorporating class-specific trainable context and positioning class labels at the front compared to utilising original shared prompts. This suggests that more relevant and descriptive text. features may lead to improvements in the quality of the aligned feature maps.
Future work: We believe the idea of incorporating the text features into an image classifier is an interesting way of improving model interpretability. Tt. might be able to hint at human understanding of the image,
10

---

Published in Transactions on Machine Learning Research (10/2024)
especially if longer descriptions used. Thus, we propose to explore using image captions to provide more relevant signals for aligning image features.
Limitations: Our results may be limited by several factors. Computational constraints prevented us from conducting experiments using the ImageNetlk dataset, on which the authors claim their method performs well. That. being said, LICO was also reported to perform well on CTFAR-100, and this did not turn out to be the case in our reproduction. The differences in results might have also arisen due to the potential use of regularisation The question regarding that part of our attempted communication with the authors; however, we had to implement the method without their response.
normalization methods not mentioned in the paper during the original experiments.
5.1 What was Easy
From a technical perspective, most of the content of the paper was approachable. The explanations in the paper are quite clear, except for few instances - for example, the cost. function e in eq- (4) not. being defined. Hyperparameters were included where relevant, which made it quite straightforward to set up the experiments replicating the authors' claims. Moreover, in addition to working on ImageNet-1k:, Lei et al.] also conducted experiments on smaller datasets like CIFAR-100, which allows for attempts at reproduction even with a limited computational budget.
5.2 What was Difficult
The most time-consuming part of our reproduction study was the full re-implementation of LICO based the paper's description, as the provided codebase partial. At the time of writing, the code on the LICO repository contains only the implementation of Sinkhorn distance used in OT loss and the partial code for the calculation of MM loss. However, the latter deviates from the text of the paper the main difference being the calculation of softmax on distances rather than negated distances as described the paper. This choice was not justified by comments the codebase. There is also no code with a complete implementation of the training loop or any of the conducted experiments.
5.3 Communication with Original Authors
As described in the previous sections, the official LICO code published by Lei et al. is not sufficient to reproduce the paper without reimplementing the majorily of the method. Moreover, it should be noted that there was no code publicly available in the repository before the 12th January 2024, despite the paper having been published in the autumn of 2023.
To clarify certain implementation details not fully specified in the paper or available code, we attempted to contact the authors via email and GitHub. As we did not receive response during our study period, implementation decisions were based on our interpretation of the published paper and referenced literature.
References
Monya Baker. 1,500 scientists lift the lid on reproducibility. Nature, 533:452 454, 2016. URL https:
//api. //api.semanticscholaar ottp://coo....oooooooopoooaooood
Junyi Chai, Hao Zeng, Anming Li, and Eric W.T. Ngai. Deep learning in computer vision: A critical review
of emerging techniques and application scenarios. Machine Learning with Applications, 6:100134, 2021. ISSN 2666-8270. doi: http:///bbioo///...00///... cct/://ccaac/rrrraaa//////666660
100.100000/.mm12222200111144 URL https: / http://wwww..cieneeiree....
William Falcon and The Torch Lightning team. PyTorc Lightning, March 2019. URL https://github. ConnLLigttning-A//ighhtnii
Shanghua Gao, Zhong- Yu Li, Ming-Hsuan Yang, Ming-Ming Cheng, Junwei Han, and Philip Torr. Large-
scale unsupervised semantic segmentation. 2022. URL https://gittub.. hotp:///////SSSS/////eeeett
11

---

Published in Transactions on Machine Learning Research (10/2024)
Tao Han, Wei- Wei Tu, and Yu-Feng Li. Explanation consistency training: Facilitating consistency- semi-supervised learning with interpretability. Proceedings of the AAAT Conference on Artificial Intelli- gence, 35(9):7633 7639 7646, May 2021. doi: 10.1609/ aaai.v3559..6990 URL https://ojs. aaap::://ojaaaii.......ma.aaaaaa oooooorggg.../eeeooooeoeoooe.o/e
José Armando Hernández and Miguel Colom. Repcatability, reproducibility, replicability, reusability (4r) in journals' policies and soltware/das management in scientific publications: A survey, discussion, and perspectives. arXie preprint orXiu:2312.1102 2023.
Lisanne Koning, Johan van Riel, and Leon Sebek. Enteric methane emission of the dutch dairy herd: Average and variation of enteric methane emission among the dutch dairy herd. Technical report, Wageningen Livestock Research, 2020.
Alex Krizhevsky, Geoffrey Hinton, al. Learning multiple layers of features from tiny images. 2009.
Yiming Lei, Zilong Li, Yangyang Li, Junping Zhang, and Hongming Shan. LICO: Explainable Models with Language-Image Consistency, October 2023.
Nature Computational Science Editorial Board. Moving towards reproducible machine learning. Nature Compulational Science, 1:629 630, 2021. doi: 10. 10.10033/445588--22-0001222 URL https: h/wwr/.
nature.
Nowtricity. Nowtricity.
2024. Accessed: 2024-02-02.
Vitali Petsiuk, Abir Das, and Kate Saenko. RISE: Randomized Input Sampling for Explanation of Black-box Models, September 2018.
Vipin Pillai and Hamed Pirsiavash. Explainable models with consistent interpretations. Proceedings of the
AAAI Conference Artificial Intelligence, 35(3):2431 May 2021. doi: 10.1609/aaai 135....13333.13344
URL https://ojs.aat aaai. www.aaaaaaaaaaaaaaaa.a.aa wwtgg/indx....e http:p///A..aaatiice/ee/e///////
Vipin Pillai, Soroush Abbasi Koohpayegani, Ashley Ouligian, Dennis Fong, and Hamed Pirsiavash. Consis- tent Explanations by Contrastive Learning, April 2022.
Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, Gretchen Krueger, and Ilya Sutskever. Learning Transferable Visual Models From Natural Language Supervision, February 2021.
Ramprasaath R. Selvaraju, Michael Cogswell, Abhishek Das, Ramakrishna Vedantam, Devi Parikh, and Dhruv Batra. Grad-cam: Visual explanations from deep networks via bradient-based localization. International Journal of Computer Vision, 128(2):336 359, October 2019. ISSN 1573-1405. doi: URL http:::/aaa..oo..oo//0000///////
10.1007/s11263--19900222
Harald Semmelrock, Simone Kopeinik, Dieter Theiler, Tony Ross-Hellauer, and Dominik Kowald. Repro- ducibility in machine learning-drivee research. arXie preprint arXtv:2307.10322 2023.
Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N. Gomez, Lukasz Kaiser, and Illia Polosukhin. Attention is all you need. CoRR, abs/1706.03762, 2017. URL http:
//arxiv.org/abell argg/bbs/17// 2337
12

---

Published in Transactions on Machine Learning Research (10/2024)
A Adversarial Dataset
In order to demonstrate that delction and insertion metrics favor non-robust models, we construct the Adversarial dataset based on ImageNet-Sgo- The only difference from the base dataset is that all images for one of the classes in a dataset are marked with a small red dot of fixed size at the top left corner. With this, we aim to induce non-robustness in the model, as it is likely to start using the red dot as the sole predictor of the modified class, rather than the actual object in the picture.
We train ResNet-18 with CE loss from scratch on this dataset and observe that high importance is assigned to the location of the red dot for the class that has it. The overall deletion score is 0.129, which is comparable to models trained on ImageNet-Ssu Mcanwhile, deletion specifically for the class with the red dot is 0.026.
For the deletion metric smaller is claimed to be better, so this experiment shows that the deletion metric favors non-robust models that focus on relatively small, but perhaps irrelevant, details. thus should not be used to assess the explainability potential of an image classification model.
Data Preprocessing
During our training process; we use minimal data augmentations. Namely, RandomResizedCrop and zanddmiior1zontalFiip for the training sct, following the available code of Pillai et al. (2022) and no augmentation for the validation and lesting sets. For the Adversarial dataset (see appendix A/, we remove both augmentations to ensure the red dot is always visible and always at the same spot. Instead, we just resize the images to (224, 224).
C Ensuring Reproducibility
To ensure reproducibility, make all of our experiments deterministic by setting the random seed. So that no cherry-picking is involved, we fix the seeds to be: 1, 2, and 3. Due to computational constraints, we trained only the most important models on ImageNet50 dataset on all three seeds the remaining models were trained on soed 2.
D Environmental Impact
The GPUs used for the training, RTX 4090 and Tesla A100, consume about 200W during training. To that, we can add another 200W for the CPU, cooling, etc. One training run of LICO on CIFAR-100 or ImageNet-Ssp takes approximately 10000 seconds our case. During the reproduction efforts, we estimate that an equivalent of 100 full runs were performed.
Therefore, 0.4kW 10000s 100 = 3600s/h 111kWh of power consumed. According to the online resource Nowtricity] in the Netherlands, an equivalent of 421 grams of CO2 is emitted per kWh produced. The reproduction study thus contributed an equivalent of 111-421 = 46.7 kilograms of CO2 this is roughly equal to the COg-equivalent emissions of 3 livestock cows in the Netherlands within a single day (Koning
et al. 2020
13


Reviews:
[{"review_id": "T8Xc0enfHn", "reviewer": "Reviewer_bc35", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strengths:\n* The paper identifies some major issues with the reproducibility of results. Albeit, it must be pointed out that the implementation by the authors differs in small ways (for instance, the temperature parameter of the Manifold matching loss). In particular, the non-reproducibility of the deletion/insertion scores for the ResNet-18 + CIFAR-100 combination is a key indicator that the LICO method may not be universally applicable. It is noteworthy that the original LICO paper does not disclose the top-1/top-5 accuracy scores for ResNet18+CIFAR-100 at all.\n* The ImageNet-$S_{50}$ result is also telling, indicating the lack of generalizability of the LICO approach to a different architecture + dataset combination. \n* The paper is well-written and points to some important considerations for the measurement of saliency map efficacy for explanations.\n\nWeaknesses:\nThere are at least three important objections to the conclusions stated in this paper:\n1. The non-applicability of the deletion/insertion score on the pretext that it is applied only to a model+salience-map-algo pair when the model is frozen (whereas, in LICO, the visual encoder is trained), and that the metric favors small regions in the image.  Both assertions may be untrue. In the first case, while the LICO encoder is indeed trained to produce better saliency maps, it is not explicitly trained to do so and so, any post-hoc explanation mechanism and metric may be employed at inference time. Secondly, the authors provide evidence for the latter assertion through what they term an adversarial training dataset where *all* images of a class are augmented with a red dot and the saliency maps indicate as such. The catch here is that this is exactly the expected behavior for any saliency map generation method and metric. Instead, for example, if the model was actually adversarially attacked via a low-\\epsilon attack and then the salience map was found to favor the adversarial regions, it may have indicated an incorrect bias in favor of non-robust models.\n2. All the stated results are given for ResNet-18, and not ResNet-50 which is the main network used by LICO for most of its experiments. While this does not absolve the original paper from signaling an incomplete conclusion, the results may still be valid for models larger than a certain size. A paper refuting its claims must test LICO with both smaller and larger networks. \n3. The proposed alternative metrics, while intuitively appropriate, do not have backing evidence. For instance, the premise of the MOSU metric is that to classify an image with multiple instances of the target class, each of those instances must be attributed is an unfounded one with no supplied evidence (citations, etc.) for itself. The paper is also missing the element of human judgment for the efficacy of these new metrics.\nIn addition, the premise that the attribution for a particular classification must be limited to the boundaries of the object is not substantiated with empirical evidence or prior art. In some cases, the classification may be inferred from the surrounding context, especially if significant portions of the object in question are occluded.", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "The authors aim to test the reproducibility of another work (LICO) which claims to produce a more interpretable image classification model. LICO claims to achieve better interpretability and performance using different training losses to align the image features with the corresponding class labels encoded as text features obtained from a pre-trained vision-language model. The authors of this paper refute the claims of improved explainability and performance on image classification tasks. They also point to the shortcomings of the training approach suggested in LICO and propose improvements to it. \n\nThe chief observations in support of their view made by the authors include:\n1. the incorrect use of the deletion/insertion metrics in the context of the formulation of LICO. They claim that the deletion/insertion metrics favor non-robust models and cite this as a reason for the inappropriate use in this context. They also suggest using three other metrics - the Salience Equivariance Similarity (SES), the Segmentation Content Heatmap (SCH), and the Multi-object Salience Uniformity (MOSU) to measure the accuracy of explanations. The authors propose the MOSU metric to measure the attribution of the classification result to all instances of the class in the image.\n2. the non-reproducibility of the explainability and performance metrics reported in the original paper, for the CIFAR-100/ResNet-18 combination.\n3. lack of generalization of the claims to other datasets, notably ImageNet-$S_{50}$ - both quantitatively and qualitatively. \nThe authors also point to notable omissions such as the normalization scheme used.\n\nIn addition to all the results that refute the claims made in LICO, the authors also discuss two simple extensions to improve the salience maps - one through additional learnable parameters by way of class-specific learnable context tokens (as opposed to identical trainable tokens used in LICO), and second, by placing the class label in front of the context tokens so as to influence the latters' encoding through the causal attention mechanics which the encoding transformer typically employs.", "claims_and_evidence_yn": "Yes", "claims_explanation": "", "audience_yn": "Yes", "audience_explanation": "", "requested_changes": "The authors are requested to revisit their premise for the rejection of the deletion/insertion score methods, as well as to state their reasons for testing their position only for ResNet-18. They are also requested to furnish evidence of the efficacy of the alternative metrics for measuring the salience map explainability.", "broader_impact_concerns": "The paper could be a bit less adversarial (for example, in pointing out omissions). This would encourage the original papers' authors to approach the critique more constructively.", "additional_comments": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "NMigXgxW1r", "reviewer": "Reviewer_Th9w", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "## Strengths\n\n1. The authors make a commendable effort in trying to reproduce the work of the original paper, even more considering that the original authors never replied to their inquiries. They create a new repository where they upload both the code and the weights of the models used in the experiments that they conduct. The experiments conducted are not exactly the same as the ones presented in the original paper, due to limitations in hardware (e.g.: they didn’t present results on ImageNet-1k), but they try to make them as comparable as possible (e.g.: presenting ResNet-18 results on CIFAR-100).\n\n2. The additional evaluation metrics used to assess explainability. Namely, they use *Salience Equivariance Similarity (SES)*, Segmentation *Content Heatmap (SCH)* and the *Multi Object Salience Uniformity (MOSU)* in addition to the two metrics used in the original paper: *insertion* and *deletion*.\n\n3. The authors present two very interesting extensions to the way the LICO model uses the text encoder from CLIP. In particular, the authors:\n    - Use class-specific context tokens (the original CLIP uses the same tokens for every class).\n    - Position the label token at the beginning of the prompt so that it is also encoded (the original CLIP positions the label at the end).\n\n    These two avenues might warrant further exploration on their own.\n\n4. The environmental impact analysis is a very nice touch that more papers should show.\n\n## Weaknesses:\n\n1. The experiments conducted do not completely and exactly replicate the experiments of the original paper. Some of this is unavoidable as the original authors didn’t disclose certain information: e.g. whether to use L2 normalization or not on visual and text features. However, additional experiments could have been conducted. For example:\n    - Table 3 should present results for ResNet-50. \n    - Also, authors didn’t experiment with ImageNet-1K: while the authors justified it as not having enough computational resources, this might be important given the significance of the claims (that no result could be reproduced).\n\n2. Related to the previous point, the experiments conducted in this study used 16-bit precision. The original paper does not specify the precision used, but if they employed 32 or 64-bit precision, that could explain the difference in results and could discredit this reproducibility study completely.\n\n3. Some parts of the paper are not clear enough and require the reader to consult some references attentively. Specifically, section 3.1 where the LICO algorithm is described and section 3.4.1 where insertion and deletion metrics are presented could benefit from a clearer explanation and would help the paper be self-contained.", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "This paper aims to reproduce the algorithm implemented in another paper called “LICO: Explainable Models with Language-Image COnsistency”. In the absence of fully reproducible open-source code, the authors reimplement the model presented in the original paper. Based on experiments on CIFAR-100 and a subsample of ImageNet-S, this study concludes that the main claims of the original paper could not be reproduced.\n\nMore specifically, the original paper presents a new training framework (LICO) that generates saliency maps that would improve both the explainability and the accuracy of image classification models by adding two new components to the Cross Entropy loss function (Manifold Matching and Optimal Transport Loss). The authors of this study couldn’t reproduce the improvement in explainability nor accuracy, and couldn’t reproduce either the effectiveness of the new two components added to the loss function.", "claims_and_evidence_yn": "Yes", "claims_explanation": "", "audience_yn": "Yes", "audience_explanation": "", "requested_changes": "Following is specific feedback per page, denoted as p:\n\n## Critical changes\n\n- **Change 1 - p1 in Abstract**: *“The main claims are that LICO…”*. It is confusing whether the claims that are going to be introduced are the ones from this reproducibility study or from the original LICO paper. After reading the paper, it becomes clear that these are claims from the LICO paper, but on a first read this is not obvious. Adding something like *“from the original paper”* or similar could help.\n\n- **Change 2 - p3**: *“incorporates two losses in addition to the standard Cross-Entropy”*. It is not stated how they are incorporated which is important to understand the original paper.\n\n- **Change 3 - p3**: *“The image manifold is defined by feature maps fθ(xi) from the classifier…”*. What classifier? As stated in the weaknesses, this paragraph and the overall section might benefit from clarification.\n\n- **Change 4 - p4**: *“parameters are also used unchanged: α = 10, β = 1”*. Related to change 2, these parameters are presented without context. The final loss function should be presented or at least explained. It is not explained anywhere that the 3 losses, cross entropy - CE, Manifold Matching - MM and Optimal Transport - OT, are added and that α and β are used to control MM and OT.\n\n- **Change 5 - p4**: *“Our code diverges from the formulas by Lei et al. in the way the temperature parameter of the MM loss is implemented. Inspired by the implementation of CLIP (Radford et al., 2021), we multiply by the exponent of the trainable value instead”*. If your code diverges from the original paper, how is the comparison acceptable? If this changes won’t affect the reproducibility, please explain why.\n\n- **Change 6 - p4**: *“share unblurred”*. This expression is confusing. The original paper calls this \"pixels removed\" which is much clearer. As stated in weaknesses, this whole section where insertion and deletion are explained, could be improved as it is confusing for such simple concepts.\n\n- **Change 7 - p5**: *“The LICO method aims to change the model itself in order to make it more interpretable. Therefore, the comparison between the insertion and deletion scores of a baseline model and a model trained with LICO is not sufficient to claim that one model is more interpretable than the other.”*. This is a strong claim that might need more justification. The justification from Appendix A seems not satisfactory.\n\n- **Change 8 - p6**: *“we select smaller datasets”*. Do you mean smaller batch sizes? If you actually mean datasets, how is the comparison fair?\n\n- **Change 9 - p7**: *“Experiments on ImageNet-S50 using Insertion and Deletion metrics in Table 2 show that training with MM loss achieves a Combined score that is better compared to a ResNet-18 baseline”*. Is this true? In table 3, the best numbers are bolded in the first row, which means that the rest are worse. How is using MM loss better than using ResNet-18, which is the baseline?\n\n- **Change 10 - p7**: *“meaning that the post-hoc explanations produced for LICO models are less robust to image transforms”* How is this conclusion reached?\n\n- **Change 11 - p9**: *“MM loss results in a tiny improvement.”*. Same as Change 9.\n\n- **Change 12 - p12**: *“as it is likely to start using the red dot as the sole predictor of the modified class, rather than the actual object in the picture. We train a ResNet-18 with CE loss from scratch on this dataset and observe that high importance is assigned to the location of the red dot for the class that has it.”*. How is this proven? This point is important because it justifies one of the main claims of the paper: that insertion and deletion metrics are not good enough.\n- **Change 13 - p12**: *“no augmentation for the validation and training sets”*. Should it be test sets instead of training?\n\n## Changes that would strengthen the work\n\n- **Change 14 - p4**: *“There is no ReadMe file”*. At the time of this review, there is a README. However, it is still not useful to replicate the model.\n\n- **Change 15 - p5**: *“Where v_salience ∈ Rn denotes”*. *V_salience* could be accompanied by a formula like *v_area*. Also, explain what *i* is, in that case.\n\n- **Change 16 - p7**: *“Figure 2”*. The original images can’t be distinguished in the Figure so I can’t tell if the saliency maps are accurate or not. Either the original images should be shown or the transparency should be higher so that the original image could be distinguished.", "broader_impact_concerns": "None.", "additional_comments": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "Phgtd3VT3s", "reviewer": "Reviewer_nqwA", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strengths:\n\nThis paper is very well written overall. It provides context for the LICO framework and claims made by Lei et al. before discussing the experimental setup and results. The majority of the paper was also grammatically sound with good readability. Additionally, the analysis performed by the authors was thorough and well thought out. \n\nWeaknesses:\n\nMy biggest concern of this submission is that all the reimplementation details have not been confirmed by the authors of LICO, which makes me difficult to determine if all the results in the experiments are correct and convincing. I have noticed that the authors of the submission discussed the bad communication with the authors of LICO, but this should not be the reason that I have to trust all the results and claims of the submission. Also, from the limitations in the conclusion section, it seems that the reimplementation cannot reproduce some results in LICO paper, such as on ImageNet, and the authors of the submission are not clear about some implementation details of LICO as well. These discussions make all the findings in the submission much less convincing to me.\n\nAlso, there are some minor concerns as follows: A weaker element is evident under claim 3 in section 4.1, where claims referencing tables do not align with the data presented in the table. For example, in the interpretability subsection of claim 3, it states that “Table 2 show that training with MM loss achieves a Combined score that is better compared to a ResNet-18 baseline…”. However, table 2 does not appear to support this claim since the combined loss under the “+ MM loss” row is lower than that of the baseline. Similarly, in the following subsection, it states, “It should be noted that while in terms of top-5 accuracy LICO slightly outperforms a model trained with only MM loss, it is surpassed by using OT loss.” Unless it was meant that the model using only OT loss surpassed the model using only MM loss, this statement does not align with what is shown in the table. These sections should be reviewed to ensure the paper is strong in its entirety.", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "No", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "The authors of this paper investigate the reproducibility of the Language-Image COnsistency (LICO) training framework first introduced in the 2023 paper “LICO: Explainable Models with Language-Image Consistency” by Lei et al. In their investigation, they found that the LICO code was incomplete and largely undocumented, which prompted the authors of this paper to develop and document their own implementation of the LICO framework. When reproducing the experiments in the LICO paper, the authors found that the claims made by Lei et al. were inaccurate. Specifically, Lei et al. claimed that the LICO training framework provided enhanced model interpretability, improved classification performance, and that the combination of two loss components, the Manifold Matching (MM) and Optimal Transport (OT) loss, improved the quality of the model. The authors showed evidence suggesting that these claims may not have been as accurate as suggested by Lei et al. To address these discrepancies, the authors not only provide suggestions to improve the LICO framework but also implement these suggestions in their experiments. The authors also extend the evaluation methodology of LICO by using the Salience Equivariance Similarity and Segmentation Content Heatmap metrics. Furthermore, they present a novel metric to represent the model's ability to uniformly highlight features from multiple instances of an object, known as Multiple Object Salience Uniformity (MOSU).", "claims_and_evidence_yn": "No", "claims_explanation": "", "audience_yn": "Yes", "audience_explanation": "", "requested_changes": "I know that having a good connection with the authors of LICO may be problematic, but given current situation I feel that this seems necessary to make the submission more convincing, no matter how the submission is modified later. Alternatively, if the authors of the submission can reproduce all the results in LICO, these can be considered strong evidence for the correctness of the reimplementation, on top of which, the authors can discuss more.\n\nAlso, there are some minor changes that can enhance this submission. I think that adding the original images used in figure 2. alongside the saliency maps would allow readers to more easily see where the model’s attention is being placed. Additionally, I believe there should be a “the” before “ablation study” in the “Segmentation Content Heatmap” subsection of section 4.2.1. Specifically, in the last sentence, where it states, “Although the models trained in ablation study on loss components…”.", "broader_impact_concerns": "N/A", "additional_comments": "", "extra_scores": {}, "extra_text": {}}]

Author rebuttal:

Meta-review: Taking the stance that we are primarily evaluating correctness, this paper should be accepted (with slightly softened language). However, if were evaluating "impact", then I would give this paper a reject. The original work is just not of sufficiently well known to expect broad audience for a paper that attempts a reproduction.
Decision: Accept (poster)