Reproducibility Study of ’SLICE: Stabilized LIME for Consistent Explanations for Image Classification’
Authors: Aritra Bandyopadhyay
Chiranjeev Bindra
Roan van Blanken
Arijit Ghosh
Venue: Accepted by TMLR ()

Abstract: This paper presents a reproducibility study of SLICE: Stabilized LIME for Consistent Explanations for Image Classification by Bora et al. (2024). SLICE enhances LIME by incorporating Sign Entropy-based Feature Elimination (SEFE) to remove unstable superpixels and an adaptive perturbation strategy using Gaussian blur to improve consistency in feature importance rankings. The original work claims that SLICE significantly improves explanation stability and fidelity. Our study systematically verifies these claims through extensive experimentation using the Oxford-IIIT Pets, PASCAL VOC, and MS COCO datasets. Our results confirm that SLICE achieves higher consistency than LIME, supporting its ability to reduce instability. However, our fidelity analysis challenges the claim of superior performance, as LIME often achieves higher Ground Truth Overlap (GTO) scores, indicating stronger alignment with object segmentations. To further investigate fidelity, we introduce an alternative AOPC evaluation to ensure a fair comparison across methods. Additionally, we propose GRID-LIME, a structured grid-based alternative to LIME, which improves stability while maintaining computational efficiency. Our findings highlight trade-offs in post-hoc explainability methods and emphasize the need for fairer fidelity evaluations. Our implementation is publicly available at our GitHub repository.

Published in Transactions on Machine Learning Research (05/2025)
Reproducibility Study of 'SLICE: Stabilized LIME for Consistent Explanations for Image Classification'
Aritra Bandyopadhyay University of Amsterdam Informatics Institute
aritra. oxker.bbnigggoddgggggggeeddd....
Chiranjeev Bindra
chiranjeev. @stuudnntttdetttuuuu uvo.
University of Amsterdam Informatics Institute
Roan van Blanken
roen. www..vww.blankeen.oooomm www.voww..nnkkennsssudddn.......
University of Amstendam Informatics Institute
Arijit Ghosh
orijt.ghossh@sttueed..o..r
Universily of Amslerdam Informatics Institute
Reviewed on OpenReview: https: // openreview. net/ forun? id= 4444
Abstract
This paper presents a reprodueibility study of SLICE: Stabilized LIME for Consistent Ex- planations for Image Classífication by Bora et al. (2024). SLICE enhances LIME by incor- porating Sign Entropy-based Feature Flimination (SEFE) to remove unstable superpixels and an adaptive perturbation strategy using Gaussian blur to improve consistency in feature importance rankings. The original work claims that SLICE significantly improves explana- tion stability and fidelity. Our study systematically verifies these claims through extensive experimentation using the Oxford-IIIT Pets, PASCAL VOC, and MS COCO datasets. Our results confirm that SLICE achieves higher consistency than LIME, supporting its ability to reduce instability. However, our fidelity analysis challenges the claim superior per- formance, as LIME often achieves higher Ground Truth Overlap (GTO) scores, indicating stronger aligmment with object segmentations. To further investigate fidelity, we introduce alternative AOPC evaluation to ensure a fair comparison across methods. Addition- ally, we propose GRID-LIME, structured grid- based alternative to LIME, which improves stability while maintaining computational efficiency. Our findings highlight tradc-offs in post-hoc explainability methods and emphasize the need for fairer fidelity evaluations. Our implementation publicly available at our GitHub repository
1 Introduction
Explainability in AI is crucial as deep learning models grow increasingly complex Grobrügge el al. (024 yel interpreting their non-transparent decisions remains challenging. Post-hoe methods like Local Interpretable Model-agnostic Explanations (LIME) Ribeiro et al. 2016) address this by approximating a model's local behavior with simpler surrogates. LIME generates perturbed samples around an input and fits a linear model, but while locally faithful, explanations often suffer from inconsistency across Gosieesaa
Biecek 2019 Zafar & Khan 2019 Zhao et al. 2021 Zhou al. 2021).
Stabilized LIME for Consistent Explanations (SLICE) BBrra et al. 2024 aims resolve LIME's instability using adaptive Gaussian blurring and sign entropy-based feature el elmination (SEFE). This reproducibility

---

Published in Transactions on Machine Learning Research (05/2025)
study validates SLICE's claims regarding improved consistency and fidelity by replicating its core experiments using the Oxford-III Pets, PASCAL. VOC, and MS COCO datasets. Our implementation available at [T]
Beyond verifying SLICE's claims, this study investigates limitations and proposes alternatives. Observing LIME's instability stems partly from segmentation randomness and SLICE's significant computational cost, we introduce GRID-LIME. employs structured grid segmentation, optimized using model predictions, enhance stability while maintaining efficiency closer to LIME. Furthermore, recognizing that perturbation- based fidelity metrics like AOPC can be biased by the perturbation stralegy (e.g., SLICE's blur vs. LIME's masking), we introduce the Ground Truth Overlap (GTO) metric. GTO directly compares explanations to ground-truth segmentations, offering a potentially less biased fidelity assessment. Our findings confirm SLICE's improved consistency but challenge its superior fidelity claims, particularly when considering GTO scores and computational cost, highlighting inherent trade-offs in post-hoc explainability. This work details the reproduction of SLICE's claims, introduces and evaluates GRID-LIME and GTO, analyzes SLICE's components, and extends evaluations across multiple datasets.
2 Scope of reproducibility
To address the stability issues in LIME-based explanations, Bora et al. (2024) proposed SLICE, a method that enhances LIME by incorporating novel feature elimination strategy and an adaptive perturbation technique. Specifically, SLICE utilizes Sign Entropy-based Feature Elimination (SEFE) to remove superpixels with high sign entropy, ensuring greater consistency in explanations. Additionally, SLICE introduces an adaptive Gaussian Blur mechanism that dynamically selects the optimal perturbation level, reducing variance in feature importance rankings and improving the overall reliability of explanations.
The focus of our work is on reproducing the following claims made by Bora al. (2024):
Claim 1: LIME suffers from inconsistencies in its explanations, primarily due to variance in sign and importance ranking of superpixels. These inconsistencies stem from the perturbation method used by LIME.
The Sign Entropy-bass based Feature Elimination (SEFE) method improves stability in ex- planations by identifying and eliminating spurious superpixels.
Claim 2:
Claim 3: The introduction of Gaussian Blur with an adaptively selected hyperparameter enhances the perturbation method, leading to greater consistency in explanations.
Claim 4:
Explanations generated by SLICE demonstrate significantly better fidelity compared to those produced by LIME.
3 Post-Hoc Explainability Methods
Post-hoc explainability methods aim to interpret complex AT models without requiring access to their internal structures. Local Intorpretable Model-agnostic Explanations (LIME) (Ribciro et al. 2016) is a widely used method that approximates a model's decision function locally by generating perturbed samples around a given instance and fitting a linear surrogate model. However, LIME suffers from inconsistencies across different runs, leading to instability in feature importance rankings and sign variations.
Several extensions have been proposed to improve the stability of LIME. DLIME (Rashid el al. 2024 employs hierarchical clustering to select perturbed samples from the cluster nearest to the instance of interest, ensuring greater locality adherence. S-LIME (Shi et al., 2020 leverages a hypothesis testing framework based on the Central Limit Theorem to determine the number of samples required for stable explanations. ALIME (Knab al., (2225) int roduces an nutoencoder-basee weighting function to improve coefficient stability by refining sample selection. BayLIME (Dehghani al. (2024) formulates explanations as a Bayesian-weighted sum of prior knowledge and new observations, reducing variance in feature importance.
https://github www.:/ww/.///e..eee....oooo htt.pp///llllo.rppproroolllllll/
2

---

Published in Transactions on Machine Learning Research (05/2025)
More recent methods focus on enhancing LIME's applicability to image-based models. Beyond Pix- els (Knab et al. 2025) integrates hierarchical feature representations with segmentation foundation models to improve visual explanations. SMILE (Dchghani et al. 2024) employs statistical distances based on em- pirical cumulative distribution functions to enhance interpretability in instruction based image editing tasks. These advancements contribute to improving LIME's robustness across diverse applications.
As our study primarily investigates post-hoc explanations for image classification, focus evaluating SLICE (Bora et al. 2024), which introduces an adaplive Gaussian perturbation method and a feature elimination strategy to improve stability. For SLICE and LIME, we provide a comprehensive assessment of its effectivenoss in producing consistent and reliable explanations.
4 Methodology
The objective of this reproducibility study is to verify the stability and consistency of SLICE, which uses Adaptive Gaussian Blur and Sign Entropy based Feature Elimination (SEFE). For re-implementing the code in PyTorch, we referred to the SLICE code that was made publicly available by the authors at ?
It should be noted that the author's code is not referenced directly in the original paper. However, since the link to the author's code was provided us by the authors, we decided to use this implementation as our reference. Furthermore, we added intermediate images at every step of the process, making the interpretation pipcline more transparent. We also introduced new interpretability metric to enhance the usability of SLICE. The code for this reproducibility study is publicly available on Github 3
4.1
Model descriptions
4.1.1 LIME
Local Interpretable Model-agnostic Explanations (LIME) is widely used post-hoc interpretability method that approximates the decision boundary of black-box models using a local surrogate model. LIME operates through three key steps: perturbation, weighting, and surrogate model training. Given a black-box model f and an input instance I, LIME generates perturbed images 2 by masking superpixels. A weighting function na assigns importance to these perturbed samples based on their similarity the original input. Subsequently, a linear surrogate model g is trained on these perturbed samples z' to approximate f locally. Z is the set of all perturbed samples. The objective function for LIME is given by:
(() = Σ (=)
(1)
While LIME effectively provides locally faithful explanations, it suffers from instability in feature importance rankings and sign fluctuations across multiple runs, limiting its reliability.
4.1.2 SLICE
SLICE (Stabilized LIME for Consistent Explanations) is designed to mitigate the instability of LIME by introducing two key components: Adaptive Gaussian Blur and Sign Entropy-based Feature Flimination (SEFE). An overview of SLICE is illustrated in Figure
The first component, Adaptive Gaussian Blur, applies a range of perturbation strengths (a values) to su- perpixels using Gaussian blurring. Instead of replacing superpixels with a fixed value (as in LIME), SLICE adaptively selects an optimal a value based on entropy, ensuring perturbations are closer to the original data distribution.
Exp.tttttappaaaannnnnnnn for- Deae-Image-aae MmageeCClassifccatioo/
tree/nsin https:/ http://ggtttttbco.e httptt/bwwwcccttaa/aaaeeeeerreee
3

---

Published in Transactions on Machine Learning Research (05/2025)
The second component, SEFE, eliminates superpixels that exhibit high sign entropy, meaning their attribu- tion flips between positive and negative across multiple iterations. This ensures greater stability in feature importance rankings and improves the consistency of explanations.
The SLICE algorithm iteratively generates N perturbed samples and trains M Ridge Regression models. SEFE identifies and removes unstable superpixels, while the final ranking of superpixels is based on the learned coefficients from the regression models. Training continues until predefined number of iterations stability tolerance threshold is reached.
Generate random perturbations for Blur
Train fidge rngression models on the perturbed images sipma (o) value
SEFE algorithm thrm using best sigma to superpboots
Adaptive
Blur
Sign Entropy Elimination
Train ridge regression models entropy all superpxets
Retur ranked positive negative superpixels
Train ridge regression model eech
Compute adjusted for each
the highest adjusted R
Generate perturtations using sigma
Elminate unstable superpisels
Figure 1: Overview of the SLICE framework. SLICE improves LIME's stability by incorporating Adaptive Gaussian Blur and Sign Entropy-based Feature Elimination (SEFE).
4.1.3 GRID-LIME
Quickshift generates superpixel segments based on the intensity values of the three color channels of the image, introducing variability across runs and potentially ignoring semantically relevant boundaries captured by the model. To address this and LIME's segmentation instability, we introduce GRID-LIME, This method replaces Quickshift with structured grid-based segmentation where the optimal grid size is determined using the black-box model's predictions, aiming to improve consistency without SLICE's computational overhead.
As shown in Figure 2 GRID-LIME evaluates a set of potential grid sizes, S 82.., 8k). For each grid size E S, the image divided into non-overlapping grid cells Cej-. Perturbed images H.s are generated by setting the pixels within each cell ex to zero (or another baseline value). Tet f(x) be the model's prediction probability for the target class on the original image and f(x..)) be the prediction on the perturbed image where cell C.jj is masked.
To select the optimal grid size, we analyze the sensitivity of the model's predictions to these grid-based perturbations. For each grid size we calculate the prediction differences A..j = [f(x) f(x.,)] for all cells j. We then compute the mean A4 and standard deviation a, of these differences {A.g}j. The Coefficient of Variation (CV) for grid size a calculated as:
Oz CV.
(2)

---

Published in Transactions on Machine Learning Research (05/2025)
The optimal grid size s" is chosen as the one maximizing the CV, indicating the grid resolution that yields the highest relative variability prediction changes upon perturbation:
argmax CV,
(3)
This process ensures a balance between segmentation granularity and stability, selecting a grid that is infor- mative according to the model's sensitivity. Once s* determined, GRID-LIME proceeds with the standard LIME framework (perturbation, weighting, surrogate model training) using the superpixels defined by the optimal grid s* This structured, model-informed segmentation helps reduce the randomness associated with LIME's original segmentation step. However, limitation remains: the rigid grid structure may not always align perfectly with natural object boundaries, potentially impacting interpretability for irregularly shaped objects compared to adaptive segmentation methods. The specific range of grid sizes tested in our implementation is detailed in Appendix
Select the gnd givna the maximum variation the in predictions on the perturaations
Train regression on the per perturbed images with Dusing the best grid box sice
Return ranied positiee negelive superpixets
Adaptive Grid Box
Generate peturaed images With a D
Get black-box model predictions for each perturbation
Compute the variation the mode predictions across each perturbation
Ratum the box the maximum variation
Divide image into grids using
Illustration of GRID-LIME. GRID-LIME replaces color-space saae--space-baeed based segmentation with grid- based approach to ensure stable explanations. Here, a 0 denotes that it uses the same perturbation strategy as LIME.
Figure 2:
4.2 Datasets
To evaluate the reproducibility of SLICE and compare ils performance against LIME, conduct experiments using three widely recognized image classification datasets: Oxford-IIIT Pet Dataset, PASCAL Visual Object Classes (VOC), and MS COCO. These datasets provide a diverse range of images, enabling a comprehensive assessment of explanation consistency and fidelity.
4.2.1 Oxford-I1IT Pet Dataset
The Oxford-11IT Pet dataset (Prkkh et al. 2012) consists of 37 categories of pet images, covering various broods of cats and dogs. Each category contains approximately 200 imagos, leading to a total of 7,349 images. The dataset includes pixel-wise trimap segmentation annotations, enabling precise localization of object regions within the images. This dataset particularly useful for evaluating explanation methods due its clear object boundaries and distinct features. In our study, we randomly sample 50 images from this dataset and apply LIME, SLICE, and GRID-LIME to analyze the consistency and fidelity of their explanations.

---

Published in Transactions on Machine Learning Research (05/2025)
4.2.2 PASCAL VOC 2009
The PASCAL Visual Object Classes (VOC) 2009 dataset [Everingham et al. 2010) a benchmark dataset for object recognition tasks. Tt contains a total of 7054 images in the training and validation set. The dataset includes 20 object categories, covering a broad range of natural and man-made objects, making it suitable for testing explainability methods across different object types. Each image is annotated with bounding boxes and pixel-wise segmentation masks. Given its complexity and variation in object scales, evaluate whether SLICE and GRID-LIME can improve the stability of LIME explanations. For our experiments, we randomly select 50 images from this dataset for comparative analysis.
4.2.3
MS COCO
The Microsoft Common Objects in Context (MS COCO) dataset (Lin et al. 2015 is one of the largest and most diverse datasets for image pon and segmentation tasks. It contains over 330,000 images, annotated with 80 object categories, instance-level segmentation masks, and dense captions. This dataset presents a greater challenge due to the presence of multiple objects per image, occlusions, and background clutter, making it ideal lestbed for evaluating robustness in explanation methods. We apply LIME, SLICE, and GRID-LIME to 50 randomly selected images from the validation set assess their performance in complex visual environments.
4.3 Hyperparameters
To ensure fair and reproducible comparison between SLICE, LIME, and GRID-LIME, we carefully tune hyperparameters based on prior research and controlled experiments. Each explainer algorithm (SLICE, LIME, or GRID-LIME) is executed for 10 iterations per image to generate the results presented in Section
For SLICE, we used the authors' choice of hyperparameters for all of the experiments. We train 1000 Ridge regression models to obtain model coefficients for all superpixels. Each Ridge model is trained on 500 randomly generated perturbations of the original test image. The Adaptive Gaussian Blur mechanism in SLICE requires selecting the optimal a value. We conduct a small-scale hyperparameter search, testing values in the range [0.1, with a step size of 0.1, and select the a value yiclding the highost adjusted R².
In the Sign Entropy-based Feature Elimination (SEFE) module, the tolerance limit set to 3, and the maximum number of iterations capped at 10 to balance computational efficiency and convergence.
GRID-LIME, our proposed variant, does not require a segmentation algorithm such as quickshift. Instead, it utiliiee structured grid-based segmentation, where we empirically determine the optimal grid size by evaluating perturbation consistency across different configurations.
For all experiments, we randomly sample 50 images per dataset and apply the same hyperparameters across SLICE, LIME, and GRID-LIME. to maintain consistency. The details of hyperparameter values used in our experiments are provided in Appendix C
4.4 Experimental setup and code
In this study, we conduct experiments to evaluate the performance of SLICE, LIME, and GRID-LIME in generating interpretable explanations. Our methodology and hyperparameter values follow the original paper, ensuring reprodueibility. Additionally, introduce auxiliary experiment to assess the alignment of saliency maps with ground-truth segmentation maps. For MS COCO and PASCAL VOC datasets, we sample those images that have only one segmentation object map present for evaluation using Ground Truth Overlap (GTO).
We use 5e (He et al., 2016 and InceptionV3 (Szegeyy et al. 2016 models, both pretrained on ImageNet as used in the original paper. For superpixel segmentation, we apply quickshift (Vedaldi & Soatto 2008) from the skimage library. SLICE iteratively perturbs images by blurring selected superpixels, fits local ridge regression models, and eliminates inconsistent superpixels until convergence. The Gaussian blur parameter a is optimized via small hyperparameter search.

---

Published in Transactions on Machine Learning Research (05/2025)
To evaluate model explanations, we use multiple metrics assessing consistency and fidelity, Consistency is measured through Average Sign Flip Entropy (ASFE), which quantifies sign stability across multiple runs, and Average Rank Similarity (ARS), which evaluates the consistency ranked importance scores. Wo combine these into the Combined Consistency Metric (CCM), a metric introduced by the original paper, providing a unified measure of explanation stability.
Fidelity is assessed using the Area Over Perturbation Curve (AOPC), which tracks classifier confidence changes important superpixels are perturbed, and the Area Under Curve (AUC), which caplures the cumulative impact of these perturbations. Higher AOPC and higher AUC values for insertion and higher AOPC and lower AUC values for delction of superpixels indicate that the explainer correctly identifies the most inftuential features.
As an additional metric, we introduce Ground Truth Overlap (GTO) to evaluate the alignment between model-generated saliency maps and ground-truth segmentation maps. Unlike previous metries relying on classifier confidence, GTO directly compares the predicted importance regions with known object boundaries. Figure illustrates the computation of the GTO metric and provides an example of its calculation.
Area of Ground Truth Area of top-k positive superpixels
GTO
(4)
Area of top-k positive superpixels
Lastly, our code is available on GitHubb A detailed analysis of the computational requirements and en- vironmental impact of our experiments, including runtime and estimated carbon emissions, is provided in A Appendix D
Original Image
Ground Truth area
Positive superpixels area
Ground Truth Overlap Area
Figure 3: Ground Truth Overlap (GTO) metric. The intersection between the top-k positive superpixels and the segmentation map is computed and normalized by the area of top-k positive superpixels. In this example, GTO = 0.58.
5 Results
We have performed extensive experimentation to test the claims made by Bora et al. (2024]- Our evaluation focuses on the interpretability, consistency, and fidelity improvements claimed for SLICE over LIME. We also introduce GRID-LIME as an alternative method and assess its effectiveness.
In this section, we provide a detailed analysis of the consistency and fidelity of explanations produced by these methods. We examine the impact of Sign Entropy-based Feature Elimination (SEFE) and Adaptive Gaussian Blur, and conduct an ablation study to determine their individual contributions. Additionally, we extend our analysis beyond the original paper by introducing the Ground Truth Overlap (GTO) metric to compare explanation maps with segmentation labols.
Overall, our results partially support the claims of Bora et al. (024]. While we confirm that SLICE produces significantly more consistent explanations than LIME, we find that its fidelity improvements are less conclusive. Additionally, we identify potential biases in the evaluation methodology, particularly in the
https://github. wtw.:/BXA////eea.eaec.co cttp///BBBA//.cccooooooocccelalt

---

Published in Transactions on Machine Learning Research (05/2025)
choice of perturbation strategies. Our findings suggest that while SLICE improves consistency, further work is required to ensure fair fidelity comparisons.
Furthermore, we provide additional qualitative examples in Appendix F illustrating the differences in expla- nation maps generated by LIME, SLICE, and GRID-LIME. These visualizations help to better understand the distinct attribution patterns produced by each method. In Appendix B we also present an alternative AOPC evaluation where SLICE is tested under the same perturbation conditions LIME and GRID-LIME, leading to more comparable fidelily results.
In the following subsections, we analyze each claim in detail and present supporting evidence.
5.1 Claim 1: LIME suffers from inconsistencies due to sign flip and rank variance
To assess explanation stability, we compare the Combined Consistency Metric (CCM) distributions across methods. Figure ! presents the Kernel Density Estimation (KDE) plots of CCM scores for SLICE, LIME, and GRID-LIME, across different datasets and models.
The results indicate that SLICE produces significantly more stable explanations than LIME, as evidenced by a strong peak near 1.0, meaning that sign flips and ranking inconsistencies are minimized. In contrast, LIME's CCM values are more widely spread, confirming instability. GRID-LIME exhibits a CCM distribution between that of LIME and SLICE, indicating improved stability while maintaining flexibility.
JMMS
UMC
UFC
SUICE
AL
8.8
COP
CDM
CON
OofordrETT Tetac Incept on/y2
Teacol IOC: Inceptianiis
M5-COCO Inceptiorly2
LPNG B.cs
UNG suace
37
LPL M.
2-
1-
CaP Scor
CCM Scome
con Scare
Figure& 4: Kernel Density Estimation (KDE) plots of Combined Consistency Metric (CCM) scores for SLICE, LIME, and GRID-LIME. A higher peak in CCM indicates greater explanation consistency, while a wider spread reflects higher variability.
5.2 Claim 2: SEFE improves stability in explanations
The impact of Sign Entropy- Feature Elimination (SEFE) is evaluated using the Combined Consistency Metric (CCM), which mcasures explanation stability. Figure 5 consists of six plots, cach containing three KDE curves representing different methods. Across these plots, SLICE consistently achieves the highest CCM, with values closest to 1, confirming that it provides the most stable explanations. SLICE BL.R follows with CCM around 0.9, indicating strong stability. In contrast, SLICE SEFE alone exhibits much lower stability, with a CCM ranging between 0.2 and 0.3. While SEFE reduces sign flips and ranking inconsistencies, its direct impact on stability is limited. The improvement in CCM is primarily observed
8

---

Published in Transactions on Machine Learning Research (05/2025)
when SEFE is combined with Adaptive Gaussian Blur, rather than when used in isolation. These results suggest that SEFE alone is insufficient for improving stability in explanations and is most effective when integrated with Adaptive Gaussian Blur
5.3 Claim 3: Adaptive Gaussian Blur enhances explanation consistency
To evaluate the effect of Adaptive Gaussian Blur, we compare the adjusted RC values of Ridge regression models trained with different a values. Our results show that dynamically selecting based entropy incrcases R² suggesting that perturbations remain realistic while improving consistency. However, the improvement is not always significant across all cases. This can also be observed in Figure 5 where Adaptive Gaussian Blur (AdaBlur) achieves relatively high CCM values, but the difference compared to SLICE without Adaptive Gaussian Blur is not always significant. These findings indicate that while Adaptive Gaussian Blur contributes to stability, its impact depends on the specific sctting and datasct.
5.3.1 Ablation Study
We conducted an ablation study to replicate the findings of (Bora al.) 2024), evaluating the individual contributions of SEFE and AdaBlur in SLICE. Since SLICE consists of these two key components, we performed the ablation study by isolating each module, shown in Figure
Our results align with the original ablation study in (Bora et al. 2024 SLICE achieves the highest con- sistency when both components are included, confirming their complementary roles. Among the individual modules, AdaBlur contributes more to consistency than SEFF. The absence of AdaBlur leads to perturbed images that deviate further from the original, reducing consistency. These findings highlight the importance of using both components logether for optimal slability.
M
MAMUU
3
CO
CCM
009
OnfondarT Retsincept oni/
Pascal VOC/inosston
MS COCO-Nccppioong
AUCC_TC A.cE
QUCCLE
AUCUJ
2-
26
06
COP Surro
CCM Scoea
CON Scorc
Figure 5: Distribution of CCM scores for ablation experiment. SLICE FE shows the results for the SEFE while SLICE Blur shows the results for the Adaptive Gaussian Blur component. SLICE shows the results for both components included. (higher is better).
5.4 Claim 4: SLICE provides better fidelity than LIME
To assess fidelity, use the Area Over Perturbation Curve (AOPC) and Area Under Curve (AUC) metrics. Figure 0 presents the ECDF plots of AOPC scores. We performed the Most Relevant First (MoRF) procedure
9

---

Published in Transactions on Machine Learning Research (05/2025)
similar to the author's where insertion refers to the insertion of most positive superpixels and deletion refers to the deletion of the most negative superpixels.
In the insertion task (first row of Figure G SLICE consistently achieves higher AOPC scores than LIME and GRID-LIME, demonstrating better preservation of important features. However, in the deletion task (second row), SLICE performs worse than both methods, suggesting its deletion strategy is less effective.
Figure 7 presents ECDF plots of AUC scores, where SLICE outperforms the other methods in both insertion (first. row) and deletion (second row). A higher AUC value is preferable for insertion, while a lower AUC value is better for deletion, confirming SLICE's strong performance in both tasks. To further evaluate whether SLICE significantly differs from LIME and GRID-LIME across datasets we performed a Wilcoxon rank test on AOPC and AUC (Appendix Ap.
One potential reason for SLICE's mixed fidelity performance the difference in perturbation strategies. SLICE applies Gaussian blur-based perturbations, whereas LIME and GRID-LIME use randomized occlu- sions. This methodological difference affects how feature importance is evaluated, potentially biasing AOPC and AUC scores. To address this, we introduce an alternative AOPC evaluation (Appendix B where SLICE is tested under the same perturbation method as LIME and GRID-LIME. This ensures a fair comparison of fidelity by eliminating potential biases introduced by differing perturbation techniques.
GRID-LIME
1.0 Oxford-BBT PetsiinceptonnA
PetuRechetto
Pascall VOCilccption VI
Pascall VDC Recsiess
MSCOCO: Inception
MScOCC: RocHet50
Eas
0.6
5.20
9.4 ADPC Scores
0.1
AOFC Inserton Scores
ADFC Inaartinn Stores
ADPC Insertion Score
ADPC Insertinn Scores
AGPC Insertion Scores
Orfond-asT Patecicaptteev
1.8
Ooford-Hff PetrcRechiet0
1.0
Peocal VBCilcesptin V3
Paccal VOC:Redltet50
1.0
NCcOCO: Inception VS
3.0
McCOCC:: Beckeeso
68 0.2 6.1
3.6
-3.2 0.00
6.35
0.50
-03
3.0 0.2
6.5
003
08 02
0.8 0.4
646
0.0
6.2 0.4
0.6 0.6
0.
50 63 64
Figure 6: Empirical Cumulaaive Distribution Function (ECDF) plot of AOPC scores for different methods. The first row corresponds to the insertion task and the second row corresponds to the deletion task. A higher AOPC score indicates better fidelity in capturing important features.
5.5 Results Beyond the Original Paper
Beyond reproducing the original results, we introduce the Ground Truth Overlap (GTO) metric to evaluate how well explanations aligu with segmentation maps. Figure g presents the results.
LINE consistently achieves the highest GTO scores, indicating better alignment with ground truth segmen- tations. SLICE generally performs worse across most cases, while GRID-LIME performs better in about half the cases but remains further from LIME. This trend is expected since GTO is based on the Intersection over Union (ToU) of segmentation masks, and grid- methods naturally achieve higher overlap due to their grid boxes.
Additionally, we introduce an alternative variation of the AOPC ECDF plot in Appendix B where the insertion logic is modified. Unlike the standard evaluation, this version uses the perturbation method of LIME and GRID-LIME for SLICE instead of applying Gaussian blur. This adjustment ensures that SLICE is evaluated under the same perturbation conditions as the other methods, removing potential biases introduced by differing perturbation strategies. As result, this version provides a more direct and fair comparison of
10

---

Published in Transactions on Machine Learning Research (05/2025)
Doford Paturincapton/3
Gaford-iIT PatccRacietio
Pascal VOCincaption
Pascal VDC Resitato
NSCOCO:
MSCOCD: Rashle50
0.0
9.00
685
0.8
0.0
62
6.00
8.0 6.35
6.9 0.55 6.5 6.88
-6.2 -6.50 -0.25
Doford-HIIT PetsinceptionS
Oeford SIT Resieeee
Pascel VOCIncaption V3
Procol VOC-Resitet50
MSCOCO: Inception V5
MSCOCO: Reshor50
0.0 -3. -0.3 Daletion
1.5 Deladion
80-
AUC Daletion
5.2
-10
Daletion Scores
9.0
-0.8 -2. 6.3 0.0 ASC
0.0 3,00 -0.7H -0.50 5:29 9.00
Figure Empiriaal Cumulative Distribution Function (ECDF) plot of AUC scores for insertion (row and deletion (row 2) tasks. For insertion, higher AUC value is preferable, indicating that adding back important features improves model confidence. For deletion, a lower AUC value is betler, as it suggests that removing key features leads to significant drop in confidence.
fidelity. Under this setting, SLICE performs similarly to LIME and GRID-LIME for insertion but slightly worse for deletion.
Coford-ilT Fets. ResNet20
Phocal VDC. ResNee30
NS-COCO, Rechiet5
3.0
1.-
1.0
2.02
SUCE
5,02
Stur2 1ub CV0 10h6
09
Ground buth 100ke
04
8.4
Koppoptich
Owterd-B1 Patsc Inceptionnt
Plocal VDC: Incepterncc
MS-COCD: Inceptionis
3.40
3.2
1.0
Grocn 12b 20-0he 100.5
Granns Ooertee
0.4
6.-
00
Figure Ground Truth Overlap (GTO) plots, where the x-axis represents the number of top-k ranked superpixels ranging from to 6, and the y-axis indicates the GTO score (higher better). LIME achieves the highest alignment with segmentation maps, GRID-LJME performs moderately well, and SLICE lags behind in most cases.
11

---

Published in Transactions on Machine Learning Research (05/2025)
6 Discussion
Our results partially support the claims made by Bora et al. (2024), confirming that SLICE improves explanation consistency but challenging the assertion that provides superior fidelity. Additionally, we identified methodological limitations the original study, particularly in how perturbation strategies affect fidelity evaluations.
Across the Oxford-IIIT Pets, PASCAL VOC, and MS COCO datasets, SLICE consistently produced more stable explanations than LIME, as reflected in its higher Combined Consistency Metric (CCM) scores. However, our fidelity analysis using AOPC and AUC showed that SLICE did not always outperform LIME. Our newly introduced Ground Truth Overlap (GTO) metric further suggested that LIME often produces more regmentation-aligaeen attributions, despite its instability. This discrepancy likely arises from SLICE's use of Gaussian blur-based perturbations, which affect how fcature importance is preserved compared to LIME's direct occlusions. To address this, we introduced an alternative AOPC evaluation (Appendix B where SLICE was tested using the same perturbation strategy as LIME and GRID-LIME, ensuring fairer fidelity comparison. Under these conditions, SLICE performed similarly to the other methods for insertion but slightly worse for delction, suggesting that its fidelity advantage in the original setup may be influenced by differences in perturbation techniques hather than intrinsic method effectiveness.
Another important limitation of SLICE is its computational cost. Our runtime analysis revealed that SLICE is significantly slower than LIME, requiring up to 13 times more processing time per experiment. While SLICE improves explanation stability, its increased computational complexity and energy consumption make it less practical for large-scale applications. As alternative, introduced GRID-LIME, which replaces superpixels with structured grid. Our results showed that GRID-LIME imn-
LIME's segmentation--asee proves explanation stability while maintaining a computational cost closer to LIME than SLICE as shown in Table Iin in Appendix D This suggests that part of LIME's inconsistency stems from its segmentation process rather than its perturbation mechanism. However, GRID-LIME's rigid grid-based approach does not always align well with object boundaries, potentially affecting interpretability in certain cases.
A key limitation of our study is that our implementation of SLICE may differ slightly from the authors' original setup. While we referenced their official SLICE code, certain hyperparameter choices and evaluation metric implementations (such as ASFE, ARS, and CCM) had to be re-implemented based on the paper's 's descriptions. Despite these challenges, our extensive experimentation and sensitivity analysis suggest that our findings remain robust.
Overall, our study highlights trade-offs post-hoe explainability methods. While SLICE improves consis- tency, its computational cost and fidelity claims require further scrutiny. Future research should focus on refining perturbation techniques to overcome the shift in distribution of input image, optimizing computa- tional efficiency, and developing alternative fidelity metries that do not rely solely on perturbation- evaluations.
6.1 What was easy
The authors provided a GitHlub repository containing an implementation of SLICE, which facilitated the reproduction of initial results. We were able to validate the consistency of sign and ranking of important superpixels for two images using their provided pickle files. Communication with the authors was helpful, as they provided missing metric code for validation upon request.
6.2 What was difficult
Several challenges emerged during reproduction. The codebase was incomplete, lacking implementations for key metrics such as ASFE, ARS, and CCM. Additionally, AOPC scores were not initially included. While the authors later provided an implementation for AOPC, the other metrics had to be inferred based on the paper's descriptions.
Another major limitation computational efficiency. Running the original SLICE implementation which was in Tensorflow took approximately 15 minutes per image, making large-scale experiments infeasible. We
12

---

Published in Transactions on Machine Learning Research (05/2025)
improved runtime by rewriting the implementation in Py'oorc and incorporating multi-threading, which enabled us to conduct more comprehensive evaluations.
6.3 Communication with original authors
After reviewing the GitHub repository, we identified missing components and contacted the authors for clarification. They responded positively and provided the missing code for AOPC calculations. However, ASFE, ARS, and CCM metric implementations were never made available, requiring us to re-implement them independently. Despite this, the provided assistance facilitated certain aspects of reproduction.
13

---

Published in Transactions on Machine Learning Research (05/2025)
References
Revoti Prasad Bora, Philipp Terhörst, Raymond Veldhuis, Raghavendra. Ramachandra, and Kiran Raja. Slice: Stabilized lime for consistent explanations for image classification. In Proceedings of the IEEE/CVE /CF Conference on Computer Vision and Pattern Recognition, pp. 10988 109966, 2024.
Zeinab Dehghani, Koorosh Aslansefat, Adil Khan, Adín Ramírez Rivera, Franky George, and Muhammad Khalid. Mapping the mind of instruction-based image editing using smile, 2024. URL https://arxiv.
brg/abs/2412 16277
Mark Everingham, Luc Van Gool, Christopher KI Williams, John Winn, and Andrew Zisserman. The pascal visual object classes (voc) challenge. International journal of computer vision, 88:303 338, 2010.
Alicja Gosiewska and Przemyslaw Biecek. Do not trust additive explanations, 2019. URL https://arxi>:
org/abs/1903. 11420
Arne Grobrügge, Nidhi Mishra, Johannes Jakubik, and Gerhard Satzger. Explainability in ai-based ap- plications a framework for comparing different techniques. In 2024 26th International Conference on Business Informatics (CBI), pp. 70-79. IEEE, September 2024. doi: (..1009/cb66200442224.00000 URL
http://dx. doi
http://dx..oi..orgg// www.wot.......ooooooo 1009//BI66000 2024.00011.
Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 770 778, 2016.
Patrick Knab, Sascha Marton, and Christian Bartelt. Beyond pixcls: Enhancing lime with hicrarchical features and segmentation foundation models, 2025. URL https://aaxiii httpss:/aarxiv.org/aaaa. abt////aa/2/3//3/ 04403.07733
Tsung- Yi Lin, Michael Maire, Serge Belongie, Lubomir Bourdev, Ross Girshick, James Hays, Pictro Perona, Deva Ramanan, C. Lawrence Zitnick, and Piotr Dollár. Microsoft coco: Common objects in context, 2015. URL https://arxiv http://aaziiv.oorr//////////
Omkar M Parkhi, Andrea Vedaldi, Andrew Zisserman, and CV Jawahar. Cats and dogs. In 2012 IEEE IEEE, 2012.
conference computer vision and pattern recognition, pp.
Muhammad Rashid, Elvio G. Amparore, Enrico Ferrari, and Damiano Verda. Using stratified sampling to improve lime image explanations, 2024. URL aattps://arie httpp/////arra.oor///////0//0
Marco Tulio Ribeiro, Sameer Singh, and Carlos Guestrin. why should trust you?" explaining the predictions of any classifier. Tn Proceedings of the 22nd ACM STGKDD international conference on knowledge discovery and data mining, pp. 1135 1144, 2016.
Shong Shi, Yangzhou Du, and Woi Fan. An extension of lime with improvement of interpretability and fidelity, 2020. URL https::/t//t/ //tpxv/a..//////a www.rrivv.rgggrb//..ee orgg/ba//0040..... 12277
Christian Szegedy, Vincent Vanhoucke, Sergey loffe, Jon Shlens, and Zbigniew Wojna. Rethinking the inception architecture for computer vision. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 2818-2826. 2016.
Andrea Vedaldi and Stefano Soatto. Quick shift and kernel methods for mode seeking, In Computer Vision ECCV 2008: 10th Europeon Conference on Computer Vision, Marseille, France, October 12- 18, 2008, Proceedings, Parl TV 10, pp. 705-718. Springer, 2008.
Muhammad Rehman Zafar and Naimul Mefraz Khan. Dlime: A deterministic local interpretable model- agnostic explanations approach for computer-aided diagnosis systems, 2019. URL https://arxiv.orr abs/1906.
Xingyu Zhao, Wei Huang, Xiaowei Huang, Valentin Robu, and David Flynn. Baylime: Bayesian local interpretable model-agnostic explanations. In Uncertainty in artificial intelligence, pp. 887-896. PMLR, 2021.
14

---

Published in Transactions on Machine Learning Research (05/2025)
Zhengze Zhou, Giles Hooker, and Fei Wang. S-lime: Stabilized-lime for model explanation. In Proceed- ings of the 27th ACM SIGKDD Conference on Knowledge Discovery amp; Dala Mining, KDD '21, pp. 2429 2438. ACM, August 2021. doi: 10.1145/34445548.4467222 3467274 3447548. 3467274
URL http: //x: doi org/10..1455
15

---

Published in Transactions on Machine Learning Research (05/2025)
A Wilcoxon rank test
To confirm significance, we conducted & Wilcoxon rank test on AOPC (Table D and AUC (Table 2), where D:M denotes Dataset:Model. represents Oxford-IIT Pets, refers to Pascal VOC, C refers to MS COCO, indicates InceptionV3, and R denotes ResNet50. W stands for the Wilcoxon signed rank statistic, p-value is the probability of observing the data under the null hypothesis that the median difference is zero, Ma denoles the median difference of the pair-wise scores, and N.C refers to the nunber of negalive differences (out of 50 images). While SLICE significantly outperforms LIME in AOPC-Insertion, the AOPC-Deletion and the AUC test do not show consistent differences across all comparisons.
Table 1: Wilcoxon rank test results for compari- son of LIME((T), SLICE(S), and GRID-LJMME(G) AOPC. A tests whether the median differ- ence (OPCscore AOPCscore(y) is zero.
Table 2: Wilcoxon rank test results for compari- son of LIME(L), SLICE(S), and GRID-LIME(() on AUC. AUC(x,)) tests whether the median difference in AUC scores is zero.
Test
D:M
W p-value
MA N.C
Test
D:M
W p-value
Ma N.C
Insertion
Insertion
AOPC(S,L)
0:1
1275
8.88e-16
0.03286
0
AUC(S,L) 0:1
0
1.00e+000
-0.07546
50
AOPC(S,L)
O:R
1267
2.22e-14
0.04166
2
AUC(S.L) O:R
0
1.00e++00
-0.11903
50
AOPC(S.L)
P-
1209
1.74e-10
0.04280
2
AUC(S.L) P:1
0
1.00e+000
-0.05818
50
AOPC(S,L)
P-R
1274
1.78e-15
0.05409
1
AUC(S.L) P.R
41
1.00e++00
-0.06117
49
AOPC(S,L)
C:T
1275
8.88e-16
0.03483
AUC(S.L) C:T
0
1.00e+00
-0.06956
50
AOPC(S,L)
C:R.
1272
4.44e-15
0.04970
2
AUC(S,L) C:R
0
1.00e+000
-0.06839
50
AOPC(S,G) 0:1
1275
8.88e-16
0.03255
AUC(S,G) 0:1
0
1.00e+00
-0.11019
AOPC(S,G) O:R
1275
8.88e-16
0.04088
0
AUC(S,G) O:R
0
1.00e+00
-0.13420
50
AOPC(S,G) P-I
1210
1.57e-10
0.04353
2
AUC(S,G) P:I
0
1.00e++00
-0.07070
AOPC(S,G)
P.R
1255
3.30e-13
0.05474
2
AUC(S,G) P.R
27
1.00e+000
-0.06947
49
AOPC(S,G) C:I
1275
8.88e-16
0.03405
AUC(S,G) C:I
0
1.00e+00
-0.09347
50
AOPC(S,G)
C:R.
1246
1.54e-12
0.05021
1
AUC(S,G) C.R
1
1.00e+00 -0.08312
49
Deletion
Deletion
AOPC(S.L)
0:1
727
1.97e-01
0.00593
20
AUC(S.L) 0:1
729
1.91e-01
3.27c-05
AOPC(S,L) O:R
483
9.32e-01
-0.01037
29
AUC(S,L) O:R
989
2.44c-04
0.00554
16
AOPC(S.L)
424
9.81e-01
-0.02597
29
AUC(S.L) P.I
1079
3.36e-06
0.01253
AOPC(S.L)
P.R.
507
8.96e-01
-0.00089
26
AUC(S.L) P.R
1064
7.60e-06
0.03557
15
AOPC(S,L)
412
9.86e-01
-0.00477
27
AUC(S,L) C:I
936
1.71e-03
0.00466
AOPC(S,L)
C:R.
414
9.85e-01
-0.01545
29
AUC(S,L) C:R
1051
1.49e-05
0.01683
13
AOPC(S,G) 0:I
527
8.57e-01
-0.00324
26
AUC(S,G) O:1
907
4.29e-03
0.00284
20
AOPC(S,G) O:R
694
2.96e-01
0.00505
21
AUC(S,G) O:R
1016
7.79c-05
0.01112
15
AOPC(S,G)
P-1
206
1.00++000
-0.08023
36
AUC(S,G) P:1
1144
5.25e-08
0.02479
12
AOPC(S,G)
P-R
457
9.60e-01
-0.00174
25
AUC(S,G) P.R.
1049
1.64e-05
0.03463
13
AOPC(S,G)
C:T
281
1.00e+000
-0.02752
34
AUC(S,G) C:T
1087
2.13e-06
0.01570
13
AOPC(S,G)
C:R.
308
9.99e-01
-0.03289
34
AUC(S,G) C.R.
1060
9.38c-06
0.02105
16

---

Published in Transactions on Machine Learning Research (05/2025)
B Alternative AOPC Scores
A modified version of the ECDF plot of AOPC scores with = 0 is presented in Figure In this version, 0 denotes that SLICE does not apply Gaussian blurring; rather, it uses the same perturbation strategy LIME and GRID-LIME, masking selected superpixels by setting them to zero. This ensures fairer evaluation by placing SLICE on the same evaluation framework as the other methods.
SLICE
LIME
GRID-LIME
Oxford-IIIT Pets:InceptionV3
Oxford-IIIT Pets:ResNet50
Pascal VOCaInception V3
Pascal VOC:ResNet50
1.0
1.0
1.0
1.0
Cumulative Probability
Cumulaativ Prebabiiitt
Cumulative Probability
Cumulattive Probability
0.8
0.8
0.8
0.8
0.6
0.4
0.4
0.4
0.4
0.0
0.0
0.0
0.0
-0.017
-0.016
-0.02
0.00
0.02
-0.016
-0.014
-0.03
-0.02
-0.01
AOPC Insertion Scores
AOPC Insertion Scores
AOPC Insertion Scores
AOPC Insertion Scores
Oxford-IIIT PetstInceptionV3
Oxford-IIIT Pats:ResNet50
Pascal VOC:inception V3
Pascal VOC:ResNet50
cunulative Probability
Cumuletive Probebility
0.8
0.8
0.8
0.8
0.6
0.6
0.6
0.6
Cumulative
0.4
0.4
0.4
0.2
0.2
0.2
0.2
-0.5
0.0
0.5
-0.5
0.0
0.5
-0.5
0.0
0.5
-0.5
0.0
0.5
Figure 9: Alternative ECDF plot of AOPC scores with σ 0, using LIME and GRID-LIME's perturbation method for SLICE instead of Gaussian blur (higher better).
C Hyperparameters
Table 3: Hyperparameter values used in experiments for SLICE, LIME, and GRID-LIME. The values were determined through empirical validation and small-scale hyperparameter searches to ensure fair and reproducible comparisons. For SLICE, optimized the Adaptive Gaussian Blur using an iterative search strategy, while GRID-LIME's grid box size is determined dynamically based on perturbation consistency.
Hyperparameter
Value
Adaptive Gaussian Blur search range
{0.1, 0.2, 0.3, 0.4, 0.5)
Number of Ridge Regression Models (M)
1000
Number of perturbations per image (N)
500
SEFE Tolerance
3
Number of iterations per image
10
Number of images sampled per dataset
50
Quickshift kernel size
5
Quickshift max distance
200
Quickshift ratio
0.2
17

---

Published in Transactions on Machine Learning Research (05/2025)
Computational requirements and environmental impact
The table below presents an analysis of computational runtime and associated carbon emissions for different model-explainer-dateen combinations. The explainers considered include LIME, SLICE, and GRID-LIME, applied ResNet50 and InceptionV3 models trained on three datasets: Oxford-ITIT Pets, Pascal VOC, and MS COCO. The objective is to assess the environmental impact of these machine learning experiments in terms of both execution time and carbon emissions.
The values in the table were estimated using the ML CO2 Impact calculator tool that provides carbon emission estimates based on hardware type, runtime, and cloud provider. One NVIDIA A100 GPU was used for all computations. The estimated carbon footprint for each experiment is measured in kilograms of CO2 equivalent.
The ML CO2 Impact calculator determines emissions based on multiple factors, including hardware effi- ciency, runtime, and the energy grid's carbon intensity. The results indicate clear differences between the explainability methods. LIME and GRID-LIME have consistently lower runtime and emissions compared to SLICE. InceptionV33 requires more computational resources than ResNet50, leading to increased carbon emissions. Dataset size also influences results, with larger datasets requiring more processing time and energy.
Table 4: Computational runtime and carbon emissions experiment.
Explainer
Model
Dataset
Time (min)
Carbon emissions (kg COze)
LIME
ResNet50
Oxford-IIIT Pets
34
0.05
LIME
ResNet50
Pascal VOC
35
0.05
LIME
ResNet50
MSCOCO
37
0.06
LIME
InceptionV
Oxford-IIIT Pets
51
0.11
LIME
InceptionV's
Pascal VOC
52
0.11
LIME
InceptionV3
MS COCO
52
0.11
SLICE
ResNet50
Oxford-IIIV Pets
0.65
SLICE
ResNet50
Pascal VOC
328
0.65
SLICE
ResNet50
MS COCO
334
0.65
SLICE
InceptionV3
Oxford-IIIT Pets
576
1.08
SLICE
InceptionV3
Pascal VOC
1.08
SLICE
InceptionV3
MS COCO
593
1.08
GRID-LIME
ResNet50
Oxford-IIIT Pets
35
0.05
GRID-LIME
ResNet50
Pascal VOC
39
0.06
GRID-LIME
ResNet50
MS COCO
41
0.07
GRID-LIME
InceptionV33
Oxford-IIIT Pets
52
0.11
GRID-LIME
InceptionV3
Pascal VOC
53
0.11
CRID-LIME
InceptionV3
MS COCO
53
0.11
hhttps:/mmcc2..p www.lii....gttbb...oo. wwt.hu..../inpccc.......oo
18

---

Published in Transactions on Machine Learning Research (05/2025)
E
Results from original paper
We reproduced the authors' Kernel Density Estimation (KDE) plots of the Combined Consistency Metric (CCM) scores for SLICE using their TensorFlow implementation, as shown in Figure
Oxford-IIT 5
Pascal VOC:Resnet50
SLICE
SLICE
0.13
022
0.4
0.5
0.0
1.0
0.0
0.
0.8
13.1
1.0
CCM Scores
CCM Scores
Oxford-IITT Pets:inception33
InceptionV/3
SLICE
SLICE
1-
0.0
0.4
0.0
0.4
n.8
Scores
Scores
Figure 10: Kernel Density Estimation (KDE) plots of the Combined Consistency Metric (CCM) scores for SLICE using the author's code.
19

---

Published in Transactions on Machine Learning Research (05/2025)
F Qualitative Examples
To provide additional insight into the explanations generated by each method, we present qualitative ex- amples of LIME, SLICE, and GRID-LIME. These visualizations highlight the ranked positive and negative superpixels, showcasing how each explainer method attributes importance different image regions.
F.1 LIME Explanations
Figure Qualitative examples of LIME explanations, displaying ranked positive (highlighted warm colors) and negative (highlightod in cool colors) superpixels. LIME assigns importance by sampling perturbed versions of the image and fitting linear model to approximate the model's decision boundary.
F.2 SLICE Explanations
Figure 12: Qualitative examples of SLICE explanations, showing ranked positive and negative superpixels. SLICE applies spatially-aware perturbations and adaptive feature elimination, improving local consistency in aotsibutions.
F.3 GRID-LIME Explanations
Figure 13: Qualitative examples of GRID-LIME explanations, where attributions are structured based on predefined grid. Positive and negative superpixels are ranked similarly to LIME but constrained within fixed spatial regions, leading to different interpretations.
20

---

Published in Transactions on Machine Learning Research (05/2025)
F.4 Consistency of LIME, SLICE, and GRID-LIME
LIME
SLICE
GRID-LIME
Iter
Iter 2
Iter
Iter 7
Iter 10
Figure 14: Examples of positive superpixels returned multiple of LIME, SLICE, and GRID-LIME. LIME exhibits the highest variability, while SLICE demonstrates the most consistent attributions.
LIME
SLICE
GRID-LIME
Figure 15: Examples of nogative superpixels returned over multiple runs of LIME, SLICE, and GRID-LIME.
21


Reviews:
[{"review_id": "kedpVH1Cd6", "reviewer": "Reviewer_CUpd", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strengths:\n(1) The authors reproduce SLICE using pytorch, making the method easier to use in the future. \n\n(2) The authors analyze SLICE in detail, including the segmentation-related performance and fidelity. They propose new metrics to evaluate explainability methods and find the directions for possible future improvements. \n\nWeaknesses:\n(1) The writing should be improved a lot. For example, all the figures have low resolutions and the details are difficult to read. \n\n(2) The reproduced code is not validated. At least in one experiment the authors should provide the results done by the original SLICE code. \n\n(3) The importance of GRID-LIME is not emphasized enough. For example, in the main paper the authors should have a table showing the inference speed of all the methods to validate the claim \"GRID-LIME improves explanation stability while maintaining a computational cost closer to LIME than SLICE\".", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "This work reproduces and evaluates a post-hoc model explainability method called SLICE. SLICE (Bora et al., 2024) enhances another method called LIME by incorporating adaptive Gaussian blurring and sign entropy-based feature elimination to improve explanation consistency. The authors of this work validate SLICE using images from datasets including MSCOCO, Oxford-IIIT Pets and PASCAL VOC (50 images from each dataset). They find SLICE can produce more consistent explanations. Moreover, by fidelity analysis, they find SLICE is influenced by perturbation techniques and its deletion strategy is worse than LIME on the proposed AOPC score. The authors additionally propose Ground Truth Overlap (GTO) metric to validate SLICE, and they find LIME often produces more segmentation-aligned attributions. Finally, they mention that SLICE is much slower than LIME, and they propose one additional method called GRID-LIME that replaces LIME’s segmentation-based superpixels with a structured grid, obtaining faster and more stable results.", "claims_and_evidence_yn": "Yes", "claims_explanation": "", "audience_yn": "Yes", "audience_explanation": "", "requested_changes": "As mentioned in the weaknesses, the authors should replace all the figures in the paper with higher resolution images. Moreover, the authors should add additional experimental results to validate the reproduced code and also the value of their proposed GRID-LIME, since from current results, GRID-LIME has worse explanation consistency than SLICE and generates worse object boundaries.", "broader_impact_concerns": "no", "additional_comments": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "4PTRCOizCq", "reviewer": "Reviewer_XQ5V", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "## Strengths\n1. Overall, this manuscript is well-written.\n2. This manuscript provides a method for generating perturbed images named GRID-LIME, and a metric GTO assessing the overlap between ground-truth and explanations.\n\n## Weaknesses\n1. Simply reproducing a paper doesn't automatically make this paper above the acceptance bar in TMLR, unless the manuscript delivers significantly meaningful arguments during the reproducing progress.\n2. Some parts in the presentation needs further clarification. I'll specify in the requested changes.\n3. The newly proposed GRID-LIME and GTO are not well justified.\n5. This reproducibility work only evaluated resent50 and inceptionv3, this lack of the variety of architectures undermines the value of this work. I'll reserve until I see more architectures are included.", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "This manuscript presents a reproducibility study for the \"SLICE\" paper [1]. By repeating their experiments, the authors verify the claims in SLICE on several benchmark datasets: Oxford-IIIT Pets, PASCAL VOC, and MS COCO.\n\nBeyond the reproducibility study, this manuscript also presents two increments GRID-LIME and GTO. \n\nThe claimed contributions are: \n1. Reproducibility study for paper SLICE [1].\n2. GRID-LIME: a colour-based superpixel segmentation \nmethod that can enhance LIME’s stability.\n3. GTO (Ground Truth Overlap) metric: a metric for explanation fidelity.\n\nBeyond repeating the experiments, this manuscript presents something new.", "claims_and_evidence_yn": "Yes", "claims_explanation": "", "audience_yn": "Yes", "audience_explanation": "", "requested_changes": "1. I'm wondering the justification of various perturbation method. The perturbation sampling injects our priors (perhaps bias sometimes), does this prior/bias ensure the faithful explanation?\n\n2. I read through the manuscript, I searched for the motivations for proposing LIME-GRID and GTO but I couldn't see them. Could you please clarify? I would like to suggest the authors justify the motivations as earlier as possible in eg Introduction. Thank you.\n\n3. Section 4.1.3:\n> (A)Since LIME relies on color-space segmentation (quickshift), its explanations suffer from randomness. (B) GRID-LIME eliminates this variability by enforcing consistent superpixel regions that utilize the black-\n\nCould you please clarify the logic between (A) and (B)? I couldn't understand.\n\n4. Section 4.3:\n> ForSLICE,we train 1000 Ridge regression models to obtain model coefficients for all superpixels. EachRidge model is trained on 500 randomly generated perturbations of the original test image.\n\nCan you clarify the choices of these settings?\n\n5. Section 4.4: \n    Can you also include other architectures eg vision transformation?\n\n6. In lime-grid, when you zero super pixels, you significantly change the distribution of input X. The distribution of significantly changed input domain is out-of-distribution from what the models trained with. \n\n    Please clarify how do you consider regarding this, and how do you deal with the perturbation inputs.\n    \n\n7. Could you please clarify why the baselines are different from the original SLICE paper?\n     > 5.1 Claim 1: LIME suffers from inconsistencies due to sign flip and rank variance\n\n8. I didn't find that you discussed the \"Deletion and Insertion Game\"? Could you please clarify?", "broader_impact_concerns": "NA", "additional_comments": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "mamURgJVNw", "reviewer": "Reviewer_KMy9", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "**Strengths**\n* The paper is clear and well-written\n* Reproducibility of explanation methods is a timely topic\n* Code is made available and it is well-structured\n\n**Weaknesses**\n* Presentation of Grid-LIME should be made more formal\n* I have a few clarifying questions on a couple of claims\n\nI will expand on these below and I am looking forward to discussing with the authors!", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "The submission presents a reproducibility study of SLICE, a post-hoc explanation method that improves upon the widely-used LIME with Adaptive Gaussian Blur and Sign Entropy Feature Elimination. The proposed study focuses of four claims made in the original paper. Furthermore, the submission proposes a different variation, Grid-LIME, which is more computationally efficient and partially improves on the original LIME. Finally, the study includes a measure of overlap of the produced explanations with ground-truth segmentations.", "claims_and_evidence_yn": "Yes", "claims_explanation": "", "audience_yn": "Yes", "audience_explanation": "", "requested_changes": "**Original implementation**\n\nCould the authors expand on the need to re-implement SLICE? It is mentioned that some evaluation metrics were missing from the original implementation, but also that they re-implemented the method in PyTorch. This seems motivated by computational efficiency needs to run large-scale experiments. What was the original implementation in? What were its efficiency limitations?\n\n**Presentation of LIME**\n\nIs there a typo in \"generates perturbed images $z$ by masking superpixels\"? Should $z$ be $z'$? Also, the set $\\mathcal{Z}$ is not defined.\n\n**Presentation of Grid-LIME**\n\nGrid-LIME should be presented more formally, with precise equations for the coefficient of variation and pseudocode.\n\nCould the authors expand on the sentence \"Since LIME relies on color ...\". What randomness are the authors referring to, and why does it depend on color? Without equations, it is unclear what \"enforcing consistent superpixel regions ...\" means.\n\nIn Figure 2, the bottom panel reads \"generate perturbed images with $\\sigma = 0$\". Could the authors expand on this? If $\\sigma = 0$, there is no perturbation?\n\n**Hyperparameters**\n\nSection 4.3 presents how hyperparameters were chosen, but later in Section 4.4 it is claimed that \"hyperparameters values follow the original paper\". Could the authors clarify which hyperparameters were tuned, and which were transferred from the original paper?\n\nCould the authors expand on which hyperparameters were kept constant across methods? The different methods seem to require different sets of hyperparameters.\n\nAre the models used in this study the same used in the original paper? Hyperparameters may change significantly depending on which models are used. \n\n**Reproducbility vs Replicability**\n\nRelated to the points above about re-implementation, hyperparameters, and models used in the study. If there are significant differences between the proposed study and the original paper, it may be more appropriate to call this a \"replicability\" study rather than \"reproducibility\", if the scope goes beyond reproducing the results presented in the original paper.\n\n**Ground Truth Overlap metric**\n\nGTO is a function of $k$, where $k$ is the number of top-$k$ important features included in the metric. Could the authors specify what value of $k$ was used in their experiments?\n\nEq. (2) normalizes the intersection by the area of the top-$k$ important features. This implies that GTO is a measure of \"precision\" of the explanations, i.e. what percentage of the reported area is actually important. However, a reasonable complement of this would be \"recall\", i.e. what percentage of the ground truth is reported as important. Have the authors considered including both to give a more complete representation of the performance of the different methods?\n\n**AOPC and AUC analysis**\n\nI am not sure I understand how to read Figures 5, 6, and 9. It is my understanding they report the respective ECDFs of AOPC and AUC over a dataset of images. Claims are made about the performance of different methods based on AOPC and AUC being \"higher\" or \"lower\". Are these claims made in terms of the ECDFs curves or particular statistics (e.g., the median?). These values should be included in order to verify claims.\n\nTables 1 and 2:\n* what are the columns $W$, $M_{\\delta}$, and $N.C$?\n* the tests seem to be incomplete? Why is there only one test for the Pascal VOC dataset, and 4 for the Oxford Pet dataset?\n* I was surprised to see $p$-values all exactly equal to 1 in Table 2. The AUC curves in Figure 6 definitely look different. Could the authors clarify whether the tests are performed for inclusion or deletion? Or aggregated across the tasks?\n\nCould the authors clarify these claims?\n\n---\n\n**Minor comments**\n* Section 2, claim 3: typo in \"sigma\"\n* Page 3, top paragraph: typo in \"SLICE to standard LIME\"\n* Section 4, first paragraph: it is unclear what \"interpretable intermediate images\" are\n* Sections 5.2, 5.4: Figure 7 is referenced in the text before Figure 5; \"AdaBlur\" is not defined; insertion and deletion tasks are not defined.\n* Figure 5, 6 are not readable\n* Section 5.5 feels out of place, was this supposed to be after Section 5.3?", "broader_impact_concerns": "No concerns", "additional_comments": "", "extra_scores": {}, "extra_text": {}}]

Author rebuttal:

Meta-review: Some reviewers expressed concerns about the writing quality and the clarity of the figures. To expedite the publication process, I have decided to accept the paper as is, rather than requesting minor revisions. However, I will closely review the writing and figures upon final submission. The authors are strongly encouraged to ensure that the final version is clearly written and includes high-quality figures to facilitate prompt publication
Decision: Accept (poster)