Explaining with trees: interpreting CNNs using hierarchies
Authors: Caroline Mazini Rodrigues
Nicolas Boutry
Laurent Najman
Venue: Accepted by TMLR ()

Abstract: Challenges remain in providing interpretable explanations for neural network decision-making in explainable AI (xAI). Existing methods like Integrated Gradients produce noisy maps, and LIME, while intuitive, may deviate from the model’s internal logic. We introduce a framework that uses hierarchical segmentation techniques for faithful and interpretable explanations of Convolutional Neural Networks (CNNs). Our method constructs model-based hierarchical segmentations that maintain fidelity to the model’s decision-making process and allow both human-centric and model-centric segmentation. This approach can be combined with various xAI methods and provides multiscale explanations that help identify biases and improve understanding of neural network predictive behavior. Experiments show that our framework, xAiTrees, delivers highly interpretable and faithful model explanations, not only surpassing traditional xAI methods but shedding new light on a novel approach to enhancing xAI interpretability.

Published in Transactions on Machine Learning Research (01/2026)
Explaining with trees: interpreting CNNs using hierarchies
Caroline Mazini Rodrigues'
Laboratoire de Recherche de I'EPITA - LRE Le Kremlin Kremlin-Bicétre, 94270, France & Univ Gustave Biffel, LIGM Marne-la- Vallée, 77454, France
caroline. haliineemmaiiiddrgeueeeeeeeee
Nicolas Boutry
Laboratoire de Recherche de L'EPITA LRE Le Wremlin-Bicétre 94270, France nicolas, boutry@lrde. ep
Laurent Najman
College Computing and Mathematical Sciences, Dept of Mathematics, Khalifa University, Abu Dhabi, UAE & Univ Gustave Biffel, CNRS, LIGM Marne-la- Vallée, 77454, France laurent. sareett.ajmmneesieee.
Reviewed on OpenReview: https: openreview net/ forum? id= #fyWW2551i
Abstract
Challenges remain providing interpretable explanations for neural network decision-making in explainable Al (xAI). Existing methods like Integrated Gradients produce noisy maps, and LIME, while intuitive, may deviate from the model's internal logic. We introduce a framework that uses hierarchical segmentation techniques for faithful and interpretable explanations of Convolutional Neural Networks (CNNs). Our method constructs model-based hierarchical segmentations that maintain fidelity to the model's decision-makinn process and allow both human-centric and model-centric segmentation. This approach can be combined with various xAI methods and provides multiscale explanations that help identify biases and improve understanding of neural network predictive behavior. Experiments show that our framework, 2AiTTress delivers highly interpretable and faithful model explanations, not only surpassing traditional xAI methods but shedding new light on novel approach to enhancing xAI interpretability.
1 Introduction
In modern deep learning applications, especially in healthcare and finance, there is growing need for transparency and explanation. Understanding model's rationale is crucial before relying its predictions. This need arises from biases present at various stages of model development and deployment. While some biases help in learning data distribution (Goyal & Bengio, 2022), others may indicate data imbalance, incorrect correlations, or prejudices data collection.
Explainable Artificial Intelligence (xAI) provides methods that clarify models' decision-making processes with different levels of interpretability, which can be described as the measure of how easy to understand explanation (Gilpin et al., 2018). Providing interpretable explanations especially important in the previously mentioned contexts (e.g., health), where humans need to understand models' decisions.
For this purpose, some xAI methods use object-structure-bssee visualizations to enhance human interpretation. They decompose images in ways that mimic human perception, grouping objects by attributes such as color, texture, and edges (Hubel & Wiesel, 1959). Techniques such as LIME (Ribeiro et al., 2016) and
"https://carollmzznn http:///arrlmmiiii.ggit..b...... iithub.io/

---

Published in Transactions on Machine Learning Research (01/2026)
Fidelity vs. Interpretability
IG
LIME
Tree-Occ
IG-Tree-Occ
Adaptable Hierarchical Regions
Choice using faithful interpretabe methods
Figure 1: One challenge in xAl achieving a good trade-off between fidelity and interpretability, We propose using region-based segmentation combined with hierarchies to adapt region size, while providing the flexibility use high-fidelity methods for constructing the segmented regions.
KernelSHAP (Lundberg & Lee, 2017) have used this approach effectively, segmenting images into meaningful parts to improve interpretabilily. However, the size of the segmented regions affects the information extracled: small regions can difficult to interpret, while large regions may overlook fine details. Additionally, using a segmentation framework may introduce a form human bias, encouraging the model to attribute importance structures that a human might consider relevant. This can aid comprehension but may reduce fidelity to the model's actual behavior (Miró-Nicolau et al., 2024), which does not necessarily align with human reasoning.
On the other hand, methods like Deconvolution (Zeiler & Fergus, 2014), Integrated Gradients (IG)(Sunaararajan et al., 2017), and LRP(Bach et al., 2015), which attribute importance to features (pixels), aid understanding deep learning models across various applications (Borys et al., 2023; Dharshini et al., 2023; Chaddad al., 2023), serving as better approximations of model behavior (Borys et al., 2023), and locally explaining decisions. However, they often lack interpretability due to their pixel-level explanations, which be difficult for humans to understand (Kim et al., 2018). Different tochniques tend to prioritize either faithfulness to the model's behavior human interpretability, making it challenging to balance the two.
We explore the tradc-off between model fidelity and human interprotability in explaining Convolutional Neural Networks (CNNs) (Figure 1). We introduce æAiTrees, a framework that combines hierarchical segmentation with region-based explanation methods to produce human-friendly, multiscale visualizations inspired by Multiscale Interpretable Visualization (Ms-IV) (Rodrigues et al., 2024). Unlike conventional region- based XAI methods that rely fixed segmentation scale, #AiTrees leverages hierarchical segmentation to adapt region sizes across different. levels of abstraction, mitigating the limitations of overly small or excessively coarse regions.
In addition, we propose a model-based segmentation strategy that uses pixel- wise attribution methods to approximate the model's "visual perspective," " enabling the transformation of pixel-level explanations into coherent region-based interpretations.
To assess whether the proposed hierarchical and model-adaptive design effectively balances model fidelity and human interpretability, conduct series of quantitative evaluations comparing our approach with six representative explanation techniques, including perturbation-based, heatmap-based, concept-based, and attribution methods. The evaluation relies on Uhree complementary metrics: occlusion, which measures the impact of removing regions identified as important; inclusion, which evaluates whether these regions are sufficient to support correct classification; and novel metric, Pixel Impact Rate (PIR), designed evaluate the specificity of the explanations by penalizing large regions being considered important. Across these metrics, framework achieves competitive performance with strong baselines such XRAI.
2

---

We propose zAiThees, a hierarchical segmentation explanation framework that aggregates importance across multiple spatial scales, enabling humans to better understand and diagnose model bias; We evaluate zAiTrees using several complementary metrics occlusion, inclusion, and the newly proposed Pixel Impact Rate (PIR) to assess both the sufficiency and specificity of explanations; Published in Transactions on Machine Learning Research (01/2026)
Original
Integrated
chime,, belt gong
Resnet
chainljk
Resnet
dishwasher
Resnet
Imagenet classes
prizon,
saltshaker
VGG
Resnet
Figure 2: Explanations of six image classes misclassified by VGG-16 Resnet18 models trained ImageNet. We compare well-known xAl method explanations with one configuration of IAiTrees: Tree-Occ. Methods such as Integrated Gradients are noisy and difficult to interpret. Shapes such as the grades and the fence seem to be better highlighted by Tree-Oce, which is helpful for interpretation. When compared to highly interpretable methods like LIME, Tree-Oce avoids the mistake highlighting the cat when the models predict classes such as dishwasher, saltshaker, and hamper.
Most importantly, we conduct a qualitative user study in which participants are presented with visual explanations generated by different explanation methods. The study evaluates, under three distinct bias scenarios, which methods best support (i) the detection of bias (i.e., determining whether the model is biased) and (ii) the identification of bias (ie., understanding the nature of the bias). Across nearly all scenarios and evaluation questions, proposed framework achieves the better performance.
The key contributions of this paper include:
3. We conduct human-subject study showing that our explanations more effectively help users detect and identify biases under multiple bias scenarios.
We organize the paper as follows: in Section 2, we present some prior research on xAI. Section 3 outlines the preliminary concepts used in our framework, while Section we provide detailed explanation of our methodology. In Section we present and discuss our experimental results. Finally, we conclude and discuss possible future research directions
2 Related work
Classification problems and xAI: One fundamental task in machine learning is classification. The basic concept involves working with a training dataset, denoted as DS = (I,, GTv) i Nbdmm which consists of pairs of images I, and their associated labels GT, Each label belongs one E [1, NbClasses). The goal is train a model, denoted E, to effectively distinguish between different classes within the dataset.
of classes represented by
3

---

Published in Transactions on Machine Learning Research (01/2026)
In this configuration, we express E as E o the combination of two elements: an E responsible for converting each input image I into feature vector, and Ecleessf which analyzes these features to classify the images. The outcome of this process, referred to as the "logit" for image Zi, is a vector out, E R NCCasses that signifies the activation levels across various classes. Typically, we apply a Softmaz layer to out, to determine the class with the highest aclivation, ideally aligning with the ground truth label GT, for perfect. classification.
Pixel-wise explanations: In neural networks, optimizing the model E involves the backpropagation process. Exploiting this process, certain explainable Artificial Intelligence (xAI) methods like Integrated Gradients (Sundararajan et al., 2017), Guided-Backproopggtti
(Springenberg et al., 2015), and Deconvolu- tion (Zeiler & Fergus, 2014) utilize it to identify input features that enhance the response of a specific class, aiming to maximize the value of a particular position in the output vector out, Consequently, attribution maps are generated, illustrating pixel-level explanations, as depicted Figure 2 for Integrated Gradients (IG).
region-based explanations: Additional techniques like Sensitivity Analysis (Zeiler & Fergus, 2014), LIME (Ribeiro et al., 2016), and SHAP (Lundberg & Lee, 2017) utilize occlusions of image regions to assess the network's sensitivity to cach region within an image. These methods provide explanations at region level rather than a pixel level, as illustrated in Figure 2 for LIME. More recently, a region-based technique, XRAI (Kapishnikov et al., 2019), proposed to combine Integrated Gradients (Sundararajan et al., 2017) and perturbation-basel approaches to generate saliency maps as explanations.
Concept-based explanations:
However, many of these techniques focus on explaining individual samples separately, which limits our understanding of how the model behaves globally across various scenarios. That is why methods like TCAV (Kim et al., 2018), ACE (Ghorbani et al., 2019), Explanatory graphs (Zhang et al., 2018), LGNN (Tan et al., 2022), and Ms-IV (Rodrigues et al., 2024) aim to comprehend the overall behavior of the model. In particular, Ms-IV also considers the impact of occlusions, not on individual predictions, but on the model's output space.
Neural-symbolic explanations: Techniques such as DCR (Barbiero al., 2023), X-NeSyL Hioz--ozrri et al., 2022), and the approach proposed by (Ngan et al., 2023) adopt neural-symbolic strategics to explain the decisions of neural networks within specific contexts. These methods enhance transparency by translating low- level neural activations into high-level concepts, logical rules, or hierarchical structures, making explanations more aligned with human reasoning. However, this often comes at the cost of increased complexity, as such approaches may require mamually crafted symbolic rules or domain-specific annotations to generate meaningful explanations.
3 Preliminaries
To ensure a thorough understanding of the sequel, we provide in this section the general techniques and metrics employed during this work. In subsection A, we provide brief overview of the selected hierarchical segmentation techniques, highlighting their significance. In subsection B, we shortly present the occlusion- based metrics used in the construction of our methodology.
A. Segmentation techniques: As an important step for our framework, we employ image segmentation algorithms that decompose images into more interpretable structures, enabling better human understanding and interpretation. We specifically employ hierarchical segmentation techniques due to their capability to decompose images into multiple levels of detail, from fine to coarse, mirroring how humans naturally perceive objects: initially observing the overall structure before delving into the finer details. A hierarchical segmentation algorithm produces merging tree, that indicates how two given regions merge. In this paper, we use the tree structures available in the Higra package (Perret et al., 2019; 2018): Binary Partition Tree (BPT) and Hierarchical watershed. See details in A.I.
B. Occlusion-based metrics: In this work, we use two metrics to generate our segmentation based on the model explainability: (i) Occlusion, which the impact of occluding an image region on its classification output, and (ii) CaOC, which is the intra-class impact of occluding an image region that employs a sliding metric that ranks images based on the highest activations for given class. This sliding metric measures
4

---

Published in Transactions on Machine Learning Research (01/2026)
1. Hierarchical segmentation
2. Attribute computation
Tree shaping
Hierarchical visualization
Original image
Segmentation tree
Segmentation tree
Segmentation tree
Aggrogading all
Weights for initial graph
Occlusion nodes
New attributes ger occluded region
Regionel importance map
Odtestett
Outbra
Hicrarchical segmentation method
A
Shaping T into T using the attributes
Segmnentation
B
occlusion impact
Finel visuelization
New attributes per occiuded region
Figure 3: Our framework JAiTrees operates through four key steps: 1. Generate a segmentation hierarchy using either the image's edge map for human-based segmentation or pixel-wise importance based on xAI Systematically occlude each region of the segmentation
techniques for model-based segmentation. evaluate its impact on the model's decision, obtaining an occlusion attribute for each region. Assess the persistence of the occlusion attribute using shaping approach (Xu et al., 2015; 2016). 4. Aggregate the contributions of each region from the highest to the lowest level of the tree to create comprehensive multiscale visualization.
the movement in this ranking after occluding a region of the image (detailed in A.2). Although the main experimentation uses these methods, we present in A.7 a framework's 's variation using LIME to show TAiTrees versatility with other types of xAI techniques. Explanations example Figure 9.
4 Methodology
In this section, we outline our four-step methodology (Figure 3): (1) hierarchical segmentation, (2) attribute computation, (3) tree shaping, and (4) hierarchical visualization. In step we convert the data into a hicrarchical reprosentation, ercating various regions different scales in the image. In step 2, evaluato some xAl-based altributes
(B) on the regions. In step 3, we assess the importance of the region altributes. Finally, in step 4, we explain how to generate a visualization map from the importance of the attributes.
1. Hicrarchical segmentation:
Intuitively, any hierarchical segmentation algorithm works by iteratively merging first the pixels, then the regions, according a similarity criterion. In this paper, we test ways for measuring the similarity: human-based and model-based
The based approach relies on the Structured Edge Detection (SED) algorithm (Dollár & Zitnick, 2014), which captures complex edge patterns and produces precise edge maps, in accordance with human intuition.
The model- based approach uses a visual representation of the image's pixels most influential a model's decision. Although less intuitive for humans, this approach helps understand how the model reasons. We test pixel-wise explainable AI methods: Integrated Gradients (IG) (Sundararajan al., 2017), Guided- Backpropagation (Springenberg al., 2015), Input Gradient (Shrikumar al., 2017), and Saliency (Si-
5

---

Published in Transactions on Machine Learning Research (01/2026)
Eyes outside hierarchy
Eyes Inside hlerarchy
Figure 4: The hierarchy aids to discriminate similar imporlant regions. Example of the method's behavior with the same structure inside and outside a hierarchy. The cat's eyes were replicated outside the cat's face. However, the importance of each region is the combination of the importance of each hierarchy part. Therefore, the cat's eyes inside the face (an important hierarchy region) score higher, as evidenced by the lighter regions in the right image.
monyan & Zisserman, 2015) (all from Captum framework). The methods were chosen because of their availability (in frameworks like Captum) and for their higher fidelity as pixel-wise importance attribution based on gradients (Miró-Nicolau et al., 2024).
Using such a similarity criterion, we obtain a hierarchical segmentation, which can be represented as a tree T, completing the first step of our pipeline. See Figure 3, first column.
2. Attribute computation:
The segmentation tree generated in the previous step provides many segments. We assess the model's response on each segmented region in the tree, for all regions large enough. We apply metric to evaluate the occlusion impact caused by each region. These occlusion scores reveal the influence of each segmented regions on the model's output. The metric employed to assess the impact of regions can be any occlusion-based metric. See Figure 3, second column.
3. Tree shaping: To assess the importance of the nodes' attributes, it is not enough to simply take the regions with the highest attributes: there are different levels of the hicrarchy to be analyzed. Instead, we rely on a process called shaping (Xu et al., 2015; 2016). The main idea is look at the undirected, vertices-weighted graph G, whose vertices are the node of T, whose edges are formed by the parent-children relationship in T, and whose weights are the attributes of the nodes. We now look at the level-sets of G. A vertex of G (a node of T) is important according to its persistence in the level sets of G. More precisely, a connected component is born when local maximum of the attribute appear; when two connected components merge, one of the two maxima disappear, and the time of life of this maximum is its persistence. We can compule such persistence by building new tree T on G, T is the tree of all the connected component. of the upper-level sets of G. The persistence of node of T is easily computed on T' by computing the length of the branch it belongs to. We refer to Xu et al. (2015; 2016) for more details. See Figure 3, third column.
4. Construction of the hierarchical visualization: With T from the previous step, we now produce a visualization of the important regions. Using the persistence of a node directly for visualization can yield conflicting results for interpretation. Consider an example where we want to generate explanations for a model that classifies images of dogs. The persistence might indicate that eyes are the primary features for correct classification. If the image under scrutiny shows a dog with its owner, the persistence might erroneously highlight the eyes of both the human and the dog relevant, which misleading since only the dog's eyes should matter (in an ideal, unbiased model). To avoid such effect, we recursively sum the persistence of each node from the root to the leaves of T'. This ensures that smaller segments inherit the importance of their parent nodes. In our example, if the parent segment of the eyes is the entire face, the dog's face carries importance for the model's decision, while the human face does not. By adding the dog's facial region information to the eye segments, we ensure the dog's eyes are prioritized over the human eyes and, therefore, become more prominent in the explanation. This process aggregates the importance of various scales of the image into the pixels, resulting in hierarchical, multi-scale, visualization. We show an example in Figure 4. We use this aggregated persistence as the final score for each region of the hierarchical segmentation. We
6

---

Published in Transactions on Machine Learning Research (01/2026)
explanation faithfull?
Model
Model
High impact?
Model
Low Impact?
Figure 5: Our quantitative experiments aim answer the question: Ts the explanation faithful? Tn (1), we occlude the selected important regions and evaluate the impact using the Pixel Impact Rate (PIR) based on the model'ss response. In (2), we evaluate the inclusion of only the important regions by analyzing their Softmax and Accuracy Information Curves.
select minimum importance score, and retain the regions accordingly. We superpose the retained region on the original image to generate the Final visualization (Figure 3, fourth column).
5 Experiments and results
We evaluated the methods using two architectures, VGG-16 (Simonyan & Zisserman, 2015) and ResNet18 (He et al., 2016), trained on three datasets: Cat vs. Dog (Cukierski, 2016) (RGB images size 224x224), CIFAR- 10 (Krizhevsky, 2009; Krizhevsky et al., 2009) (RGB images size 32x32), and ImageNet (Deng et al., 2009) (RGB images). Explanations were generated for 512 images from the Cat vs. Dog dataset, 10,000 images from the CIFAR-10 dataset, and 100,000 images from the ImageNet test set. A detailed description of the methods' parameters and datasets provided in A.3, A.4 and A.5. We organize our experiments and results into two categories: quantitative and qualitative analysis. In the quantitative analysis, of experiments utilizing the metrics discussed in Section 3 to assess the impact of image region occlusion of various explainable frameworks. During our qualitative analysis, we do a more subjective examination, evaluating the human interpretability of the explanations generated by the models. The experiments were conducted on GPU (NVIDIA Quadro RTX 8000 48GB). We discuss some limitations in ??.
conduct series
5.1 Quantitative evaluations
We selected state-of-the-art region-based methods as baseline (B) to be compared: Occlusion, Grad-CAM, LIME, Ms-IV, and XRAI (configuration each mothod A.4). Although ACE presents good concept-based explanations, we only use it the human evaluation experiments because, as a global explanation method, it not directly comparable to the local ones in these quantitative experiments (more details in A.6). We compare the baseline methods with different configurations of proposed methodology, showing its adaptability (all the configurations in A.3 and A.5). This is done to explore variations, including different sizes of minimal regions in the visualizations, pixel weights for graph construction in segmentation, and methods for generating hierarchical segmentation.. We show eight of them here compared to the baseline methods. When we refer to Tree- Crre-CaOC.oo or Tree W-CaOC, we mean the numan-based segmentation (edges' map) using Watershed area and CaOC as occlusion metric. When we refer to IG-Tree-Oee Oce or W--TreeW-Occ, we mean the modcl-based segmentation (using Integrated Gradients (IG) attributions) using Watershed area and Oce (simple occlusion Equation (1)) as occlusion metric. When we refer B--,, we mean the model-basee segmentation (using Guided Backpropagation (BP) attributions) using BPT and Occ occlusion metric. In the results, we refer to configurations using watershed as group C1 and using BPT as group C2.
In C1, we present results for minimal regions of 500 pixels (Cat vs. Dog and ImageNet) and 64 pixels (CIFAR-10), utilizing edges and Integrated Gradients (IG) as pixel weights, and the watershed-by-aree hierarchical segmentation. This configuration was selected for its stability across different minimal region sizes in the datasets, and its visualizations were used for human evaluation. While C2 presents results
7

---

Published in Transactions on Machine Learning Research (01/2026)
Table 1: Percentage of images with the original class changed after the exclusion of selected explanation regions. We test two configurations of our methodology (C1 and C2 other configurations in Supplementary Materials) against four region-based baseline methods, Occlusion, Grad-CAM, LIME, Ms-IV, and XRAI, in two architectures, VGG-16 and ResNet18, and datasets, Cat vs. Dog, CIFARIO, and ImageNet. We expect higher percentage of class change (Ch.) when the region is excluded. Same column shows images maintaining the original class when the output was reduced, and Total is the sum of class change (Ch.) and class reduction (Same). We compare the each method to the best configuration (BP-TreeB-Ocs showing the p-score in brackets (Menemar test).
Cat vo. Doy
Cloclo
Insggestt
of inagg
Vitrt
VOG
VISE
RecNNe
Ch.
Saser
Thxal
Ch.
Saear
Cl.
Thc.ll
C1.
Sauer
Thtal
Cl.
Saeer
Theal
Sauur
Traal
Ovalldda
E3G23-31
5.59 [540
0.50
13.31
1.99
18.50
ES6 (MU
(540
8.54
(130 23:31
016 23.31
6:06
18.20 00.01
B66 (54))
5.59
0:20 13.11
8.5 (04)
Gro4-GAM
EX013.31
1.9
0.88
E.34 but
8.54
21.11
[.8)1.8)
6158
11.55
0A8 201)
[8k8h 01.00
(0.90
XRAAI
010.3131
0188
0:35
029
(1/10
WeeW Care
8.61
015
01:01135
Tou Oue
6335
0..65
0.15
0.15 13:31
666
6156
01.00
13.31
1.48
0.75
02011.81
6:45 (740-133
0.95
5.21 006
8.49
022 33.01
0.76
11.00
12.01
0.02 (000)
8.75
01:01
8.55 (000
0.55
Ii-bbeeW-Dee Webeer
8:48 (1-7311)
0.07
16.22
01:1--80)
1481 1233:14
0..0
172 08
0.35 (144)
0.54
1.79 01.00
012 11.81
0:93
19.45
09.01
0:23 2016
9.95
16.05
01.09
8:55 (010)
0.54
1:33 03.00
1:28
0.11
01
6:43 (M46)
0.99
1.15 016
0.78 (1:0)
8.41
0.17 33.9)
0:2011111
0.06
0335
01
6:19 2016
8:13
0:35
13:91
8:41 (040
0.99
43
FP Towdlt Exor
Traii-Dcc
8.36 (9
0. 63)
064 36.0
0.96
0.30
1043
(3.5 10)
(2064)
0:08 25:591
8:83 (6.81
0.88
0.76
B:8 18.80
04 (1/)
0.31 3.00
8.33 0000
0.50
8:43
1020 10.00
011 29-0)
0.21 65)
0.17 39.01
6:06
655
10.87
0111
00,01
658
6558
(000
(010)
0.95
1000
11-93
00.00
0:11 0046
8:35 0006
0.55
Thalll Due
0:35
0.08
633
0:35
14.65
0.96
0.00
0.78
1036 00.00
8:15
1634
00:00
0:14 (009
01:00
for minimal regions of 200 pixcls (Cat vs. Dog and ImageNet) and 4 pixels (CIFAR-10), using edges and Guided Backpropagation (BP) as pixel weights, and the BPT tree. This configuration yielded the highest performance. In A.5, we provide comprehensive results for other configurations. Here, we propose four main quantitative evaluations, (i and ii) inspired by feature removal (Covert et al., 2021) and (iii and iv) inspired by Performance Information Curves (PICs) (Kapishnikov et al., 2019): (i) Exclusion of important regions; (ii) Inclusion of important regions; (iii) Softmax Information Curve (SIC); and (iv) Accuracy Information Curve (AIC). For (i) and (ii), we used the McNemar test (McNemar, 1947) to compare each method with the best configuration and determine whether there were statistically significant differences between the results. We show the main idea of this evaluation in Figure 5, and the details are discussed below:
Exclusion of important regions: important regions that explain the prediction of a model, we performed occlusion of these regions, in order to measure the impact of each selection. For methods that assign scores to regions, we masked the 25% highest scores (this excludes LIME, which inherently provides information to directly mask cach region a detailed explanation is included in A.6).
Given that each region-based based explainable Al (xAI) method identifies
The first idea for the metric was to calculate the impact on the logits after occlusion. However, any kind of perturbation can affect the logits and not necossarily the classification. In this particular case, since we are dealing with a classification problem, we consider the class change as the main evidence that an important image region has been occluded. Therefore, the values from Table 1 Ch. is the percentage of class changing images, Same is the percentage of images with same class prediction but with reduced logits (reduced classification certainty), and Total is the percentage of all images with the class negatively impacted by the removal of important regions (sum of Ch. and Same)
In Table 1, we present results (Ch., Same, Total) for each explainable technique (B Baseline, C1, and C2 our proposition) applied to a network (VGG or ResNet) classifying images from a dataset (Cat vs. Dogs, Cifar10, or Tet) Higher Ch. values indicate that the identified regions are more class-representative. High Same values complement Ch., suggesting that the best results are shown by higher Ch. and Same values. Thus, while Total sums Ch. and Same, the optimal result. is reflected by initially higher Ch. and then higher Same values.
Table 1 shows that bascline methods like Occlusion achieved Total values above 80%, but our C2 config- urations delivered the best results, particularly for class changes (Ch.). Our methods had over 60% class changes for Cat vs. Dog images, exceeded 80% for CIFAR-10, and 70% for ImageNet. Among baselines, LIME and XRAI performed best. The results highlight the superiority of our C1 and C2 configurations in identifying impactful regions, achieving robust performance across datasets, and providing more accurate insights into deep neural network interpretability.
8

---

Published in Transactions on Machine Learning Research (01/2026)
Table 2 presents results from second experiment addressing reduced precision in explanations where methods highlight the entire image as important, potentially leading to class changes upon occlusion. To address this, we propose the Pixel Impact Rate (PIR), a metric that quantifies the per-pixel impact on class activation during occlusion. Unlike the percentage of class change, PIR distinguishes whether changes are caused from full or near-full image occlusion (details in A.5 Equation 2). Higher PIR values indicate significant average impact per pixel, while lower PIR suggests occlusion of larger portions or the entire image (imprecise explanations). The table summarizes the average (avg) and standard deviation (std) of PIR across networks, techniques, and datasets.
Table 2: Pixel Impact Rate (PIR) of the chosen regions. The metric is the rate of the impact under occlusion (difference between the original class output and the output under occlusion) by the number of pixels of the occlusion mask. We test two configurations of our methodology (C1 and C2 other configurations in Supplementary Materials) against four region-based baseline methods, Occlusion, CAM, LIME, Ms-IV, and XRAI, in two architectures, VGG-16 and 18 and datasets, Cat vs. Dog, CTFAR10, and ImageNet. We expect higher values, on average, for PIR, meaning each occluded pixel has a high impact.
Cat vs. Dog
Cifar1d
Imagenet
PIR
VGG
ResNet
VGG
ResNet
VGG
ResNet
avg
std
avg
sid
avg
std
avg
std
avg
std
avg
std
Ovclusion
4.60e-03
4.06e-03
L55e-03
1.28e-05
6.95e-02
1-48e-01
9.67e-02
1.
L116-02
1.1e-012
8. 6.02e-03
7.08e-003
Grad-CAM
1.12x-03
1.026-13
76-004
2.07e-04
6.30-003
1.346002
5.380-13
1.46o-03
3.05-003
3.16-403
1.11e-03
8.026004
B
LIME
9.03c-04
1.1c-03
3.47o-04
3.89c-04
6, 6,75e-03
3.276-03
6.41e-03
3 3.25e-03
2.11e-00
2.50c-03
1.58c-03
1.75e-03
Ms-IV
4,30-004
4.7de-04
1483-04
2.15e-01
1.16c-02
1.4e-022
1.18c-02
1.33e
9.73e-04
1.1e-05
7.21e-01
7.50e-04
XRAI
L16e-03
9.92o-04
4.44o-04
6.32e-04
2.09e-02
1.776-02
1.92c-02
2.02c-02
3.05e-00
L.09-412
2.04c-03
5.90e-03
Tree-CaOC
3.61e-01
4.70e00
1.92-04
2.
4.73e-03
1.0e-02
0,50ee00
1.05e-02
1.16e-03
1.0e-05
1.10e00
1.47e-03
Tree-Oee
3.66e-04
5.30e-01
1.69-004
2.52-01
9:20e-03
1.32-022
9.88--05
1.21e-02
1.10-188
1.50-03
1.05e-03
1339-03
C1
CG-TToo-CaOC
3.46-44
3.48-004
2.36--04
3.00e-04
9:35e-13
1.30-U2
9.516-03
1:27o-02
1.54-08
1.83--15
1.46--03
1.66--13
IG-Tree-Oee
3.10c-04
3.61e-04
2.11e-04
3.050-04
L.69c-02
1.56x-02
1.63c-02
1.40-022
1.526000
1.72-403
1.4Gx-00
1.58e-03
TreoB-CaOC
2T6e-04
2311-04
1.26-04
2.26-004
8.9236-13
1906-12
1.12--02
2006--12
7:20-04
1.08-033
6.76e-14
9.63-444
TreeB-Oee
2.260-04
3.32-04
1.03-004
1.816004
1.14e-02
2.06o-02
1.15e-02
2.00c-02
5.836-04
8.19c-04
5.13c004
7,270-44
C2
BP- CaOC
3.23c-03
3.59c-02
2.58e-03
1.81-002
1.94c-01
3.876-01
1.94c-01
3.52e-01
1:436-02
7.37c-02
1.19c-02
5.15e-02
BP BP-TreeB-Oce
8.64e-04
1.60e-02
1.18e-03
8.90c-03
1.10e-01
1.3c-011
1.18e-01
4.1ee01
4.51e-03
3.13e-02
3.25e-03
2.35e-02
In the PIR experiments (Table 2). C2, particularly BP-TreeB-CaOC, achieved the highest average PIR values, with Occlusion and XRAI also performing well. These methods demonstrated strong region specificity, enhancing the impact of occluded pixels. However, it is also important to consider the method's stability across different images, indicatod by smaller PIR standard deviation (std). While C2 presented higher PIR, C1 showed greater consistency (smaller std).
Inclusion of important regions: Additional experimentation was conducted to demonstrate method's capability to identily an image region with sufficient. information for the original class. The goal of this experiment is to determine whether the selected important region, when the only one left unoccluded in the image, can maintain the classification in its expected class. This experiment clucidates the critical rolc of these identified regions, providing strong evidence that they indeed contain essential information for accurate classification. We occluded all regions in the images except. for the one selected by each method. We then calculated the percentage of images that changed class. The results are presented in Table 3. Lower percentages indicate better performance, as they mcan that smaller percentage of images changed class, demonstrating that the chosen regions were sulficient to preserve the class for most of the images.
Table 3 demonstrates that both LIME and our configurations (C1, C2) effectively identify regions that describe a class. However, BP-TreeB-Oce outperformed LIME, with fewer images changing class, indicating it provides more essential information for class attribution. Additional insights include: Occlusion com- binod with our methodology yields superior local explanations, and "modcl"-basel" sogmentation enhances explanation fidelity. Overall, our method outperformed traditional xAI baselines, including LIME, which still delivered consistently good results.
SIC/AIC for hicrarchy evaluation: As explained in Section 4, the hicrarchy of our explanation is combined by summing up importance regions values. Therefore, to select different hierarchies it suffices to filter by different scores. In this line, we evaluate our methodology by imposing different thresholds for the explanations. Inspired by the metrics Softmax Information Curve (SIC) and Accuracy Information Curve (AIC) proposed by Kapishnikov et al. (2019) we calculated the Softmax and Accuracy curves by including only selected image regions as model input. To preserve the original data, distribution, we integrated these
9

---

Published in Transactions on Machine Learning Research (01/2026)
Table 3: Percentage of images with the original class changed after the inclusion (exclusively) this same regions. We test two configurations of our methodology (C1 and C2 other configurations in Supplementary Materials) against five region- based baseline methods, Occlusion, Grad-CAM, LIME, Ms-IV, and XRAI, in two architoctures, VGG-16 and 18sNet18, and datascts, Cat vs. Dog, CIFAR10, and ImageNet. We expect lower when the region in included. We compare each method to the best configuration (BP-TreeB-Occ) showing the p-score in brackets (Menemar test).
% of images
vs.
Dog
Cifar10
Imagenet
VGG
RosNet
VGG
ResNet
VGG
ResNet
Occlusion
0.47 [0.0)
0.50 (0.0)
0.89 (0.0)
0.50 (0.0)
0.90 (0.0)
0.99 (0.0)
Grad-CAM
0.51 (0.0)
0.30 (2.35-5)
(0.0)
(0.0)
(0.0)
B
LIME
0.16 (2.02-10) 10)
0.30 (6.44-5)
0.44 (0.14)
0.49
(2.62-9)
0.93 (0.0)
0.95 (0.0)
Ms-IV
0.20 (3.11-14)
0.54 (0.0)
0.78 (0.0)
0.81 (0.0)
0.88 (0.0)
0.92 (0.0)
XRAI
(0.0)
0:11 (2.03-14)
0.71 (0.0)
(0.0)
Treo-CaOC
0.26 [0.0]
0.32 (3.34-7)
0.80 (0.0)
0.85 (0.0)
0.06 (0.0)
0.97 (0.0)
C1
Tree-Oce
0.17 (0.0)
0.23 (0.0)
0.54 (0.0)
0.61 (0.0)
0.90 (0.0)
0.91 (0.0)
IG-Tree-CaCC
0.41 (8.12-11
0:43 (0.95)
(0.0)
(0.0)
(0.0)
IG-Tree-Oec
0.19 (1.37-12)
0.42 (0.0)
0.68 (0.0)
0.69 (0.0)
0.92 (0.0)
0.93 (0.0)
TroeB-CaOC
0.16 (1.8-9)
0.22 (0.15)
0.79 (0.0)
0.81 (0.0)
0.95 (0.0)
0.96 (0.0)
TreeB-Occ
(0.0)
(8.44-7)
(0.0)
0.51 (0.0)
(0.0)
(0.0)
BP-TreeB-CaOCC
0.38 (3.7-5)
0.28 (0.79)
0.87 (0.02)
0.86 (0.01)
0.97 (0.0)
0.58 (0.0)
BP-TTreeB-Oce
0.04
0.18
0.45
0.53 0.51 (0.0)
0.50 (0.0)
important regions back into a blurred version of the original image (details in A.5). The regions were sclectod based on thresholds of 0.5%, 1%, 2%, 3%, 4%, 5%, 7%, 10%, 13%, 21%, 34%, 50%, and 75% percent, representing the most significant region values according to each evaluated xAI method. These thresholds, represented the x-axis, indicate the percentage of important regions required to affect accuracy and class activations. Figure 6 shows the results for 1,000 randomly selected images for AIC (due to time consumption restrictions time analysis in A.5) from the ImageNet dataset and VGG16 model, with additional results for ResNet18 and the Cat vs. Dog dataset provided in A.5.
Aggregated Softmax
Aggregated Accuracy Carve
JEE-O Mo-ry RUCHE
SAAL ALCHOARG
AUCH0400
Octkakee AUCHE
Touck Dcc
AUCH6440
AUCHE
TO OCL ALE-11
THA--CAC AMC-- BIU
Theett CLOC, AUCHe
RONIVICCAIC ALCI- NIS
b)
Figure 6: Softmax (a) and Accuracy (b) when including regions filtered by different percentage thresholds of most important scores. We evaluate each threshold as a hierarchy level in eight configurations of IAiTrees (C1 and C2), in a bottom-up approach (from smaller highly important regions to the bigger structures). We compare these configurations to the baselines: LIME, XRAI, Grad-CAM, Ms-IV, and Occlusion, by filtering the maps using the same threshold. The curves are averaged across 1,000 randomly selected images from Imagenet dataset. AUC values included in the graphs. BP-TreeB-Oce and BP- BP-TreeB-CaOC considerably surpassed the other curves. However, notice good early behavior of the methods except for Grad- MAdM Ms-IV and Occlusion.
Based on Figure 6, the methods BP-TreeB-CaOC, BP-TreeB-Oce, XRAI, and LIME achieved the highest AUC scores, in that order. When considering more restrictive levels of the hierarchy (using 0.5% and 1.0% thresholds), most methods- eecod-txccptttt for Grad-CAM, Ms-IV, and configurations, along with LIME and XRAI, effectively identified the most important regions, with C2 slightly outperforming CI. However, C2's greater variation in PIR results (Table 2) suggested that it
ceresion-erfoomee well. xAiTrees
10

---

Published in Transactions on Machine Learning Research (01/2026)
is the explanation interpretable?
classification
Model
appear
Bios 3 exxplanation
Detections
(II) identtfication human!
What is important?
Unknown: explanaltion
Figure The objective of this qualitative experiment is to evaluate the interpretability of the methods under biased conditions. We ask participants to identify what appears to be the important part in each explanation, and we categorize their responses as follows: (i) the participant detected the presence of bias, (ii) the participant identified the bias, (fii) the bias not detected, (iv) the explanation was unclear (unknown).
often highlighted larger regions, diluting per-pixel impact. To support qualitative experiments and simplify human analysis, chose the C1 group to minimize region size. For human analysis, did not filter the explanations, but we indicated the hierarchy levels by region brightness, making distinctions clear.
5.2 Qualitative analysis
As qualitative experiments, we want to visually evaluate the explanations for different interpretability tasks. In this section, we perform experiments to (i) identify reasons for misclassification of images, and (ii) evaluate explanations through the human interpretation of blased-trained networks.
We searched for examples that were misclassified by models (VGG-16 and Resnet18) trained on ImageNet (Deng et al., 2009). Figure shows the explanations generated by Integrated Gradients, CA,, Occlusion, LIME, and Tree- Oce (500 pixels minimal region) of six images incorrectly classified. In Figure 2, the first column displays classes (such as chime, fence, dishwasher, among others) alongside examples of misclassified images. These images should have been classified cat or dog. We then apply methods used in previous quantitative comparisons to generate visual explanations for why these images were misclassified. The figure illustrates that methods like Integrated Gradients, Grad-CAM, and Occlusion (Oec) may cause confusion in preciscly identifying what caused the misclassification and may lead to poor human interpretation (we properly evaluate this in next experiment Human evaluation in bias analysis). Although I.MEE and our proposed Tree-Occ method can pinpoint interesting regions, the Tree-Oce method better illustrates the motivation behind misclassified results, as evident in the last column. For instance, in the fence example, it highlights the diamond pattern found fences, while in the dishwasher example, it focuses solely on the sink region, disregarding the cat. Considering the hierarchical characteristic of our methodology, we can perform a deeper analysis of the explanations by selecting regions by the percentage of importance to be visualized. Examples in A.6.
Comparison of misclassified images:
Human evaluation in bias analysis: As previously ioned, we used the configuration C1 for human- interpretation evaluation (Figure 7). We trained three Resnet18 modcls subjected to data bias: (a) Bias model trained with dogs and only cats on cushions; (b) Bias 2 model with cats and only dogs with grids; (c) Bias 3- - and a model with dogs and only cats with humans (details of validation accuracy and visualizations in A.6). We presented the same five image visualizations (from corrected classified images by the biased class) for the baseline methods and the methods from C1. We intended to verify if: (i) humans can detect the wrong focus given based on a class prediction (Detection); and (ii) humans can recognize which was the cause of the bias (Identification).
To test (i) and (ii), for each Bias (a.b, or c) type we produce for each of the xAI methods an explanation image. By presenting five image explanations (the same images) for each of the xAT methodologies, asked
11

---

Published in Transactions on Machine Learning Research (01/2026)
volunteers, based on the explanations provided, what they think the highlighted regions referred to (generated explanations and extra experiments in A.6).
Table 4: Human evaluation assessed bias detection and identification using five image explanations from biased datasets: (1) dogs with only cats on cushions, (2) cats with only dogs on grids, and (3) dogs with only cats and humans. Metrics included bias detection (Detection (i)), bias identification (Identification (ii)), misunderstanding of explanations (Not identified), and focus on animals (Animal). Higher detection and identification rates were desired, with lower misunderstanding rates. Concept-aware methods like Ms-IV, ACE, and Tree-CaOC performed better, with Tree-CaOe using human-based segmentation achieving the best results across all biased datasets.
Gred-CAM
LIME
Ma.TV
ACR
Treus
TweeCGDDD
Detection (i)
24.4
0.0
31.7
14.6
31.7
0.0
19.5
47.4
12.5
0.0
Identification (II)
12.2
0.0
7.3
14.6
0.0
2.4
21.1
0.0
Bias 1
No blas
41.5
97.6
31.7
48.8
41.4
70.0
56.
26.3
52.5
87.5
Unknown
34.1
2.4
36.6
36.6
26.8
30.0
24.4
26.3
35.0
12.5
Detection (i)
7.3
0.0
0.0
7.3
24.4
58.5
56.4
57.9
42.1
29.2
Identification (ii)
0.0
43.9
41.0
47.4
34.2
26.8
Bias 2
No bias
81.1
100.0
86.6
61.0
48.8
2.4
12.8
10.5
23.7
34.1
Unknown
12.2
0.0
17.1
31.7
26.8
39.0
30.8
31.6
34.2
36.6
Detection (i)
35.0
17.1
43.9
51.2
29.3
9.8
41.1
56.4
50.0
55.0
Identification (ii)
19.5
12.2
32.4
36.6
4.9
33.3
22.5
37.5
Bias 3
No bias
29.3
70.8
29.3
4.9
24.4
26.9
7.7
5.2
2.5
2.5
Unknown 36.6
12.2
26.8
43.9
46.3
63.4
51.3
38.5
47.5
42.5
Table 4 presents the results of evaluating 41 individuals from diverse continents (South America, Europe, and Asia), fields (Human, Biological, and Exact sciences), and levels of Al expertise (ranging from no knowledge to expert, with over half being non-experts). We show some participants statistics in A.6. The experiment aims to identify effective methods for revealing trained-with biases. For each xAI method (IG, Grad-CAM, Occ, LIME, Ms-IV, ACE, Tree-Occ. Tree- CaOC, 2, and 3), we show the percentage of participants who detected, identified, or did not identify the bias the explanation. Detection indicates perceiving the xAI explanation as either background or reflecting the bias, while Identification denotes successful interpretation of the explanation as the induced bias. Not Identification refers to being unable to interpret the explanation. Higher percentages in the Identification row are desirable. If not, we prioritize high values in the Detection row. Lower values in the Not Identification or Animal rows indicate clearer human interpretation of our trained-with bias
IG-Tree- (G-Tree-CaOC) used to explain biases (1,
Table shows that IG and Grad-CAM struggled with interpretability, leading to many Not identified or Animal responses, indicating unclear explanations for imposod biases. Methods leveraging contextual or global explanations, such as Ms-IV, ACE, Tree-CaOC, and IG-Tree-CaOk CaOC, achieved better detection and identification results due to their global approach to model knowledge. However, this was not always enough for humans to provide complete interpretation of the model's knowledge. Tree-CaOC, combining global metrics (CaOC) and human-based segmentation, consistently achieved the best results for all three Bias categories, highlighting its superiority in enhancing human interpretability.
6 Limitations
The computational time is a limitation when using time-cxpensive methods to attribute region scores, such as LIME. We show the time comparison including the baseline methods in A.5 Table 15. That is why we limited our analysis to Tree-Oer Oce and Tree-CaOC.
The analysis was limited to specific task and datasct: classification of cats and dogs. Applying the method to other tasks, such as representation learning, or to different modalities, such as text, requires adaptations, which are discussed in Section A.7.
The proposed version of zAiTrees framework is dependent on the base methods used. Therefore, by using an odge-basod segmentation method, we will not obtain a semantic-based explanation, i.e., the final technique will inherit. the limitations of the base methods. Future works will be focused semantic segmentation.
12

---

Published in Transactions on Machine Learning Research (01/2026)
7 Conclusion
In this paper, we present a framework, xAiTrees, aimed at integrating multiscale region importance in model predictions, providing more faithful and interpretable explanations. Our approach outperformss traditional xAI methods like LIME, especially in identifying impactful and precise regions, in datasets such as Cat vs. Dog, CIFAR-10, and ImageNet. Qualitative analysis demonstrates that our Tree-Oce method better clucidates misclassification motivations and provides clearer, hicrarchical interpretations of model predictions. Techniques like Tree-CaOC, merging global-aware metries with human-based segmentation, excel in detection and identification tasks, achieving superior results in human interpretability. In summary, framework delivers highly interpretable and faithful model explanations, significantly aiding in bias detection and identification, and demonstrating its effectiveness in the field of explainable AI. Therefore, potentially aiding to reduce the societal negative impact that could be generated by deep learning models in high-risk decision-making process.
Reproducibility Statement
We release the code at https://githul: wom//aarolMeeiiiirrrraaaaooaaaaa wttprressonnnnniihioooeeeeeeeeee with_trees// and detail our experi- mental setup and disclose all hyperparametors in the Appendix.
References
Sebastian Bach, Alexander Binder, Grégoire Montavon, Frederick Klauschen, Klaus-Robert Muller, and Wojeiech Samek. On pixel-wise explanations for non-linear classifier decisions by layer-wise relevance propagation. PLoS One, 10(7): 2015. doi: 10.1371 /oo./ralll..oo..011.... 013014.....
Pietro Barbiero, Gabriele Ciravegna, Francesco Giamnini, Mateo Espinosa Zarlenga, Lucie Charlotte Magister, Alberto Tonda, Pietro Lió, Frederic Precioso, Mateja Jamnik, and Giuseppe Marra. Interpretable neural- symbolic concept reasoning. In Proceedings of the 40th International Conference on Machine Learning, ICML'23. JMLR.org, 2023.
Katarzyna Borys, Yasmin Alyssa Schmitt, Meike Nauta, Christin Scifert, Nicole Krämer, Christoph M Friedrich, and Felix Nensa. Explainable AI in medical imaging: An overview for clinical practitioners - beyond saliency-based XAT approaches. European Journal of Radiology, 2023. 2
Ahmad Chaddad, Jihao Peng, Jian Xu, and Ahmed Bouridane. Survey of explainable ai techniques in healthcare. Sensors, 23(2), 2023.
Jean Cousty, Gilles Bertrand, Laurent. Najman, and Michel Couprie. Watershed cuts: Minimum spanning forests and the drop of water principle. IEEE transactions on pattern analysis and machine intelligence, 31 18):1362-1374 2008. 16
lan C. Covert, Scott Lundberg, and Su-ln Lee. Explaining by removing: a unified framework for model
explanation. Journal of Machine Learning Research, 22(209), 2021. 8
Will Cukierski. Dogs vs. cats redux: Kernels edition. https://wwa wwt.glee.com/oometttttiotooo
2016. Accessed: 2024-05-20. 7
Jia Deng, Wei Dong, Richard Socher, Li-Jia Li, Kai Li, and Li Fei-Fei. Imagenet: A largo-scale hicrarchical image database. In IEEE Conference on Computer Vision and Pallern Recognition (CVPR), pp. 248 255.
leee, 2009. 7, 11
S. Priya Dharshini, K. Ram Kumar, S. Venkatesh, K. Narasimhan, and K. Adalarasu. An overview of interpretability techniques for explainable artificial intelligence (XAI) in deep learning- based medical image analysis. In International Conference on Advanced Computing and Communication Systems (ICACCS), volume 1, pp. 175-182, 2023. 2
Piotr Dollár and C Lawrence Zitnick. Fast edge detection using structured forests. IEEE transactions on pollern analysis and machine intelligence, 37(8):1558-1555 15700, 2014.
13

---

Published in Transactions on Machine Learning Research (01/2026)
Natalia Díaz- Rodriige,, Alberto Lamas, Jules Sanchez, Gianni Franchi, Ivan Donadello, Siham Tabik, David Filliat, Policarpo Cruz, Rosana Montes, and Francisco Herrera. Explainable neural-symbolic learning (x-nesyl) methodology to fuse deep learning representations with expert knowledge graphs: The monumai cultural heritage use case. Information Fusion, 79:58-84 -3,8883, 2022. ISSN 1566-2535. doi:
hutp:://dbi..org//1.11000. 1011/////0.00// 00121.99.222 URL httpp:://www.scienceeie httpi/errettcccccccceeeeeee com/sciencce/articlee p11/$$156625552001988..
Amirata Ghorbani, James Wexler, James Zou Y, and Been Kim. Towards automatic concept-based explana- Lions. In Advances in Neurol Informalion Processing Systems, volume 32, pp. 1 10, 2019. 4, 33
Leilani H. Gilpin, David Bau, Ben Z. Yuan, Ayesha Bajwa, Michael Specter, and Lalana Kagal. Explaining explanations: An overview of interpretability of machine learning. In 5th International Conference on Data Science and Advanced Analyties (DSAA), pp. 80-89, 2018. doi: 11L10//DDAA2201..00000 1
Anirudh Goyal and Yoshua Bengio. Inductive biases for deep learning of higher-level coguition. In Proceedings of the Royal Sociely A, volume 478, pp. 1-49, 2022.
Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition. In 29th IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 770-778, 2016. doi: N0.1109/CVPR2001
7
David H. Hubel and Torsten N. Wiesel. Receptive fields of single neurons in the cat's striate cortex. Journal of Physiology, 591,, 1959.
Andrei Kapishnikov, Tolga Bolukbasi, Fernanda Vi'egas, and Michael Terry. Xrai: Better attributions through regions. In 18th International Conference on Computer Vision (ICCV), pp. 4947-4956, 2019. 4, 8, 9, 18, 20
Been Kim, Martin Wattenberg, Justin Gilmer, Carrie J. Cai, James Wexler, Fernanda B. Viégas, and Rory Sayres. Interpretability beyond feature attribution: Quantitative testing with Concept Activation Vectors (TCAV). In 35th International Conference on Machine Learning (ICML), pp. 2677, 2018. 2, 4
Alex Krizhevsky. Learning multiple layers of features from tiny images. Technical report, University of Toronto, 2009.
Alex Krizhevsky, Vinod Nair, and Geoffrey Hinton. Cifar-10 (canadian institute for advanced research). Technical report, Canadian Institute for Advanced Research, 2009. URL https://www. cs toronto..... edu/ -kriz/cifee html. 7
Scott M. Lundberg and Su-In Lee. A unified approach to interpreting model predictions. In 31sl International Conference on Neural Information Processing Systems (NeurIPS), pp. 4768-4777, 2017. doi: 10.5555/ 3295522.32952230. 2, 4
Quinn McNemar. Note on the sampling error of the difference between correlated proportions or percentages. Psychometrika, 12(2):153-153) June 1947. doi: 40.1007/b6022299999 URL bf02295996. 8
Miquel Nicó-Nicolau, Antoni Jaume Capó, and Gabricl Moyà-Alcover. Assessing fidelity in xai post- hoc techniques: A comparative study with ground truth explanations datasets. Artificial Intelligence, 335:104179, 2024. ISSN 0004-3702.
doi: http:://abb.orr/10.000//// artint 202.1mtt.22..11179.. URL https:
//www www.sciiecedirectttt htt.o////c.neeearrrrca//////////
2, 6
K. H. Ngan, J. Phelan, E. Mansouri-Benssass J. Townscnd, and A. S. d'Avila Garcez. Closing the neural- symbolic cycle: Knowledge extraction, user intervention and distillation from convolutional neural networks. In 17th International Workshop on Neural-Symoolic Learning and Reasoning, volume 3432, pp. 19-43, January 2023. URL https://openaceess city paper by its authors. Use permitted under Creative Commons License Attribution 4.0 International (CC BY 4.0). 4
aaitt.ac.... ac. attpy/..c.c/.///aa/aa/// ww/iid/eeteaaeeeeet htld//pprin//3164// c 2023 Copyright for this
14

---

Published in Transactions on Machine Learning Research (01/2026)
Benjamin Perret, Giovanni Chierchia, Jean Cousty, Silvio Jamil Ferzoli Guimaracs, Yukiko Kenmochi, and Laurent Najman. Higra (hierarchical graph analyisis) documentation, httpss//hhigaa..raaaeeeeooo
2018. Accessed = 2024-03-07. 4
Benjamin Perret, Giovanni Chierchia, Jean Cousty, Silvio Jamil Ferzoli Guimaraes, Yukiko Kenmochi, and
Laurent Najman. Higra: Hierarchical graph analysis. SoftwareX, 10:100335, 2019. 4
Huy Phan. Pytorch models trained on cifar-10 dataset. https://githan htt.ggghhhuu.ccohhuoeeeeeo conp/auyapphaa//Pyrrro..... 2021. URL https://doii www.....g11.00. org/10..281//eeooo 4431043. Accessed: 2024-05-20. 18
Marco Tulio Ribciro, Sameer Singh, and Carlos Guestrin. "Why Should Trust You?": Explaining the predictions of any classifier. In 22nd International Conference on Knowledge Discovery and Dala Mining (KDD), pp. 1135 1144, 2016. doi: (0.1145/2939672.22225 29337778. 1,
Caroline Mazini Rodrigues, Nicolas Boutry, and Laurent Najman. Unsupervised discovery of interpretable visual concepts. Information Sciences, 661:1-26, 16, 2024. 2, 4, 17, 18
Avanti Shrikumar, Peyton Greenside, and Anshul Kundaje. Learning important features through propagating activation differences. In International Conference on Machine Learning (ICML), volume 70, pp. 3145-30 3153,, 2017.
Karen Simonyan and Andrew Zisserman. Very deep convolutional networks for large-scale image recognition. In 3rd International Conference on Learning Representations (ICLR), pp. 1 14, 2015. 5, 7
Jost Tobias Springenberg, Alexey Dosovitskiy, Thomas Brox, and Martin Riedmiller. Striving for simplicity: The all convolutional net. In 3rd International Conference Learning Represendations (ICLR), pp. 1 1-14, 2015. 4, 5
Mukund Sundararajan, Ankur Taly, and Qiqi Yan. Axiomatic attribution for deep networks. In 34th International Conference on Machine Learning (ICML), pp. 3319 3328, 2017. doi: 00.5555/3305880033000222
2, 4,
Randy Tan, Lei Gao, Naimul Khan, and Ling Guan. Interpretable artificial intelligence through locality guided neural networks. Neural Networks, 155:58-73, 2022.
Yongchao Xu, Thierry Géraud, and Laurent Najman. Connected filtering on tree-based shape-spaces. IEEE transactions on pottern analysis and machine intelligence, 38(6):1126-110) 1140, 2015. 5,
Yongchao Xu, Edwin Carlinet, Thierry Géraud, and Laurent Najman. Hierarchical segmentation using tree- based shape space. IEEE Transactions on Pattern Analysis and Machine Intelligence, 39:1-14, 04 2016. doi:
2016 5,
Matthew D. Zeiler and Rob Fergus. Visualizing and understanding convolutional networks. In 13th European Conference on Computer Vision (ECCV), pp. 818-833, 2014. 2, 4
Quanshi Zhang, Ruiming Cao, Feng Shi, Ying Nian Wu, and Song-Chun Zhu. Interpreting CNN knowledge via an explanatory graph. In Conference on Artificial Intelligence (AAAI), pp. 4454-4463, 2018. doi: 10.5555/850403333544511.
A Appendix
A.1 A. Segmentation techniques
As an important step for our framework, we employ segmentation techniques so we can decompose images, based on specific attributes, into more interpretable structures, enabling better human understanding and interpretation. We specifically employ hicrarchical segmentation techniques due to their capability decompose images into multiple levels of detail, mirroring how humans naturally perceive objects: initially observing the overall structure before delving into the finer details.
15

---

Published in Transactions on Machine Learning Research (01/2026)
I:
Hierarchical Segmentation
5 regions
regions
50 regions
I0a
10.0
Figure 8: An example of a hierarchical segmentation tree, where the root represents the entire image and each subsequent level recursively divides into an increasing number of subregions, reaching the leaf nodes that correspond to individual pixels.
Trees: tree is an acyclic graph, consisting of nodes that connect to zcro or more other nodes. It starts with "root" node that branches out to other nodes, ending in "leaves" with no children. image representation, the root node represents the entire image, and each leaf represents pixel, resulting in as many leaves as pixels. The structure between the root and leaves groups pixels into clusters at each level based on similarity metrics, with each level abstracting the one below. Using segmentation tree, we make cuts various levels to obtain different numbers and sizes of segmented regions. We present example in Figure 8.
Binary Partition Tree (BPT): A Binary Partition Tree (BPT) is data structure in which each node represents a region of the image. Similarly, the tree starts with a root node representing the entire image and branches out through a series of binary splits until reaching the leaf nodes, representing the individual pixels. Different from the tree, in which a node could have multiple splits, in the BPT cach split, divides region into two smaller sub-regions based on criterion.
Watershed: This algorithm (Cousty et al., 2008) constructs hierarchical segmentation tree based minimum-spanning forest rooted in the local minima of an edge-weighted graph. In this context, local minima are points in the graph where the surrounding edge weights are higher, representing the lowest values in their neighborhood. These minima serve as starting points for the segmentation. The algorithm iteratively merges regions beginning from these local minima, guided by the edge weights that indicate dissimilarity between adjacent pixels. By progressively combining these regions, the algorithm builds the segmentation tree, effectively capturing the hierarchical structure of the image.
16

---

Published in Transactions on Machine Learning Research (01/2026)
Figure 9: Explanation obtained using Tree-LIME- xAiTrees combined to LIME instead of occlusion to score regions. We show here the adaptability of the framework to different xAI techniques. Due to the high time consumption of Tree-LIME (A.5 Table 15), we present some preliminary results in A.7.
A.2 B. Dcclusion-based metrics:
Here, we discuss the metrics used to generate our segmentation based on the model explainability (block B in Figure 3). We present two metrics: (i) Occlusion, which is the impact of occluding an image region on its classification output, and (ii) CaOC which is the intra-class impact of occluding an image region. For (i), we assess how the output of a model changes when an image region is occluded. For (ii), we employ a sliding metric that ranks images based on the highest activations for given class. We then measure the movement. in this ranking after occluding region of the image, determining the intra-class impact of the occlusion.
output out, for image I, By concealing portions of this image, creating new image 4,- we obtain different model output out, The significance of the occluded area concerning a particular class e is assessed by comparing the outputs:
Occlusion: Let say have model E producing
out,,e out.e
(1)
If there is a significant difference, it indicates that the model strongly relies on this region for class activation, meaning that these regions have a high impact on the model's decision.
CAOC: In the Ms-IV method, introduced by Rodrigues et al. (Rodrigues et al., 2024), CaOC employs rankings to assess how occlusions affect the model's output space. A ranking is sequence of objects ordered according to a specific criterion, from the object most aligned with it to the least aligned. Suppose the criterion is to maximize class c. In that case, the first index i in this sequence represents the object (in our case, the image I,) with the highest activation for class c in the output out, If we define a function argsort obtain the indices of an ordered sequence of objects, we can derive the sequence of image indices that maximize class c: Seq. = arysort (out..es decreasing), with out.e the vector of outputs for class of set of input images (Z))
CaOC computes an initial ranking Seq, for a subset of images DS C DS, and then a subsequent rank- ing Seg', after occluding one region of image I, E DS' The significance of this occluded image region for the model is determined by the difference in the positions of this image in the rankings given by |position II) position II)).
This metric aims to assess the impact of occluding image regions not only against the original output out, but also against the outputs of a range of images. Incorporating the model's output space into the analysis ensures that explanations consider the broader context (global model's behavior). Hence, we can characterize it as globally aware, even when explaining a single sample.
Although the main experimentation uses these methods, we present in A.7 a framework's variation using LIME to show æAiTrees versatility with other types of xAI techniques. Explanations example in Figure 9.
17

---

Published in Transactions on Machine Learning Research (01/2026)
A.3 Tested framework's configuration
We tested four different sizes of minimal region for fittrrngg the initial segmentation. For Cat Dog and ImageNet datasets: 200, 300, 400, and 500 pixels. For CIFAR-10 10: 4, 16, 32 and 64 pixels.
For the model-based segmentation and datasets Cat vs. Dog and CIFAR1O, we tested four xAI techniques to generation the initial graph G on Figure 3: Integrated Gradients (IG), Guided- Bacebol--acckropaattiinn (BP), Input X Gradient (I X G), and Saliency (S). Given the number of images (time computation), for ImageNet dataset we tested only Integrated Gradients (IG), Guided-Bann Baddedd-ackprooagation (BP).
We tested three algorithms to construct the hierarchical segmentation: Binary Partition Tree (BPT), Watershed with Area, and Watershed with Volume.
We tested two different occlusion based metrics to obtain the impact of regions used to shape the hierarchical tree: CaOC and OCC.
When we refer to Tree-CaOC or Tree WreOWC we mean the human-based segmentation (edges' map) using Watershed area and CaOC occlusion metric. When we refer to IG-Tree- or IG-TreeW-Oce mean the model-based segmentation (using Integrated Gradients (IG) attributions) using Watershed area and Occ (simple occlusion Equation (1)) occlusion metric. When refer to Be-,reeB-Occ, mean the model-based segmentation (using Guided Backpropagation (BP) attributions) using BPT and Oce occlusion metric.
A.4 Parameters of the baseline methods
For Grad-CAM method, we used the last convolutional layer of each architecture with layer Grad-CAM from captum framework. For Occlusion (from Captum framework) we used, for Cat vs. Dog and ImageNet, step of 3x7x7 and sliding window of 3x14x14. Since CIFARIO is much smaller, the step was 3x2x2 and sliding window of 3x4x4. For LIME, we used the standard configuration for Cat vs. Dog and ImageNet (Quickshift kernel size of and, Quickshift, kernel size of 2 for CIFAR10. All the other methods followed the standard configuration. For Ms-IV, used the original configuration from the paper (Rodrigues et al., 2024). For XRAI, we used the original implementation (Kapishnikov et al., 2019) of the fast version.
A.5 Quantitative evaluations
Models' description: Table 5 shows the number of images in train and validation sets for Cat vs. Dog and CIFARI0 datasets. We also include the train and validation accuracies for the models ResNet18 and VGG-16 used in the quantitative evaluations.
Cat vs. Dog models were trained with initial weights from ImageNet, learning rate le - cross-entropy loss, the Adam optimizer, and early stop in 20 epochs of non-improving validation loss.
CIFAR10 models were adapted to receive 32x32 input images, and they were trained with initial weights from ImageNet, lcarning rate le - 2, cross-entropy loss and the stochastic gradient descent optimizer (code from Phan (2021)).
The ImageNet models used the pre-trained weights from the PyTorch implementation.
Table 5: Number of images and accuracy train and validation sets for ResNet18 and VGG-16 models. We train the models with two different dataset: Cat vs. Dog and CIFAR10.
Train
Val.
Num. images
Acc. (%)
Num. images
Acc. (%
ResNet18
98.21
97.86
Cat vs. Dog
19,891
5,109
VGG-16
99.04
98.61
Res VGG-16
99.56
92.53
CIFAR1O
50,000
10,000
99.84
93.54
18

---

Published in Transactions on Machine Learning Research (01/2026)
Table 6: Percentage of images with the original class changed after the exclusion of selected explanation regions for Cat vs. Dog dataset. Highlightod in blue are the configurations presented in the main paper. We tested hierarchies constructed by filtering out smaller regions than 200, 300, 400 and 500 pixels, segmentation based on Edges, Integrated Gradicnts (IG), Buiidedd-Backroopggattoe (BP), Input X Gradients (I X G) and Saliency. We tested three different strategies to for the first hierarchical segmentation: BPT, watershed with area attribute, and watershed with volume attribute. Same column shows images maintaining the original class when the output was reduced, and Total is the sum of class change (Ch.) and class reduction (Same).
Cut Dog
% of imnuga
Edgge
IG
VGC
BP
I G
Saliency
Edgge
IG
ResNet
RP
XX G
Salloncy
Ch
Sonse
Cb.
Same
Ch.
Cb.
Same
Ch.
Somse
Cb.
Same
Ch.
Souae
Ch.
Same
DPT
0:35
0.31
200
Watershed area
Ca0C
Ooc
0:30
0.15
0.66
0.23
0.42
0:41
0.22
0.02
0.53
0.31
0.62
0.34
0.44
0.51
0.31
0.45
0.63
0.16
0.52
0.50
0.53
0.37
0.30
0.45
0:68
0.17
0.51
0.46
6.6d
Watersbed volume
CaDC
018
0.88
0.30
0.45
0.12
0.48
0.18
0.47
0:33
0.20
0.38
0:44
0.17
0.98
0.31
0.00
0.37
0:45
0.16
0:45
Oec
0:35
0.64
0.51
0.61
0.31
D.64
0.87
0.57
0.52
0.63
0.38
0.62
0.25
0.65
0.25
0.50
0.30
0.60
0.82
0.64
BPT
CsbC
Ooc
0:36
0.50
0.45
0:35
0.22
0.21
0.30
0.007
0.56
0.61
0:30
0.314
0.17
0.17
0.07
0.11
0.06
0.58
0.38
0.41
0.41
0.28
01.33
0:41
0.07
0.01
0.19
0.27
0.53
0.08
0.30
0.57
0.00
0.06
0.20
0.15
0.35
0.39
0.41
0.51
300
Waterdhed arross
C.0C
Ooe
0.16
0.30
(1:5
0:49
0.43
0.25
0.43
0.42
0.22
0.45
0.02
0.24
0.53
0.05
0.51
0.15
0.50
0.43
0.35
0.20
0.42
0.59
0:30
0.20
0.61
0.63
0.59
0.37
0.47
0.50
0:30
0.18
0.47
0.65
0.51
0.19
6.01
6/45
Watersbed volume
CoOC
0.17
0.32
0.61
0.48
0.45
0.64
DPT
Occ
08:36
0.47
0.45
0.24
0.08
0.47
0.41
0.52
0.16
0.23
0.30
0.42
0.39
0:37
0.05
0.00
0.11
0.14
0.95
0.329
0.55
400
Watersbed area
C2DC
Occ
(0:16
10:00
0.:9
0.64
00.61
00:00
0:46
0.50
0.42
0.25
0.54
0:15
0.22
0.08
0.55
0.51
0:17
0..22
0.50
0:21
0.34
0.:1
0:41
0.30
0:31
0.61
0:31
0.221
0.18
0.51
0.40
0.20
0.30
0:16
0.63
0.31
6.21
6.82
Waterdaed volume
C20C
Onc
0:35
0.118
0.63
0:49
0.35
0.221
0.00
0.47
0.30
0.16
0.01
0.45
0.55
0.22
0.57
0.65
0.36
0.51
0.50
0.62
0.21
0:34
0.40
0.61
0.36
0.15
0.61
0.45
0.13
0.25
0.00
0.47
0.10
0:39
0.65
0:42
0.33
0.20
0.64
BPT
CsDC
Ouc
0.35
0.45
0.49
0.40
0.12
0,11
0.06
0.04
0.49
1.46
0.29
0.36
0.07
0.07
0.03
0.04
0,41
0.47
0.42
0.49
0,30
0.41
0.41
0.51
0.04
0.04
0.13
0.17
0.31
0.58
0.34
0.58
0.04
0.04
0.11
0.11
0.87
0.27
0.42
6.54
G00
Watershed
CsDU
0.16
0.31
10:91
0126
0.41
0:43
0.30
0,36
0:36
0.90
0.06
0.41
Watersbed volume
0:35
Exclusion of important regions: Given that each region-based explainable AI (xAT) method identifies important regions in an image to explain the prediction of model, we performed occlusion of these regions to mcasure the impact of cach selection and evaluate the methods.
Except for LIME, which proposes a ranking of the most important image segments, the methods we used as a baseline provide values for measuring the importance of pixels, which in visualization is similar to regions or segmentation for humans. However, if we select all the positive importance values provided by these mothods, we are likely to cover a large part of the image. By selecting regions with only the top 25% higher values, we reduce the size of the mask. In fact, different datasets and models will show different visualization behaviors, so we chose to define high threshold that is fixed (not specific to a single dataset) and common to all methods in order to have fair comparison. Therefore, for methods that assign scores to regions, we masked the 25% highest scores.
We present, in Tables 6, 7, and 8, the complete experiments of different configurations of our framework for the datasets Cat vs. Dog, CIFAR10, and ImageNet respectively.
To address the issue of unhelpful explanations resulting from methods selecting the entire image as important, potentially leading to class changes upon occlusion, we introduce a novel metric termed Pixel Impact Rate (PIR). The idea of PIR is to evaluate the impact per pixel/pe per image:
PIR values:
PIR(erp:) out, class orig out, Lolass_orig
(2)
num pizels exp
where exp, is the explanation or image i, out, orig is the original logit corresponding to the analyzed class, out, outi,class__or orig is the logit after the perturbation, and num pizels exp is the number of occluded pixels.
This metric quantifics the impact on class activation per occluded pixel. Complementing the percentage of class change, PIR distinguishes whether changes are primarily caused by complete or near-complete occlusion of the image. Higher PIR values indicate that each occluded pixel has significant average impact, suggesting
19

---

Published in Transactions on Machine Learning Research (01/2026)
Table 7: Percentage of images with the original class changed after the exclusion of selected explanation regions for CIFAR10 dataset. Highlighted in blue are the configurations presented in the main paper. We tested hierarchies constructed by filtering out smaller regions than 4, 16, 32 and 64 pixels, segmentation based on Edges, Integrated Gradicnts (IG), Gundedd-Backrropagaall Saliency. We tested three different strategies to for the first hierarchical segmentation: BPT, watershed with area attribute, and watershed with volume attribute. Same column shows images maintaining the original class when the output was reduced, and Total is the sum of class change (Ch.) and class reduction (Same).
(BP), Imput X Gradients (I X G) and
CIFARIO
% Images
Edges
VGG
I.
Salfency
Edges
RowNet
X.X G
Solleacy
Ch.
Ch.
Ch.
Ch.
Ch.
Same
Ch.
Ch.
Ch.
Cb.
Ch.
Saine
BPT
Ooc
0.81
0.12
0.91
0.05
0.31
0.00
0.85
0.10
0.00
0.91
0.07
6:18
0.85
0.14
0.:17
0.75
0.17
0.83
0..0
0.:30
0.13
0.50
0.10
0.52
0.11
0.81
Watershed area
CsOC
0.16
0.31
0.27
0.33
0.27
0.35
0.27
0.33
0.20
0.33
0.13
0.90
0.21
0.37
0.21
0.57
0.25
0.57
0.25
0.30
Ooc
0.72
8:22
0.74
0.23
0.73
0.22
0.70
0.22
0.77
0.24
0.70
0..25
0.74
0.24
0.71
0.25
0.76
0.251
0.74
0.25
Wateeshed volume
CaOC
0.16
0.34
0.26
0.35
0.27
0.35
0.26
0.94
0.36
0.84
0.18
0.60
0.24
0.87
0.21
0.88
0.25
0.57
0.24
0.57
Dor
0.73
0.21
0.75
0.91
0.79
0.25
0.75
0.25
0.79
0.25
0.71
0.21
0.71
0.25
0.77
0.98
0.75
0.33
0.71
0.25
BPT
CSOC
000
0111
0.07
[111
0.00
0:17
0.05
0.18
0:31
0.12
0.12
0.00
0.14
0.11
03300
0.00
0.11
0.21
0.38
One
0.50
009
0.70
017
0.75
0.30
0.63
0.055
0.70
0.50
0.09
0.61
0.099
0.00
0.57
0.08
0.70
Watershed
CoOC
0.1
0.35
0.22
0.35
0.22
0.35
0.92
0.76
0.22
0.85
0.22
0.39
0.25
0.35
0.21
0.70
0.25
0.25
0.71
0.37
Watershed volume
CaOC
Ooc
0.16
3.22
0.33
0:1
0.22
0.73
0.24
0.35
0.72
0.21
0.35
0.24
0.75
0.53
0.34
0.22
0.22
6,72
0.24
0.71
0.19
0.39
0.24
0.24
0.74
3.24
0.21
0.71
0.40
0.27
0.75
0.20
0.23
0.05
0.74
0.24
0.37
BPT
CaOC
Oze
2.19
BM3
B0S
0.44
001
0.05
0.07
0.62
0.09
0.09
0.40
0.001
0.03
0,70
0.16
0.17
0.15
0.40
0.07
0.05
0.04
0.07
0.44
0.05
0.85
0.10
0.56
0.11
0.08
0.88
0.00
0.04
0.05
0,70
0.16
0.18
0.221
32
Watershed area
C2OC
Ooe
0.15
3.69
0.20
0:25
0.75
0.21
0.96
0.22
0.74
0.15
0.37
0.22
0.20
0.75
0.96
0.21
0.21
0.77
0:85
0.224
0.18
0.BS
0.31
0.222
0.23
0.74
0.90
0.23
0.77
0.21
0.41
0.25
0.23
0.75
0.00
0.221
0.771
0.22
0.00
0.33
Watershal volume
CaOC
On
0.16
0.70
0.29
0.90
0.20
0.77
0.36
0.21
0.19
0.70
0.87
0.25
0.20
0.71
6.22
0.96
0.21
0.79
6.21
0.85
0.09
0.15
0.34
0.22
0.73
0.23
0.40
0.25
0.21
0.00
0.41
0.56
0.75
0.21
0.46
0.23
0.21
0.73
0.06
0.25
RPT
CoCC
0000
0.005
0.08
0.01
0.11
0:25
0100
0.13
0,122
0.43
0.05
0.19
0.50
0.27
0.02
0.33
0.34
0.21
0.53
Watershed arem
CaoC
0.13
0.16
0.20
0.29
0.15
0.27
0.20
0.25
0.22
0,90
0.15
0.19
0.23
3.83
0.21
0.84
0.22
0.34
0.22
0.36
Oor
0.00
815
0.73
0:21
0.73
0:21
0.75
0.30
0.30
0.23
0.00
0.15
0.73
0.23
0.79
0.23
0.71
0.22
0.70
0.25
Watershed volume
CaOG
0.12
B116
0.19
0:31
0.18
0.30
0.20
0.29
0.21
0.25
0.15
0.18
0.21
9.36
0.21
0.37
0.21
0.35
0.22
0.34
Ose
3.00
0.14
0.71
0.28
0.65
0.25
0.73
0.21
0.70
0.22
0.00
0.15
0.71
0.21
0.67
0.27
0.71
0.21
0.7L
0.21
that concealing larger portions or the entire image leads to lower PIR. indicating less precision in the concealed area.
Tables 9, 10, and 11 display for each network, and tested configurations of our framework, the average (avg) and standard deviation (std) of PIR, for the datasets Cat vs. Dog, CIFAR10, and ImageNet respectively.
Inclusion of important regions: Additional experimentation was conducted to demonstrate method's capability to identify an image region with sufficient information for the original class. The goal of this experiment is to determine whether the selected important region, when the only one left unoccluded in the image, can maintain the classification in its expected class. This experiment clucidates the critical role of these identified regions, providing strong evidence that they indeed contain essential information for accurate classification. We occluded all regions in the images except for the one selected by each method. We then calculated the percentage of images that changed class. We present the results from the three datascts, Cat vs. Dog, CIFAR10, and ImageNet, and all the tested framework configurations in Tables 12, 13, and 14, respectively. Lower percentages indicate better performance, meaning that a smaller percentage of images changed class, demonstrating that the chosen regions were sufficient to preserve the class for most of the images.
SIC and AIC for hierarchy evaluation: Inspired by the metrics Softmax Information Curve (SIC) and Accuracy Information Curve (AIC) proposed by Kapishnikov et al. (2019) we calculated the Softmax and Accuracy curves by including only selected image regions as model input. We used the parameters from the original paper: maintaining 10% of the original pixels and using linear interpolation to generate the blur. We used the thresholds of 0.5%, 1%, 2%, 3%, 4%, 5%, 7%, 10,, 13%, 21%, 34%, 50%, and 75% percent, representing the most significant region values according to each evaluated xAI method. Instead of using the image entropy values as the x-axis we used the thresholds. Figure 10 presents the curves for the mean of 1,000 randomly selected ImageNet images, and Figure 11 presents the results for 512 analyzed images from the Cat vs. Dog dataset.
20

---

Published in Transactions on Machine Learning Research (01/2026)
Table 8: Percentage of images with the original class changed after the exclusion of selected explanation regions for Imagenet dataset. Highlighted in blue the configurations presented in the main paper. We tested hierarchies constructed by filtering out smaller regions than 200, 300, 400 and 500 pixels, segmentation based Edges, Integrated Gradients (IG), and Stidded-Backroopagatie (BP). We tested three different strategies to for the first hierarchical segmentation: BPT, watershed with area attribute, and watershed with volume attribute. Same column shows images maintaining the original class when the output was reduced, and Total the sum class change (Ch.) and class reduction (Same).
Imagenet
VGG
ResNet
% of images
Edges
IG
BP
Edges
IG
BP
Ch.
Same
Same
Ch.
Same
Ch.
Same
CaOC
0.49
0.00
0.02
0.36
0.25
0.44
0.01
0.04
BPT
Oce
0.57
0.38
0.35
0.01
0.71
0.16
0.60
0.35
0.50
0.02
0.74
0.13
CaOC
0.26
0.51
0.27
0.51
0.27
0.51
0.27
0.45
0.31
0.45
0.30
0.45
200
Watershed
Occ
0.42
0.55
0.44
0.54
0.45
0.52
0.48
0.50
0.50
0.47
0.52
0.46
CaOC
0.26
0.51
0.26
0.51
0.25
0.52
0.28
0.46
0.31
0.44
0.28
0.46
Watershed volume
Occ
0.44
0.54
0.39
0.59
0.38
0.60
0.49
0.49
0.47
0.51
0.45
0.53
CaOC
0.22
0.46
0.00
0.01
0.10
0.30
0.24
0.42
0.01
0.02
0.09
0.25
BPT
Oce
0.55
0.38
0.20
0.01
0.64
0.17
0.58
0.35
0.32
0.01
0.65
0.13
CaOC
0.24
0.51
0.27
0.52
0.26
0.51
0.27
0.45
0.31
0.45
0.29
0.45
Watershed
Oce
0.41
0.56
0.43
0.54
0.44
0.53
0.46
0.51
0.50
0.48
0.51
0.46
CaOC
0.24
0.51
0.26
0.52
0.25
0.52
0.27
0.46
0.30
0.45
0.28
0.46
Watershed volume
Oce
0.42
0.54
0.38
0.59
0.37
0.61
0.47
0.49
0.46
0.51
0.43
0.54
CaOC
0.21
0.42
0.00
0.01
0.09
0.25
0.23
0.38
0.00
0.02
0.08
0.21
BPT
Oce
0.54
0.37
0.13
0.00
0.58
0.16
0.57
0.34
0.22
0.01
0.59
0.13
CaOC
0.24
0.51
0.26
0.52
0.25
0.52
0.26
0.46
0.31
0.46
0.29
0.46
400
Watershed area
Oce
0.40
0.56
0.43
0.54
0.53
0.45
0.51
0.49
0.48
0.50
CaOC
0.24
0.51
0.25
0.52
0.52
0.26
0.46
0.30
0.46
0.27
Watershed volume
Oce
0.42
0.54
0.38
0.59
0.61
0.46
0.50
0.45
0.52
0.55
CaOC
0.09
0.00
0.22
0.22
0.16
0.01
0.07
BPT
0.00
0.01
0.16
0.55
0.00
0.01
CaOC
0,23
0.26
0.52
0.25
0.52
0.26
0.46
0.30
0.46
500
Watershed area
Oce
0.40
0.43
0.53
0.53
0.44
0.51
0.48
0.48
0.50
CaOC
0.23
0.25
0.52
0.24
0.53
0.26
0.46
0.29
0.46
0.27
Watershed volume
Oce
0.41
0.54
0.37
0.59
0.36
0.60
0.46
0.50
0.44
0.52
0.42
0.55
21

---

Published in Transactions on Machine Learning Research (01/2026)
Table 9: Pixel Impact Rate (PIR) of selected explanation regions for Cat vs. Dog datasct. Highlighted in blue are the configurations presented in the main paper. The metric is the rale of the impact. under occlusion (difference belwwen the original class outpul. and the outpul. under occlusion) by the number of pixels of the occlusion mask. We tested hierarchies constructed by filtering out smaller regions than 200, 300, 400 and 500 pixels, segmentation based on Edges, Integrated Gradients (IG), Guided-Baas Bae-BBacpoopagaaa
(BP), Imput X Gradients (I X G) and Saliency. We tested three different strategies to for the first hicrarchical segmentation BPT, watershed with area attribute, and watershed with volume attribute. We expect higher values, on average (avg). for PIR, meaning each occluded pixel has a high impact.
2.00 UL
2.080-14
1.00-0
5.05004
1:30--0
2070-04
4.00-144
3.55-0
3.
1:20-00
1.80-UL
3.086-04
2.50e-01
3.000004
8.55-015
2
2.000-14
2.47-0M
2716-04
Sallency
130000
10N-N
315-44
200-00
289-0K
2.0-044
9-0005
976-0
247-044
2100
100-04
9496-U
2400-04
2012-14
188-06
1770-04
8.50015
7.7-05
2003-04
1.08-1
157-04
2.176
6.06-4
1.80800
6.70600
2:00-00
1.7600
3.5000
3.80-44
1.00-02
1.7640
3,00-4
3.3600
1.2562
1.46-4
3.36-0
3.25-44
3.1664
3. 3.67 08
1.7%
3.51644
3:30000
3.900
11.90-04
3.27600
1.72600
2.70000
2.50004
2,0001
2.00000
2,47600
1.36-000
2.000004
2.27600
2.06-00
2118-00
1.01600
9.01e-01
2:30c.00
2.13604
2.0064
200-04
1.83 02
5.000AI
6.306-14
1.50000
0.0000
1.20-00
7-41-00
4220016
502-0L
1.3600
3.0001
470-01
3.00000
1.20-112
5
20000L
2.400116
3.00-tt
250e-1
ReeNNe
2:00-002
200-0L
2.
1.39-00
1226-00
230-14
2770OL
2216-0
1.59-00
200-04
210-
209-06
101000
1-176-0
803-04
211e04
220-0
10le04
std
2.25 02
1.800022
5.00800
5.0%
1.70000
3.28
4.0664
4.2004
1.00004
4.00000
1.17502
3.89600
4.2000
4.00-04
4.80-00
2.0000
3.8004
3.000 03
3.34-44
3.55000
3.38
2.30000
2.09005
28000
ZAINH
2.85001
2.10OL
1.00-05
2.000LL
2.36
2.57-0t
2.000L
1.50055
2.
2.166
2.26-M1
1.0g0l
1.800414
5.17H
1.500-0
5.110
1.07-0
451e0L
3.700L
1.20-H0
1.30-01
3.00000
3.5001
2.706-04
1.10-0
1.00-OL
2.00-0
2.020004
2.0-01
200-01
2.00
3.03-44
2.03-0
2.
1.00-0
US
2.73-14
2.50-001
9.50-01
9.51e-US
2206-4
2:21e-06
1.75-004
0.208-15
8.00-05
1.90-00
1.00-t
102-00
Cot vo. Dlog
1.20
9.79
2.15e:01
0.00001
3.976
5.00-44
2.17001
2.000:01
5.85004
6.17604
5.5te01
1.20001
2.00-11
2.03-001
4.30001
4.23604
3.50001
3-300-01
1.50-4
1.700-11
3.90-01
3.280044
3.0001
3.50-01
std
Sallency
1,62 01
1.60-
5:0001
4.00-01
4176-00
4.06
1.60--01
4.40
4.580MM
9:50-01
3:00 01
1.00c-M
1.53e-01
374-
3.00-01
2200
1.46-M
1:60-01
3:40-0
818--M
2.76-11
250c
9.72e 02
7.56-04
5.00-01
835e-04
7.50004
1:90e-22
5,98-04
4,00-04
0.00-0
6.28600
1.7k6-0
1.70-002
4.80004
4.200-04
4.0+01
0,0060
7.846-02
3.20-00
3.0000
8.7600
3.h6ll
39t0lt
2.70
aaog
73
A
5.50e011
555004
20Te00
4.00-01
40063
3.00001
1.50
22-000
3 56-33
3:8001
2.80000
300001
346601
3
3,30 02
9.66-11
70
9.000-1
5.36-04
70-00
3.10e-00
6.26--44
5.70-04
7.30-01
5.300-01
1.35--22
3:00-00
4.50-004
4.00-04
4.50-01
0.50-01
9.06-11
3:00-00
1.0--0
4.00044
4.06-01
3.86-01
VGG
5:20e CO
6.16o04
0.00000
4.06004
1,900-03
4.57004
4.11600
1.8604
4.06-00
1.90+00
3786-0
385000
3.70004
3000-4
301e01
1.00-05
378641
307-00
520664
320664
avg
1.79e 02
7A3h-4
50te01
8:06-04
4,8%-04
0.800-02
3
5.00-04
4.30-04
5256-0
4.00000
0.96602
3216-02
4,00004
4.26-04
4.06-0
600600
584-02
321-00
3.00-01
361004
3.lell
320e01
2.50-00
5.43-01
16-01
5.36-01
1.00-01
4.0--01
3.60-11
1.00-01
3.088-01
1.00-22
2.5--00
3.30-01
3.44-11
3.0001
3330-01
9.78-1
206-00
300-01
290-01
316-01
1015-10
1.1e-00
1000-000
102--0
2000-01
3236-04
614--0
7306-M
5000001
7.00-00
272--1
3.1e-01
5226-04
550-01
476-00
2716-M
313--01
4700-00
5.00004
500-04
417e-0
291e
2:00.
2200-04
6.46-04
0:3660
6:16-04
20le01
2226-04
4.0000
4.07604
4206-04
424-00
1.00-04
2206-4
4U7-004
4.00004
381644
3.0000
1.50004
2.06-4
301e00
3.00-4
3.00-64
321e64
avg
CaOC
Oce
CaOC
Oce
CaOC
Oce
CaOC
Oce
CaOC
Oce
CaOC
Oce
CaOC
Oce
CaOC
Oce
C&O0
Occ
CaOC
Oce
CaOC
Oce
C&OC
Oce
Watershed volume
Watershed volume
Watershod volume
Watershed volumee
Watershed area
Watershed area
PIR
Watarshed
Waterslied
200
2000
400
800
22

---

Published in Transactions on Machine Learning Research (01/2026)
Table 10: Pixel Impact Rate (PIR) of selocted explanation regions for CIFAR10 datasct. Highlighted in blue are the configurations presented in the main paper. The metric is the rate of the impact under ocelusion (difference between the original class output. and the output under occlusion) by the mumber of pixels of the occlusion mask. We tested hierarchies constructed by filtering out smaller regions than 200, 300, 400 and 500 pixels, segmentation based on Edges, Integrated Gradients (IG), Guided Guided-Backer ---BkBppppagaation (BP). Input X Gradients (I X G) and Saliency. We tested three different strategies to for the first hicrarchical sogmentation: BPI, watershed with area attribute, and watershed with volume attribute. We expect higher values, on average (avg), for PIR, meaning each occluded pixel has a high impact.
3.5te002
9.00--22
1.01aa111
9-30-12
9:41e-12
1.50-002
1
3,000012
4.20-122
3.75--12
1002-1122
2.51-022
2.03-02
2200-122
2.500-02
4.470-18
7.070-1
1:30-02
1.53e-12
1.25--12
1.47e-02
vill
Sallency
8.11m82
2,00-00
6.00e-12
6,376112
5.47--2
5.7
1.M4012
1.50e
8:06-122
3.76-12
2.50e-12
3.47--12
5.0ee0
1.00-112
1.80-12
2.60e-12
1.75-012
2.56-12
2.te-t
6.50611
1/0e-02
1.72e-12
1.60-02
3.000a
4.00-0)
1.00
1.0680
0.00-00
1.00-001
3.21e00
6.06-01
3.00-11
4.200022
3.00002
4.20012
3.336-01
2.26w122
2.50-02
2.60-12
2.0000
1.200-02
1.02600
1:4e-00
and
IX G
ang
2.3ho00
1,60x0)
0.306-00
0,76600
6.00e-02
1.20w00
1.06-01
3000-02
3644022
8776602
6.58-02
1.5600
251662
1.86-0
2.5000
4.710.02
1.00-00
1.60
1.06-02
1080-00
1116-0
1.000000
1.00--L
9-37-09
1.00-01
2.1e00
3.09-0L
3.00
4.20-022
4.00%
02
2.29-001
20
2.176-00
2.05--0
2
2.546-0
1.760
01
119--00
1
02
1:23-02
190-00
ResNet
BP
1.966811
1.18-01
6.37o-12
6,200-112
5.70-02
0.20e-112
1.02-01
0.056-12
2.97-2222
3.40-122
2.806-12
28-06-022
4.700-122
1.76--12
2.33-02
2,36-122
d. 02
2,06-112
0.236-03
1.58-02
8.06-18
1.55-02
3.76m/1
4.06-01
1:00000
1.000111
9.73-12
1:00-01
3.20-11
3.00-01
4.57-12
3,00022
2.00-011
3400-01
226--2
2.51-022
22256112
257-22
1.06-011
2.06-11
121e-02
1.00-02
1.26-022
1.0--02
1G
2.280l1
1.66-)
6.37--02
6.470-12
584m02
621e-02
1.00001
9.70-00
5.00802
2916-02
3550-12
9300002
5,806002
1.56-02
2.0e000
1856-0
2.86002
4.34-022
2,596022
0.58-05
1.50e
1.006022
1.0e-2
2.000-72
2,00000
3.
3.460-22
3.50-00
3 550-0222
8 130-0
1.00-00
2.506022
3.006-22
2.9--22
1.300-00
6:900-00
1.00
200002
2.00e
4.30-00
1.06--00
1.27e
1.04-22
Eddgss
1.12-22
115--02
196
200-02
191-02
196-02
5.06-0
500-40
1.74-02
188672
170--1
1816-22
136600
1.29-02
1:53-02
122 02
1.0-022
5. 04
2.46-088
530-08
9.16
5.6-088
930-03
CIFARL1O
0.530-12
4.01:022
1.37-011
1.56-01
1.20~01
1.00e-01
1.43-02
1.90-022
4.73o0122
5.30002
1.200002
4.970012
8.17e005
1.000-122
2,37-12
2.00-02
2.000112
2.750-12
4.270-055
7.00-11
1.4e-02
1.06e-02
1.86-022
1.5e-02
wtdl
Sallonry
avg
20
7.
6.60m20
0.30m10
1.5te00
250002
4.14-02
2.00-00
8.720002
5.00000
1,6--2
1.50-22
2,86-02
1.00e-00
6:56-00
1000-00
1.80-02
0.16-00
1.5e-00
3.0-01
5.10011
1.36--11
1.60-0
1-20-11
1.08-01
500-01
4386-01
4.714-222
500--22
450--02
5A15--22
2-06-0L
3.08-OL
2:00-100
200--22
20
2.18-20
1.00-01
2.06011
130-00
1500-22
158-20
IX G
1.00661
1.36-0)
02
7.806
9.236-22
7:1
0.08-
1.00-01
2.00002
4.01802
2.90-02
0,70600
6.206-02
1.64-02
2.50600
1.06-02
2.59-402
2.10002
9.
1.70600
8.64-08
1.70002
1.876-111
4.3k-0)
1.00e:01
1.T1--11
1.25-01
1.72-01
3.00-01
3.776001
4.86-12
6.1 5.15-922
5.036002
2.35e-01
3.136-11
2.41n022
2.75-00
2.75--02
1.560
1.2e-02
1.00002
1.25-112
1.te00
VOG
BP
avg
1.34--111
1.10-0I
8,500
0.550-122
7:60--0
9.30e-
1.0e-00
3,300-12
2.70e-12
1.62
2.47e-02
1.500-12
2.30-12
2.0--22
8.00-1
1.62
1.50ett
3-50-111
4.0001
01
1:30--0
1.00-011
1.20011
1.50--01
3-00-1
1.30-01
4.00-22
1.06-12
5.00-022
2.50--1
2.40-22
18--22
2.36-022
2776-22
1.
286-01
1050-22
1.00-11
195-22
IG
2.06-01
1:30-0)
7.94--02
9.706022
7:08-00
9.11-02
1.00001
9.966022
2.000022
2.500022
1181-022
0.200022
0.256-12
1.0-02
2.57e02
1.00-02
2.50-22
2.006022
1.09622
1.0-02
1.00-1122
2.09-022
3.01e-02
3.770-12
3,00-02
3.70022
7:50005
9.086005
2.00012
2,00-022
2.86-02
200-012
3.75605
6,286-05
1.09-022
2.12602
1.00e 1.866122
2.00-02
1.070005
4-46-05
100002
1.00002
1.06-92
1226-02
Eddgos
wvg
8:20--1
1.1
1.81e
1.800-112
1.66--12
1.700-00
2,74-03
0.57e
1.46-022
1.76602
1.4e
1856
0.
4,00
1,00w02
1.te-02
1.56-022
2.74e 04
2,50-003
4/73e-0
9,30e 0J
4.80-113
0.30e-05
CaOC
Oee
CaOC
Ont
C2OC
Oce
CaOC
Oce
CaOC
Dox
CsOC
Oze
C2OC
Oce
CaOC
Ooe
Onx
CDOC
Oon
C2OC
Oce
CsOC
Oce
Watershed area
Watershot volumee
Watershed area
Watershed volume
Watershed nrsa
Watershed volume
Watersed volume
BPF
Watershed
23

---

Published in Transactions on Machine Learning Research (01/2026)
Pixel Impact Rate (PIR) of selected explanation regions for ImageNet dataset, Highlighted in blue are the confiuuration presented in the main paper. The metric is the rate of the impact under occlusion (difference between the original class output and the output under occlusion) by the number of pixels of the occlusion mask. We tested hierarchies constructedd by filtering out smaller regions than 200, 300, 400 and 500 pixels, segmentation based on Edges, Integrated Gradients (IG), and Guided- Backpropagationn (BP). We tested three different strategies to for the first hicrarchical segmentation. BPT, watershed with area attribute, and watershod with vohume attribute. We expect higher values, on average (avg), for PIR, meaning each occluded pixel has a high impact.
5.15e-02
2.35e-02
2.80e=03
2.55e-08
2.74e-08
2,52x-03
4.5
2.17c-02
2.15e-03
2.01e-03
2. 2.12e-03
1.05e-03
4.08e-02
2,03e-02
1.78e-09
1.64c-03
1.76G-03
1.61e-03
3.14c-02
3,67e-02
1.51e-08
1.38e-03
1.15e-08
1.36e-08
BP
avg
1.19e-02
3.25e-03
2.11e-03
2.04e-12
1. 1.96e-03
2.05e-03
9. 9.30e-02
3. 3.09e-03
1.70e-02
1.64e-03
1.58c-03
1.65e-03
7.55e-03
2.75e-03
1.45e-113
1.39e-03
1.35e-03
1.39e-02
3. 3.67e-03
6. 6.23e-03
1.25e-03
1.22x-03
1.17e-03
1.200e-0
5.60e-022
1.35c-02
3.12e-03
2.88e-02
3.00e-03
2.79c-03
4.88e-02
9.12e-03
2.43e-03
2.24e-03
2.37c-03
2.17e-03
4.32c-02
6.85e-03
1.99c-03
1.88e-03
1.97c-03
1.81e-03
6.2 6.20c-03
3. 3.96c-02
1.60e-03
1.58e-0
1.64e-03
1.54e-03
std
ResNet
IG
9. 9.66e-113
1.39c-03 09
2.48e-1 03
2.46e-199
2.33e-03
2.39c 03
7-31e-08
7.61e-04
2.01e
1.99e-13
1.90c-03
1.92e-03
5.95e 03
4.5 4.55e-01
1.70e-113
1.69e- 03
1. 1.62e-03
1.63e-0h
3.07e-04
5.12c-18
1.46e-09
1.4Ge 03
1.40c-08
1.42e-05
avg
9.63e-04
7.27e-04
2.77e-03
2.58e-03
2. 2.65e-03
2. 2.40c-03
7.45e-04
5.93e-04
2.11e-03
1.99e-03
2.02c-03
1.85c-03
6. 6.06e-04
5.05c-04
1.74e-08
1.62e-03
1.66c-03
1.52e-03
5. 5.07e-044
4.42c-04
1.476-03
1.39c-03
1.39c-03
1.30e-08
Edges
6.76e-14
5.13e.04
1.78e-03
1.67e-13
1.67e=03
1.49o-03
5.2 3.26e-04
4.54e-04
1.476-03
1.339-113
1.38e-03
1.25e-03
4.21e-04
4.08e-04
1.266e133
1.20c.03
1.18c-03
1.08e-03
3.44c-04
3.60e-14
1.10e-03
1.05e-03
1.02e~03
9.48e-MM
avg
Imagenet
7-37e-12
3.43e-02
4.07e-03
3.7de-13
3.92e-03
3.60e-03
6.01e-02
3.24e-02
2.8de-13
2.62e-13
2.71e-03
2.58e-03
5.30e-02
3.23e-02
2.18--18
2.02e-03
2.12e-03
1.98e-03
4. 4.78e-02
4.75e-02
1.77e-03
1.65e-03
1.72e-03
1.58e-03
std
BP
1.43x-02
4.51e-03
2.76e-03
2.67e-03
2.58x-03
2.75e-03
1.11e-02
4.34c-03
2.07e-03
1.98e-08
1.92e-03
2.02x-03
9.36e-03
4.14e-03
1.68--03
1.61e-03
1.56e-03
1.61e-03
7.97e-03
7.91e-03
1.41e-03
1.37e-03
1.31e-03
1.34e033
avg
7.08e-112
2. 2.15c-02
1.29c-03
4.100-113
4.16e-03
3. 3.90e-03
6.40c-02
1.48c-02
2.97e-13
2.75p-133
2. 2.90c-03
2. 2.68e-03
5.48e-02 02
1.03e-02
2.25e-13
2.12c-03
2. 2.19c-03
2.04e03
8.14e-03 03
4. 1.88e-02
1.83e-03
1.72c-03
1.78e-03
1.66e-133
VGG
IG
1,12e-02
1.85c-03
3.07e-03
3.04--03
2. 2.86e-08 03
2.96e-00 03
8.18e-03 03
9.02c-04
2.27e-08
2.22e-03
2.10c-03
2.14e-03
6.42e-03 03
5.00c-04
1.81e-08
1.78e-03
1.60c-03
1.71e-03
3.13e-04
5.34c-08
1.54e-03
1.52e-03
1.44c-03
1.46ie-03
avg
1.08e-03
8. 8.19--04
3.63e-03
3.36G-03
3.40e-03
3. 3.09c-03
8 8.29e-04
6.70c-04
2.54c-03
2.35e-03
2.38c-03
2.16e-03
6.86e-04
5.82c-04
1.966-03
1.84c-03
1.85e-03
1.69e-03
3.76e-04
3.20c-04
1.60e=03
1.50--03
1.51e-03
1.38e-03
Edges
7.21le-04
5.83e-04
2.15e-03
1.97e-113
1.97e-03
1.74c-03
5.63e-04
5. 5.16c-04
1.65e-03
1.52e-03
1.52c-03
1.35e-03
4.57e-04
4.67e-04
1.35e-13
1.27e-03
1.25c-03
1.14e-03
3.75e-04
1.25e-04
1.16e-03
Oce 1.10e-03
1.07e-03
9.84e-04
avg
CaOC
Occ
CaOC
Occ
CaOC
Oce
CaOC
Occ
CaOC
Occ
CaOC
Oce
CaOC
Oce
CaOC
Oce
CaOC
Oce
CaOC
Oce
CaOC
CaOC
Oce
Watershed volume
Watershed volune
Watershed volume
Watershed volume
Watershed area
Watershed area
Watershed area
PIR
BPT
BPT
BPT
BPT
Watershed
Table 11:
200
300
400
500
24

---

Published in Transactions on Machine Learning Research (01/2026)
Table 12: Percentage of images with the original class changed after the inclusion (exclusively) of selected explanation regions for Cat vs. Dog dataset. Highlighted in blue are the configrrations presented in the main paper. We tested hierarchies constructed by filtering out smaller regions than 200, 300, 400 and 500 pixels, segmentation based on Edges, Integrated Gradients (IG), Mhided-Backpropagee
(BP), Input X Gradients (I X G) and Saliency. We lested three different strategies to for the first. hierarchical segmentation: BPT, watershed with area attribute, and watershed with volume attribute, We expect smaller rate values of class change.
Cat vs. Dog
of images
VGG
ResNet.
Edges
IG
BP
I X G
Saliency
Edges
IG
BP
X
Saliency
CaOC
0.16
0.49
0.38
0.49
01.37
0.22
0.45
0.28
0.47
0.35
BPT
Oce
0.11
0.18
0.04
0.24
0.11
0.19
0.40
0.18
0.43
0.31
CaOC
0.30
0.45
0.39
0.46
0.42
0.34
0.47
0.43
0.45
0.49
200
Watershed area
Oce
0.22
0.26
0.24
0.31
0.30
0.41
0.49
0.52
CaOC
0.43
0.50
0.45
0.34
0.47
Watershed volume
Oce
0.24
0.30
0.36
0.28
0.29
0.30
0.50
0.46
0.47
0.50
CaOC
0.19
0.49
0.37
0.49
0.36
0.22
0.49
0.28
0.49
0.30
BPT
Occ
0.10
0.27
0.05
0.35
0.10
0.17
0.46
0.22
0.48
0.25
CaOC
0.30
0.43
0.39
0.44
0.43
0.33
0.46
0.40
0.44
0.47
300
Watershed area
Oce
0.18
0.20
0.25
0.24
0.29
0.24
0.46
0.43
0.47
CaOC
0.45
0.43
0.45
0.45
0.46
Watershed volume
Oce
0.21
0.29
0.34
0.27
0.29
0.25
0.49
0.45
0.45
0.45
CaoC
0.21
0.49
0.39
0.49
0.36
0.22
0.50
0.29
0.50
0.29
BPT
Occ
0.10
0.34
0.07
0.40
0.11
0.18
0.48
0.25
0.49
0.27
CaOC
0.26
0.41
0.38
0.44
0.42
0.31
0.47
0.38
0.44
0.46
400
Watershed area
Occ
0.17
0.19
0.21
0.22
0.29
0.24
0.43
0.36
0.40
0.45
CaOC
0.27
0.44
0.43
0.46
0.45
0.33
0.47
0.46
Watershed volume
Oce
0.19
0.26
0.30
0.28
0.28
0,22
0.47
0.44
0.43
CaOC
0.20
U.49
0.41
0.49
0.35
0.22
0.50
0.31
0.50
0,29
BPT
Occ
0.08
0.38
0.11
0.43
0.12
0.16
0.49
0.29
0.50
0.26
CaOC
0.26
0.41
0.37
0.44
0.41
0.32
0.43
0.35
0.41
0.46
500
Watershed area
Oce
0.17
0.19
0.22
0.22
0.28
0.23
0.42
0.32
0.40
0.42
CaOC
0.25
0.45
0.42
0.44
0.32
0.44
0.45
0.45
Watershed volume
Occ
0.28
0.31
0.26
0.29
0.22
0.42
0.41
25

---

Published in Transactions on Machine Learning Research (01/2026)
Table 13: Percentage of images with the original class changed after the inclusion (exclusively) of selected explanation regions for CIFAR10 dataset. Highlighted in bluc are the configurations presented in the main paper. We tested hierarchies constructed by filtering out smaller regions than 4, 16, 32 and 64 pixels, segmentation based on Edges, Integrated Gradients (IG), Suidded-Backroopagatio (BP), Input Gradients (I X G) and Saliency. We tested three different strategies to for the first hierarchical segmentation: BPT, watershed with area attribute, and watershed with volume attribute. We expect smaller rate values of class change.
CIFARIO
% of images
VGG
ResNet.
Edges
IG
BP
X
Saliency
Edges
IG
BP
X
Saliency
CaOC
0.79
0.86
0.87
0.87
0.86
0.81
0.86
0.86
0.87
0.86
BPT
Oce
0.44
0.38
0.45
0.30
0.66
0.51
0.46
0.53
0.41
0.70
CaOC
0.80
0.89
0.88
0.89
0.88
0.83
0.88
0.87
0.88
0.88
Watershed area
Occ
0.55
0.81
0.81
0.81
0.81
0.59
0.81
0.79
0.80
0.80
CaOC
0.80
0.89
0.88
0.89
0.88
0.83
0.88
0.88
0.88
0.88
Watershed volume
0.82
0.82
0.81
0.80
0.81
0.80
0.81
0.79
CaOC
0.83
0.85
0.86
0.87
0.83
0.86
0.86
0.85
0.87
0.83
BPT
Occ
0.54
0.41
0.43
0.44
0,60
0.61
0.51
0.54
0.54
0.63
CaOC
0.80
0.87
0.87
0.87
0.87
0.83
0.87
0.86
0.87
0.87
16
Watershed area
Occ
0.54
0.77
U.76
U.77
0.77
0.59
0.76
0.77
U.77
CaOC
0.87
0.87
0.87
0.87
0.87
Watershed volume
Occ
0.53
0.78
0.78
0.77
0.76
0.58
0.78
0.77
0.77
0.76
CaOC
0.87
0.87
0.87
0.88
0.83
0.89
0.88
0.85
0.89
0.81
BPT
Occ
0.62
0.59
0.52
0.62
0.61
0.68
0.64
0.60
0.68
0.62
CaOC
0.80
0.85
0.85
0.86
0.85
0.83
0.85
0.85
0.86
0.86
32
Watershed area
0.73
0.73
0.74
0.74
0.72
0.74
0.74
CaOC
0.79
0.86
0.85
0.86
0.85
0.83
0.85
0.85
0.86
0.85
Watershed volume
Occ
0.52
0.75
U.75
0.74
0.72
0.58
0.75
0.75
0.75
U.73
CaOC
0.89
0.89
0.88
0.89
0.86
0.90
0.89
0.87
0.90
U.86
BPT
Occ
0.71
0.74
0.66
U.77
0.67
0.75
0.77
0.72
0.79
U.69
CaOC
0.84
0.85
0.83
0.85
0.83
0.82
0.84
0.83
64
Watershed area
0.68
0.67
0.68
0.71
0.61
0.69
0.67
0.68
0.71
CaOC
0.80
0.84
0.84
0.85
0.83
0.84
0.83
0.83
0.83
0.83
Watershed volume
Occ
0.54
0.71
0.73
0.70
0.69
0.61
0.71
0.72
0.70
0.69
26

---

Published in Transactions on Machine Learning Research (01/2026)
Table 14: Percentage of images with the original class changed after the inclusion (exclusively) of selected explanation regions for Imagenet dataset. Highlighted in blue the configurations presented in the main paper. We tested hierarchies constructed by filtering out smaller regions than 200, 300, 400 and 500 pixels, segmentation based on Edges, Integrated Gradients (IG), and Guided-Bacee Bondedd-Bckkroppgaaton (BP). We tested three different strategies to for the first hierarchical segmentation ion: BPT, watershed with area attribute, and watershed with volume attribute. We expect smaller rate values class change.
Imagenet
% of images
VGG
Nes
Edges
IG
BP
Edges
IG
BP
CaoC
0.95
1.00
0.97
0.96
1.00
0.98
BPT
Occ
0.72
0.69
0.51
0.74
0.57
0.50
Caoc
0.98
0.99
0.98
0.98
0.99
0.99
200
Watershed
Occ
0.92
0.93
0.91
0.93
0.94
0.90
CaOC
0.98
0.99
0.99
0.98
0.99
0.99
Watershed volume
Occ
0.91
0.97
0.97
0.92
0.96
0.96
CaOC
0.94
1.00
0.97
0.95
1.00
0.98
BPT
Occ
0.72
0.82
0.55
0.74
0.72
0.55
CaoC
0.97
0.98
0.98
0.98
0.99
0.98
300
Watershed area
Occ
0.92
0.93
0.90
0.92
0.94
0.90
CaOC
0.97
0.99
0.99
0.97
0.99
0.99
Watershed volume
Occ
0.91
0.96
0.96
0.91
0.96
0.96
CaOC
0.94
1.00
0.97
0.95
1.00
0.98
BPT
Occ
0.73
0.88
0.59
0.75
0.81
0.60
CaOC
0.97
0.98
0.97
0.97
0.99
0.98
Watershed area
Occ
0.91
0.93
0.90
0.92
0.93
0.89
CaOC
0.97
0.99
0.98
0.97
0.99
0.99
Watershed volume
Occ
0.90
0.96
0.96
0.91
0.96
0.96
CaOC
0.94
0.92
0.98
0.94
0.86
0.98
BPT
Occ
0.74
1.00
0.63
0.75
1.00
0.63
CaOC
0.96
0.98
0.97
0.97
0.99
0.98
500
Watershed area
Occ
0.90
0.92
0.89
0.91
0.93
0.89
CaOC
0.96
0.98
0.98
0.97
0.99
0.98
Watershed volume
Occ
0.89
0.96
0.96
0.90
0.96
0.96
27

---

Published in Transactions on Machine Learning Research (01/2026)
Aggragated
Aggragated Accuracy
MAN, AUCHS N6
GraDiCA
MNA dUCHE 891
Mr ALCHOARS
Occia AAR-I
TraD Oct,
Trch ALC
DO C--Te Dcc. AUCHOUOO
EIAA ABC-3
Brock CAOC, AUCHO SE2
b]
Aggregated Softmax Curve
Aggregated Accuracy Curve
JPL AUCHOASE
Teat DUt AREHO
THEE OCL AUCHE EID
Trxsh ALCHOROO
AUC-2481
SEWAACCIDE, RUCHE
Figure 10: Softmax (a,c) and Accuracy (b,d) when including regions filtered by different percentage thresholds of most important scores, for VGG16 (a,b) and ResNet-18 (c,d) models. We evaluate each threshold as a hicrarchy level in cight configurations of rAiTrees (C1 and C2), a bottom-up approach (from smaller highly important regions to the bigger structures). We compare these configurations to the baselines: LIME, XRAI, Grad-CAM, Ms-IV, and Occlusion, by filtering the maps using the same threshold. The curves aggregation by the average of 1, 1,000 randomly selected images from Imagenet dataset. AUC values are included in the graphs. Br-TreBB-Occ and However, we notice good early behavior of the methods except for Grad-CAM, Ms-IV and Occlusion.
considerably surpassed the other curves.
28

---

Published in Transactions on Machine Learning Research (01/2026)
Aggregated
Aggregated Accuracy
USB KOC-0.20
AMCHa
AUCHEOAS
AUCHOSE3
Mady AUCHE 756
AUCHE
TRESKOOS ALCHOTES
AUCHE 776
OUL JUC-A BI
CAOL, AUCHE
THsB-CAOC,
Fred CaOC. ALCHOOOA
LA- AUCHOSOE
Aggregated Softmax Curve
Aggregated Accuracy Curve
AUC-0.14
ANC-EE
GESOICAL AUCH0.04
GISDIAAN AUEHD
TEE8-OOL MACHE363
Teat OCL, AUCHO
(I-TENB- AUCHEOOT
(5TeeW-OO Drc AUCHEES7
D3TTou ALEHO DOR
Teeli ALCHO 993
ENTTeD CROC. ALCHO BS0
MP-THMB-COO AUC-EEIE
Figure 11: Softmax (a,c) and Accuracy (b,d) when including regions filtered by different percentage thresholds of most important scores, for VGG16 (a.b) and ResNet-18 (c,d) models. We evaluate each threshold as a hierarchy level in eight configurations of IAilrees (C1 and C2), in a bottom-up approach (from smaller highly important regions to the bigger structures). We compare these configurations to the basclines: LIME, XRAI, Grad-CAM, Ms-IV, and Occlusion, by filtering the maps using the same threshold. The curves are aggregation by the average of 512 images from Cat vs. Dog dataset. AUC values are included in the graphs.
29

---

Published in Transactions on Machine Learning Research (01/2026)
Time analysis: Table 15 shows the execution times to explain image (averaged from 100 randomly selected cat vs. dog images). Tree-Oce is much faster option for providing explanations than SOTA methods such as XRAI. However, the execution time of the xAiTrees framework is related to the choice of scoring method used to assign importance to regions (CaOC, Occ, or LIME), therefore methods such as LIME that take more time to generale explanations will increase the time of xAiTrees, as shown the Tree-LIME example.
Table 15: Execution times to explain an image (averaged from 100 randomly selected cat vs. dog images).
Mean time
TreeB Occ
BP-TreeB Occ
Tree Occ
IG-TreeW
LIME
XRAI
Grad-CAM
Occ.
Ms-IV
100 Images
Oec
Mean
7.87
11.97
0.009
6.65
0.27
0.60
0.65
0.76
1.31
Avg.
0.82
1.25
0.0002
2.47
0.03
0.08
0.09
0.10
0.08
Mean
5.64
10.19
0.007
3.41
0.17
0.50
0.57
0.58
0.84
ResNet
Avg.
0.40
1.36
0.0002
0.03
0.07
0.06
0.05
0.06
0.07
TreeB
BP-TreeB
Trree
IG- WreTTreWW
TreeB
BP- TreeB
Tree
IG- TreeW
CaOC
CaOC
CaOC
LIME
LIME
LIME
LIME
Mean
0.71
0.81
1.53
31.91
11.07
49.96
51.98
VGG
Avg.
ResNet
0.68
1.04
19.44
Avg.
0.08
0.08
0.10
0.07
4.75
3.99
6.99
2.90
30

---

Published in Transactions on Machine Learning Research (01/2026)
A.6 Qualitative analysis
Models' description: Table 16 shows the number images in the train set for three Cat vs. Dog 18 biased models. For Bias 1 the biased class is composed of only cats on top of cushions. For Bias the biased class is composed of only dogs next grades. For Bias 3 the biased class is composed of only cats with humans. We also include the accuracy percentage per class when predicting non-biased validation set. composed by 5,109 images.
The biased models were trained with initial weights from Imagenet, learning rate 5e - 7, cross-entropy loss, the Adam optimizer, and early stop in 20 epochs of non-improving validation loss.
Table 16:
Number of images (for a normal and an induced biased class) for training three biased ResNet18 models. We also present the accuracy of the models when predicting each class image from a non-biased validation dataset (5, 109 images).
Acc. orig. val normal (%)
Acc. orig. val bias (%)
Normal class
Bias class
Bias
138
86.91
69
84.82
Bias
85
97.97
56
37.81
Bias
161
86.28
46
80.96
Comparison of misclassified images: Considering the hierarchical characteristic of our methodology, we can perform deeper analysis of the explanations by selecting regions by the percentage of importance be visualized, as shown in Figure 12. In the last level of the dishwasher example, the model seems to focus on the cat's dish after having focused on the sink (in the previous level).
Tree-Occ
Tree-Occ 0.25
Tree-Occ 0.50
Tree-Occ 0.75
chainlink fence
hamper
dishwasher prison
Figure 12: Different visualization levels on the explanation hicrarchy. We illustrate deeper analysis of the explanations of four images from Figure using Tree-Oec (minimal region size 500 pixels). We can note the evolution of the importance in the images' shapes, for examples: in the hamper image, although the hamper is the most important, the cat has also an important that disappears at the more selective level (Tree-Ocs 0.75); in the dishwasher the initial explanations show the sink as important but the most selective level, the cat's dish is the only remaining. This analysis can be helpful to understand the reasoning behind predictions.
31

---

Published in Transactions on Machine Learning Research (01/2026)
Human evaluation in bias analysis: As mentioned the paper, used the configuration C1 for human-interpretatioo evaluation compared to baseline techniques: IG, Grad-CAM, OCC, LIME, Ms-IV, ACE. We presented the same five image visualizations (from corrected classified images by the biased class) for the baseline methods and the methods from C1.
The idea was to analyze the impact of the visualizations on people from different backgrounds. We limited ourselves to people over the age of 18 and recorded their selfiidentificttion as expert, non-expert, field of expertise, and country. We show some statisties of each group of 41 people that participated in our evaluation.
Expertise areas:
48.8% of people from computer science;
17.1% of people from human sciences;
19.5% of people from life sciences;
9.8% of people from exact sciences (not in Computer Science);
4.9% of people from none of the above.
Graduate degree:
20.0% of PhDs;
32.5% of Masters;
30.0% of Bachelor's degree;
17.5% of High school diploma.
AI expertise:
22.0% None;
36.6% Basic;
9.8% Intermediate;
7.3% Advanced;
24.4% Expert (working with AT).
We intended to verify if: (i) humans can detect the wrong focus given based on a class prediction (Detection): and (ii) humans can recognize which was the cause of the bias (Identification).
To test (i) and (ii), for each Bias type we produce for each of the xAI methods an explanation image. By presenting five image explanations (the same images) for each of the xAI methodologics, we asked voluntcers, based on the explanations provided, what did they think the highlighted regions referred to. The five image explanations presented in Figure 13 for each Bias type (1-(a), 2-(b), and (c))..
32

---

Published in Transactions on Machine Learning Research (01/2026)
Here,
display
the text provided to the volunteers for this experiment:
[FORM] Part I Determining the focus of the images: For each question, we provide two rows of images:
The first row displays the original images, each representing a specific class.
The sccond row showcases an image for each image from the first row, highlighting the important parts for the class.
[IMPOTANT]] What is a class?
A class refers to category type of object, animal, characteristic depicted in the images. For instance, class of cat images would include images featuring cats, while a class of dog images would comprise images featuring dogs. Similarly, class of cartoon images would include images characterized by cartoon- features. In essence, a class represents a distinct category used to classify and organize images based on their content or characteristics.
Throughout the questions, our objective is to identify the common important parts present in the images of the first row, indicated by the corresponding images in the second row.
If no common important parts are identified for most of the images, the answer should be Not identified. And for each method visualization:
For the following three questions, the second row of images displays significant image components to the class of the animal. What are the significant components of the images highlighted, as depicted in the second row of images?
To test ACE similarly we did with the other methods, we highlight the top five concepts found (described as sufficient in the original ACE paper (Ghorbani et al., 2019)) in the same five selected images. However, we also show the visualizations of the ten most activated images for the top five found concepts in Figure 14.
In our final qualitative experiment, using the same methods as the previous human evaluation, we presented four image explanation visualizations for non-biased models to determine xAI model prefer- ences. The images are presented in Figure 15. We presented the following explanation and question: [ORMM] Part II: Choosing the best representation:
For the next questions, you will be asked to answer which image number do you prefer to describe the class we indicate.
You should choose the image that seems to highlight class features in an easier way to understand. Which image do you think better shows representative parts of the animal?
For the two first images (Figures 15 (a) and (b)), over 70% preferred Tree-Oce and Tree-CaOC: over others. For the third image (Figure 15 (c)), IG was preferred by 26.5%, followed by Tree-OCC and Occlusion with 20.6%. Grad-CAM was preferred in the fourth image (Figure 15 (d)). with 60.6%, followed by Treo-CaOC with 18.2%. The visualizat ions suggest preference for explanations that highlight the complete concept (cat. dog) rather than focusing specific small animals' regions.
33

---

Published in Transactions on Machine Learning Research (01/2026)
Original
Grait CAN
LIKE
NeAv
NEJU
Tree Oic-Oot Tree Parv
Tree-Or Thae-Weru
Thea-Purl
Thee-drt The
IC-
Than One
(a) Cushions
(b) Grids
Original
Gra
Oct
LINE
ACE
Tree-Wer Thee Thee Moi
(c) Humans
Figure 13: Explanations visualizations used on our human-based evaluations for bias detection and identification, of all the ten compared methods: IG, Grad-CAM, OCC, LIME, Ms-IV, ACE, Trle.MsIV, Tree-Oee, IG-Trcc-Msiv, and IG-Treo-Occ. We showed the same five image explanations for all the methods.
34

---

Published in Transactions on Machine Learning Research (01/2026)
(a) Cushions
(b) Grids
(c) Humans
Figure 14: Original explanations of the top 5 concepts generated by ACE for the three biased modcls. Instead of showing the 10 most concepts' activated images we draw these five top concepts on the selected images from Figure 13 have a fairer comparison with the other methods.
A.7 Possible adaptations of the framework
Regarding the adaptability of the framework for using other xAI methods to score regions, present. in Table 17 preliminary results of the exclusion of important regions experiment (Section and A.5) by using LIME in the place of Occlusion and CaOC. The results demonstrate Tree-LIME improves the class change under important regions' occlusion for the Cat vs. Dog dataset. (compared to Table 1). We also show two image explanations generated by Troo-LIME in Figure 16. We do include this variation in all the experiments due to the time consumption (Table 15). However, these preliminary results demonstrate Tree-LIME increase the class change (Ch.) reaching higher percentages than the best configuration presented in the main paper.
Regarding the adaptability of the framework to other lasks such as learning representations, one suggestion would be calculating the distance between the two learned representations (original and after occlusion) to attribute regions' scores instead of verifying the logits different as classification tasks. The type of distance applied should be tested. A more sophisticatod evaluation of impact (scoring the hicrarchy regions) would be to include network to quantify the quality of the representation for the task at hand.
35

---

Published in Transactions on Machine Learning Research (01/2020)
Original
ACE
Tree HaiV
IC Dee
IC- Haff
ACE
Tree-Oc
Tree-Matt
IC Oet
IC-Tre Hal
Line
NuW
IG
Occ
LIKE
Na-VV
Gred CAM
original
Original
IC-Tre--Mal
IG-hre--atl
Figure 15: Explanations of visualizations used on our human-based evaluations for preference analysis, of all the ten compared methods: IG, Grad-CAM, OCC, LIME, Ms-IV, ACE, Treo-MsIV, Treo-Oce, IG-Troe-Msiv, and IG-Tree-OO Occ. We showed the same five image explanations for all the methods.
Percentage of images with the original class changed after the exclusion of selected explanation regions. We tested TreeW, IG-TreeW, TreeB, and BP-TreeB combined to LIME (instead using occlusion score regions) in two architectures, VGG-16 and ResNet118, for the Cat vs. Dog dataset. We compare the results of the four configurations the best configuration using Occlusion (BP-TreeBB-O p-score in brackets (Menemar test). We expect higher percentage of class change (Ch.) when the region is excluded. Same column shows images maintaining the original class when the output was reduced, and Total is the sum of class change (Ch.) and class reduction (Same).
Table 17:
showing the
Cat
Dog
% of images
VGG
ResNet
Ch.
Same
Total
Ch.
Same
Total
Tree W-LIEE
0.34 (0.0)
0.64 (0.0)
0.98
0.43
(1.2-4)
0.56 (7.2-10)
0.98
IG-Tree W-LIME
0.45 (2.2-9)
0.54 (2.7-10)
0.99
0.43
(2.0-4)
0.55 (2.5-9)
0.98
TreeB -LIME
0.57 (0.05)
0.42 (0.01)
0.98
0.64
(2.3-3)
0.35 (0.34)
0.99
BP-TMeeB-LIME
0.70 (2.2-9)
0.29 (2.7-10)
0.99
0.77
(2.0-4)
0.21 (2.5-9)
0.98
BP-TreeB-Occ
0.63
0.35
0.98
0.55
0.37
0.92
Figure 16: Examples of Tree-LIME demonstrating the adaptability of xAiTrees framework.
Concerning other modalities, such as text as input, we could consider a tokenization process as the segmentation. A first idea would be to define a tokenization algorithm that can learn merge rules and use them to construct the segmentation tree. Each node this tree can be considered a segment. Another idea would be to use grammar rules to define parts of sentence, such nouns, verbs, and declinations. However, this approach would be focused specific language.
36


Reviews:
[{"review_id": "l04EYLjU6Y", "reviewer": "Reviewer_9sNp", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Although explainability (xAI) is a very relevant and timely area of research, there are a number of problems with the paper. The main problem has to do with the limited novelty: the proposed methodology puts together a number of existing methods and approaches with some justification for the choices made, but the paper does not introduce a new xAI approach. The main motivation is rather flawed: reasoning! Despite appearing in the title of the paper, there is very little about reasoning that is discussed or formally defined in the paper. As a result, there is no relationship to be identified between the proposed methodology and reasoning as understood in the context of knowledge representation in AI. Another main problem is the limited related work. \n\nThe paper focuses on local explainability of images and uses a well-known image xAI method (LIME) in its pipeline. It is well-known now that LIME can be very limited in what it explains (https://arxiv.org/abs/1908.03020) and how that explanation is measured. A reference is made in the paper to the \"fidelity\" of the explanation. This can be measure (see e.g. https://arxiv.org/abs/2106.14556), but the experimental evaluations are mostly concerned with specific cases and subjective evaluations. The limitations of occlusion based approaches are also well-studied. Although the tree structure is used in the proposed methodology to reduce the problem of explainability based on occlusion, the main evaluation is based on the direct comparison of occlusion inside and outside the hierarchy. It is difficult to see how such a measure can be reliable. This kind of evaluation seems to be highly context dependent. The rest of the evaluation is based on human inspection. Here, a lot more detail would need to be provided on the evaluation method, how many participants, questionnaires, etc. The pointed out limitations of Grad-CAM are also well known. In summary, with limited novelty, one would expect extensive and detailed critical comparative evaluations of results to have been carried out, but the empirical analysis is also limited, as already noted. \n\nThe metric \"original class changing after the exclusion of selected explanations\" seems to tell us very little. The process that generates the segmentation hierarchy is crucial and is not presented in any level of detail. What motivated the choice of the specific shaping approach? \nVisualization by itself does not really provide explanations and I can't see at all how it might help with reasoning. The \"occlusion scores\" are not defined. The choice of certain approaches to be part of the proposed pipeline is said to be based on the fact that the methods are state of the art at pixel-wise importance attribution. This claim is not backed by references. State of the art wrt which metric?\n\nThe generation of the initial tree and the process to refine the tree are crucial but are discussed in little detail. I'd refer to this paper which extracts a tree from a CNN layer: https://ceur-ws.org/Vol-3432/paper3.pdf. It could offer a way of comparing results, even though the method used there isn't for local xAI.\n\nHow would the results be affected by the performance of other approaches such as Segment Anything? Various other approaches exist that seek to benefit from some concept hierarchy, typically in the learning of the part-of relation (see e.g. https://arxiv.org/abs/1705.08968). Some comparison or at least a discussion of pros and cons, and limitations of the proposed method, should have been discussed.", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "No", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "The paper proposes a methodology for explaining images classified by neural networks followed by a number of experimental evaluations.", "claims_and_evidence_yn": "No", "claims_explanation": "", "audience_yn": "Yes", "audience_explanation": "", "requested_changes": "see above", "broader_impact_concerns": "not applicable", "additional_comments": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "nZr9eGuedo", "reviewer": "Reviewer_3YHd", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "The method seems really interesting, even if the paper does not really convey why the hierarchical structure is needed, and the provided illustrations are not really convincing.\n\nThe main issue addressed in this paper is not precisely defined: Use a bold sentence to tell the reader what the main addressed problem is.\n“We propose to explore the trade-off between explaining Convolutional Neural Networks (CNNs) with model faithfulness and human interpretability.” This is not a clear definition of the problem, thus a loss of space (and time). Then “we introduce a framework that combines hierarchical segmentation with region-based explanation methods.” Does that mean that no other framework already does this ? If so, then what difference does it have to yours? Why would PMLR readers use this framework rather than any other one (such as Multiscale Interpretable Visualization)?\n\nYour contribution seems to be something like “we provide an explanation-model agnostic framework to transform pixel-wise attribution methods to region-based ones”. (correct me if I am wrong, I have not really been able to identify the exact problem+contribution.)\n“We can transform any pixel-wise attribution into region-based explanations.”\n\nConfusing vocabulary: “Reasoning” is usually used to refer to logic based (or at least symbolic) agents. CNNs do not reason per se, they infer. \n\nBold unsupported claims: see in the next section for examples.\n\nUnclear figures: This is a crucial point for me. see next section for requested changes.\n\nNo limitation figure", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "First, let me apologize for the delay. I had done the review once but unfortunately lost my notes and could not recover them. I thus did this review a second time.\nThe authors developed a hierarchical segmentation based XAI framework to improve CNN’s model explanations, applied to OCC, LIME, … etc. They experimentally show that their methods identify important zones of the images for the decisions, and that it augments bias detection and identification scores. \n\nMost crucial claims seem to be supported by the experimental evidence, but they are not clearly outlined. \n\nWith a greatly improved presentation, this paper should be of interest for PMLR's readers.", "claims_and_evidence_yn": "Yes", "claims_explanation": "", "audience_yn": "Yes", "audience_explanation": "", "requested_changes": "**Improve clarity of your contributions in the introduction.**\n\nMany sentences bring no new/relevant information, due to their imprecise aspect. E.g. “*This need arises from biases present at various stages of model development and deployment*.”\n\nTime and space are precious values, use it to precisely explain the problem you are addressing, the method you developed, the evaluation you conducted.  \\\n“*To meet the demand for explanation,*” can be omitted.\n\nPlease verify that your claims are supported by published available work, or by your experimental evaluation. e.g.:\n\n* “*This needs to be clarified: “using a segmentation framework introduces human bias, which [...] may reduce faithfulness to the model’s actual behavior.*” Why ?\n* “*However, these methods are approximations of model behavior. Different techniques prioritize either faithfulness to the model’s behavior or human interpretability, posing a challenge in balancing the two.*”  \n\nIf existing published work supports this, just point to it, otherwise, you need to develop your argumentation. Anyways, not sure if these help the reader understand which precise scientific problem you are trying to tackle.\n\n**Use figures as the concise version of your paper.** Many readers, after reading the abstract, skim through the figures and tables. Thus, I advise incorporating these points within the figures/tables:\n\n\n\n* What is the (unsolved) problem at hand that your method tackles or improves?\n* How does your method work (nicely done in Figure 3)\n* Different figures that each answer an evaluation criteria. Ideally, your experiment section is composed of precise scientific questions aimed to be answered. For example: \\\n5.1 Does xAITrees provide more accurate important regions ?   \\\n5.2 Does xAITrees help with bias detection and identification ?  \\\n\nEach of your figures then answer these questions. Please convert difficult to read tables into bar diagram, and place the table in appendix.\n\n**Clarify your figures, and their captions**. E.g. Figure 1. This figure is supposed to highlight the main scientific problem addressed in this paper. Thus, the first sentence should describe it (in bold). E.g.:\n\n>**Current explanation methods are difficult to interpret, our Tree-Occ method improves interpretations**, as shown by the fact that it highlights the fence…\n\n(I am personally not convinced by the “better” explanation of Tree-Occ here). A user study might be useful to determine if the Tree-occ’s segmentation provides better interpretations. \n\nOn this figure, the y label is “Imagenet classes”, that should correspond to the label associated with the depicted images, within the Imagenet dataset. However, the caption claims that these are classes predicted by VGG and Resnet. I doubt that, seeing that some have one, and others many classes. Do VGG and Resnet agree on the classes to output for all of these classes? \n\nIt seems that for each explanation model, the first column is the explanation map, and the second is the aggregated mask and image. There is thus only one mask per image, does it come from VGG or ResNet ? \n\nFinally, your example of the cat image is quite interesting. There are indeed several objects corresponding to classes present in imagenet (the cat and the dishwasher). You are writing: “* Tree-Occ avoids the mistake of highlighting the cat when the models predict classes such as dishwasher*”. Does the used model *really* predict “*dishwasher*” ? I would expect it to predict “*cat*”, particularly when looking at the other explanation methods. (this also applies to the other depicted images containing cats).\n\nFigure 2 seems to be an experimental result, supporting a point, why is it placed so early in the paper? Again here, no clear message can be extracted. Figures should be standalone, and understandable even if the reader has not yet read the full text. \n\nFor Figure 4, you want to show the superiority of Tree-method over method, then use e.g. dotted line for the original method and full line for your adapted ones, keep the same color to compare them. Further, it seems that TreeB-Occ and TreeW-Occ really underperform there. While you can comment on this for precision and fairness of your method, you can drop them from this figure to augment the visual clarity (btw, you could use smaller linewidth, and a smaller alpha of e.g. 0.8). \n\n\n**While the reported results of table 4 are very relevant, the metrics are really unintuitive.**\n\n“Animal” and “Not identified” are imprecise labels that give the wrong intuition. Reading “Identification” and “Not identified”, I get the impression that the first one is 100 - the other. This table also lack second order measures (like std)\n\n**No *Limitation* section**. The paper should include a limitation section. Is your method applicable to more modern architectures ? Only if these explanation methods are ? What is the time needed to generate an explanation compared to the baselines ?", "broader_impact_concerns": "Impact of better explanations methods, and bias identification could be discussed in a dedicated section.", "additional_comments": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "ifTmLZ6bQN", "reviewer": "Reviewer_PDyw", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strengths: \n- Novelty in Model-Based Segmentation: The introduction of model-based hierarchical segmentation, where the segmentation hierarchy is derived from pixel-wise XAI attributions (like IG, BP), is a novel and interesting contribution. This aims to align the segmentation structure more closely with the model's internal reasoning compared to standard, human-centric segmentation.\n\n- Comprehensive Framework (xAiTrees): The paper proposes a well-structured, four-step framework that integrates hierarchical segmentation, attribute computation (importance scoring), tree shaping (persistence analysis), and hierarchical visualization. This provides a systematic approach to generating multiscale explanations.\n\n- Framework Flexibility: The paper demonstrates that xAiTrees can be adapted by plugging in different segmentation criteria, hierarchical segmentation algorithms, and region scoring metrics.\n\nWeaknesses:\n- Complexity of the Framework: The multi-step process involving choices of segmentation type, pixel-wise method (for model-based), occlusion metric, tree shaping, and aggregation heuristic adds complexity. Understanding the contribution of each component and tuning the framework might require significant expertise.\n- Interpretation of Model-Based Segmentation: While novel, the claim that segmentation based on IG/BP truly reflects the model's \"vision\" warrants further discussion. These methods capture gradient information, but whether the resulting segments align with meaningful semantic concepts used by the model (or just group pixels with similar attribution values) is not fully explored. There's a potential risk of the explanation being an artifact of the specific pixel-wise method chosen.\n- Dependence on Base Methods: As acknowledged, the quality of the final explanation heavily depends on the quality of the underlying segmentation algorithm and the chosen region importance scoring method (Occlusion, CaOC, etc.). xAiTrees inherits the limitations of these components.", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "This paper introduces xAiTrees, a framework designed to generate more faithful and interpretable explanations for CNNs by utilizing hierarchical segmentation techniques. Addressing the trade-off between model fidelity and human understanding seen in existing XAI methods, xAiTrees constructs multiscale explanations by combining hierarchical structures (derived from either traditional edge detection or novel model-based attribution maps) with region importance scoring (like Occlusion). The authors claim this approach, particularly the model-based segmentation variant, significantly improves upon methods like LIME and Grad-CAM, offering clearer, more precise identification of impactful regions, better elucidation of misclassifications, and enhanced effectiveness in detecting model biases, as supported by quantitative evaluations and human studies.", "claims_and_evidence_yn": "Yes", "claims_explanation": "", "audience_yn": "Yes", "audience_explanation": "", "requested_changes": "Please try to discuss more about the weaknesses.", "broader_impact_concerns": "NA", "additional_comments": "", "extra_scores": {}, "extra_text": {}}]

Author rebuttal:

Decision: Accept (poster)