From Pixels to Perception: Interpretable Predictions via Instance-wise Grouped Feature Selection
Authors: Moritz Vandenhirtz
Julia E Vogt
Venue: ICML 2025 poster (2025)
Keywords: instance-wise feature selection
feature selection
interpretability
perception-adhering masking
Primary area: social_aspects->accountability_transparency_and_interpretability
TL;DR: We propose an inherently interpretable method whose prediction bases on a learned masked version of the input image, designed to align with human perception.

Abstract: Understanding the decision-making process of machine learning models provides valuable insights into the task, the data, and the reasons behind a model's failures. In this work, we propose a method that performs inherently interpretable predictions through the instance-wise sparsification of input images. To align the sparsification with human perception, we learn the masking in the space of semantically meaningful pixel regions rather than on pixel-level. Additionally, we introduce an explicit way to dynamically determine the required level of sparsity for each instance. We show empirically on semi-synthetic and natural image datasets that our inherently interpretable classifier produces more meaningful, human-understandable predictions than state-of-the-art benchmarks.

From Pixels to Perception:
Interpretable Predictions via Instance-wise Grouped Feature Selection
Moritz Vandenhirtz ! Julia E. Vogt'
Abstract
The LASSO by Tibshirani (1996) pioneered the learned se- lection of features. While LASSO's global selection is fixed across the dataset, Chen et al. (2018) argue that the subset of relevant features can differ for each data point and formally introduce instance-wise feature selection. Since then, there has been a growing interest in using instance- wise feature selection for the image modality (Jethani et al., 2021; Bhalla et al., 2024; Zhang et al., 2025). As methods adapt from tabular to image data, they need to define the feature space over which to carry out the selection. An intuitive choice is to directly sparsify on pixel-level, generally aiming to mini- mize the number of active pixels while retaining as much predictive performance as possible. In this work, we ques- tion this choice and argue that for human-understandable predictions, the sparsification of features must occur in the space of perceptually meaningful regions.
Understanding the decision- docision-making process of machine learning models provides valuable in- sights into the task, the data, and the reasons be- hind a model's failures. In this work, we propose method that performs inherently interpretable predictions through the instance-win wise sparsifica- tion of input images. To align the sparsification with human perception, we learn the masking in the space of semantically meaningful pixel re- gions rather than on pixel-level. Additionally, we introduce an explicit way to dynamically deter- mine the required level of sparsity for each in- stance. We show empirically on semi-synthetic and natural image datasets that our inherently in- terpretable classifier produces more meaningful, human-understanddbb predictions than state-of- the-art benchmarks.
Humans perceive an object not as individual pixels, but as a function of its parts: structural units that are perceptu- ally meaningful atomic regions (Palmer, 1977; Biederman, 1987). As these parts differ for each instance, the selec- tion should not rely on fixed pixels or patches, but rather on instance-wisg wise regions of semantic meaning. We show in Fig- ure that masking directly in pixel space allows for simple, undesired solutions of high sparsity but low informativeness. That is, an uninformative, evenly spaced mask can cover most of the pixels while leaving the predictive performance unaffected; behavior that does not provide any insights into the model behavior. Thus, we argue that, similar to how Group Lasso (Yuan & Lin, 2006) jointly sparsifies all components that make up feature, one should jointly mask all pixels that make up a perceptually meaningful region. Similarly, we advocate for binary instead of continuous- valued masks such as those in COMET (Zhang et al., 2025), as dimming pixels may obscure information to the human eye, but it does not meaningfully affect classifiers.
1. Introduction
Knowledge is power (Bacon, 1597). Knowing how ma- chine learning models make their predictions empowers users to understand the underlying patterns in the data, iden- tify potential biases, reason about their safety, and build trust in the models' decisions. In high-stakes domains, where model decisions can have significant consequences, the field of interpretabilityy is crucial to provide this under- standing (Doshi-Velez& & Kim, 2017). In this work, we focus on providing inherent interpretability- - subfield where the explanations are necessarily faithful to what the model com- putes (Rudin, 2019)- - through the lens of sparsity in the input features (Lipton, 2016; Marcinkevičs Vogt, 2023). This constraint makes it easier to understand which specific inputs contribute to output (Miller, 1956), leading to more human-understandable predictions.
In this work, we propose P2P, novel method that learns mask in the space of semantically meaningful regions?. This unit of interpretation is already prominent in the field of explainability (Ribeiro et al., 2016; Lundberg, 2017), but to the best of our knowledge, we are the first work to consider this input-dependent definition of a feature within
'Deparmeent of Computer Science, ETH Zurich, Switzerland. Correspondence to: Moritz Vandenhirtz <morriz.vandenhittzee enh.cthz.ch>
Proceedings of the 42nd International Conference Machine Learning. Vancouver, Canada. PMLR 267, 2025. Copyright 2025 by the author(s).
'The code is here: www. www.github. com com/mvandenbii/22

---

From Pixels to Perception
Inherent Interpretabilityy In this work, we focus on in- herently interpretable models. Here, by design, the expla- nations faithfully capture model's behavior (Rudin, 2019). These explanations can come in many forms, as they rely on the chosen model class. An established class of inherently interpretable methods are prototypes- models (Kim et al., 2014; Chen et al., 2019; Fanconi et al., 2023; Ma et al., 2024) that make a prediction based on the similarity of an input sample to prototypical data points of each class. An- other line of work utilizes intermediate, human-interppe numaaninterpretable concepts upon which the final prediction is based (Koh et al., 2020; Espinosa Zarlenga et al., 2022; Marcinkevičs et al., 2024; Vandenhirtz et al., 2024; LCM team et al., 2024). Also, Carballo-Castro et al. (2024) and Huang et al. (2024) show that these concepts and prototypes can be combined.
Original
Pixel Mask
Dark Mask
P2P
Figure I. Masking 80% of input under different constraints. All 3 masks lead to similar predictive performance but only P2P provides interpretability by sparsity.
the context of instance-wis feature selection. Moreover, the proposed approach is equipped to model the relationship across parts to capture part-object relations. Lastly, we address the limitation of a fixed level of sparsity and propose a dynamic thresholding that adapts to the required amount of information needed to make an informed prediction. The proposed approach achieves high predictive performance while relying on sparse set of perception-adhentee regions, thereby enabling human- numan-understandabbl predictions.
Another notable approach is B-cos networks (Böhle et al., 2022; 2024; Arya et al., 2024), which achieve inherent inter- pretability by constraining the model's dynamical weights to align with relevant structures.
Main Contributions This work contributes to the line of research on instance-wist feature selection in multiple ways. (i) We propose a novel, semantic region-based approach to sparsify input images for inherently interpretable predic- tions. (ii) We propose dynamic thresholding that adjusts the sparsity depending on the required amount of informa- tion. (iii) We conduct a thorough empirical assessment on semi-synthetic and natural image datasets. In particular, we show that P2P (a) retains the predictive performance of black-box box models, (b) identifies instance-specific rele- vant regions along with their relationships, and (c) faithfully leverages these regions to perform its prediction. Together, these results highlight that our method represents a signifi- cant advancement the field of inherent interpretability.
Instance wise Feature Selection This work operates in the subfield of instance-wise feature selection. Based on classical statistics (Tibshirani, 1996; Yuan & Lin, 2006), se- lecting a subset of important features has been a paramount problem with many applications (Saeys et al., 2007; Reme- seiro & Bolon-Canedo 2019; Tadist al., 2019). Due to the vast amount of data available nowadays, methods are so complex that they select a sparse set of features on an instance-level, upon which prediction is made. Chen et al. (2018) introduced this instance-wise feature selection and select a feature subset by maximizing the mutual infor- mation of the subset and the target. Others take a similar approach and minimize the KL divergence of predicting with the full set versus the subset (Yoon al., 2018; Covert et al., 2023). A commonality of all presented methods is that a forward pass consists of (i) a selector that determines the masking, and (ii) a classifier whose predictions are based on the masked input. REAL-X (Jethani et al., 2021) argue that the selector can encode predictions within its mask and train the classifier with random masking to avoid this shortcut (Geirhos et al., 2020). With similar goal in mind, iterative
2. Related Work
Interpretability Interpretability is useful in situations where a single metric is an incomplete description of the task at hand (Doshi-Velez & Kim, 2017). The field can be broadly divided into inherent interpretability and post- hoc explanations (Lipton, 2016; Rudin, 2019; Marcinkevičs & Vogt, 2023). Post-hoc explainability methods try to ex- plain the outputs of black-box models by attributing each input feature with an importance score. The most common approaches for measuring feature attribution are approx- imations via locally linear models (Ribeiro et al., 2016; Lundberg, 2017). or aggregating gradients with respect to the input (Springenberg et al., 2014: Selvaraju et al., 2017: Shrikumar et al., 2017; Sundararajan et al., 2017). However, an issue of such methods is that they explain a complex black-box box model by simplifying approximations. As such, it is unclear whether the model that is being explained be- haves similarly to its approximation (Adebayo et al., 2018: Hooker et al., 2019; Fryer et al., 2021; Laguna et al., 2023).
Oosterhuis et al. (2024) define the selector process that only sees the previously selected features. RB- AEM, the feature selection stage of ISP (Ganjdanesh et al., 2022), expands upon REAL-X by introducing a geometric prior that avoids independent pixel selection. More recently, DiET (Bhalla et al., 2024) find a mask by ensuring a robust, distilled model's predictions align with a pre-trained model. Lastly, COMET by Zhang et al. (2025) learns mask such that the discarded pixels are uninformative for a pre-trained classifier. Notably, COMET's masking is not binary but continuous-valued - a design choice that boosts predictive performance but raises questions about the contribution of supposedly unimportant, dimmed pixels on the prediction.
2

---

From Pixels to Perception
3. Method
of architectures that can directly adapt to different input shapes for future work. Upon having obtained selection probability for each region, we employ the Gumbel-Softmax trick (Jang et al., 2017: Maddison et al., 2017) to sample a mask from ((x) while preserving differentiability.
Formally, wise feature selection in image classifi- cation seeks
arg min me (0.1)
||>=]]| s.t. xm) = p(y)))) 2)),
(1)
Recall the optimization problem in Equation (2), Apart from maximizing the predictive performance given the masked input, we require a sparsity loss that regularizes the active number of pixels. Note that we regularize the number of pixels, not the number of regions, to avoid introducing an inductive bias toward selecting larger regions. We denote the expected number of selected pixels as
where denotes the mask that is defined over pixel space with dimensionality (H, W) and I.m = m @ æ. An excep- tion is COMET (Zhang et al., 2025) that defines m E (0, 1). As outlined in Section and supported by Figure 1, there exist many meaningless m that appear to be good solutions to Equation (I). This is because the masking of a pixel does not meaningfully alter the content of an image. To allevi- ate this problem, we propose to optimize over the space of semantically meaningful atomic regions such that each selectable feature is a perceptually meaningful part. We de- note this partition by Ω = {R,,.... RD ). Note also that the use of patches to reduce the masking space dimensionality does not fulfill these considerations, as they do not depend on the content of an instance. In line with most related work mentioned in Section 2, we believe the selection must be binary, as continuous-values mask does not remove any information on the numerical level in which machine learn- ing models operate. Lastly, we reformulate the optimization problem to avoid the overhead of estimating p(yl æ)) and to allow for a more explicit control over the sparsity regularizer. The reformulation similar in spirit to Chen et al. (2018)'s s motivation and will prove useful when introducing dynamic thresholding. Our method P2P optimizes
Σ j=1
p HW
Pmy PmylByl
To fulfill the thresholding inequality, we define the following masking loss
- llo((1 - -p) pf
Cm =
(3)
otherwise,
where the non-zero component can be interpreted as (pollp) with Po acting as a prior that encourages p to go towards zero. The formulation with a threshold alleviates the need to perform an extensive hyperparameter search to obtain desired masking level, as p is no longer regularized as soon as it is below т. That is, the regularization loss allows for explicit control over the desired sparsity strength.
Relationships of Parts Aristotle said, "the whole is not the same as the sum of its parts" (Ross, 2016), but so far, each region is treated separately by our selector. Modeling the part relationships can enhance the selector's learning and selection capabilities. It can be used steer which re- gions should be selected together, thereby highlighting their complementary value. As such, modeling the relationships of parts can inform which parts form a whole and identify object-contes relationships. Such an analysis can provide additional insights into the model's understanding of the input, thereby enhancing its inherent interpretabillty.
arg max p(y| s.l. (0.1).
HW Σ
my RIIST
where RRy denotes the number of pixels in region Ry and the masked input I.m is defined elementwise as (xm)) = Ther if it's corresponding region Ry is activated (i.e, my = 1). and 0 otherwise. In summary, P2P maximizes the like- lihood while constraining the number of active regions, weighted by their pixel count.
From Pixels to Perception In Figure 2, we present schematic of our method From Pixels to Perception (P2P). To obtain instance-wise perceptually meaningful atomic re- gions Ω we use the SLIC Superpixels algorithm (Achanta et al., 2012), designed to quickly generate such regions. An ablation study Appendix B shows that the specific choice of algorithm not important. With additional do- main knowledge, more informed selection could be made, provided the proposed regions are sufficiently fine-grained and computationally efficient. To address the challenge that regions vary per- instance, we perform masking by predict- ing the selection parameters at the pixel-level and aggre- gating them within each region. We leave the exploration
We equip P2P with part-relationship modeling capabilities by parameterizing the part selection probabilities pm as non- dia-oiagonal logit-normal distribution' (Atchison & Shen, 1980):
Pm LogitNorrma((4, Σ)
Modeling probabilities with a logit-normal distribution has previously proven effective in the context of segmenta- tion (Monteiro et al., 2020), rectified flow (Esser et al., 2024), and concept-based models (Vandenhirtz et al., 2024).
AA logit-normal distribution is probability distribution of a random variable whose logits are normally distributed.
3

---

From Pixels to Perception
Importance Predictor
Xm ~ D(x)
"Primate"
Region Proposer
Figure 2. Schematic overview of P2P. A frozen region proposer partitions the input into perceptually meaningful parts Ri.D, which assigned a learned selection probability. The mask is then binarized by sampling, leading to æ... that serves as the input to the classifier.
As described in the previous paragraph, the region- wise parameters predicted at pixel-level and then aggregated within each region. To ensure that the covariance matrix is positive semi-definite, we do not directly predict its entries, but characterize each entry as a dot product of learnable part-specific embeddings E, R. (k),n)) (E(x)) hor Thus, the parameters of the logit-normal distribution are computed as follows:
time.
The masking threshold is found by observing the certainty of the classifier:
inf(r') max p(ve
(mm)) 28,,
(4)
CEC
where p(- denotes the classifiers predicted probabilities for given class and 8 is the certainty that the user wants to obtain. Importantly, P2P only looks at the certainty, but not the actual prediction, until having made a choice of r. This important because if we replaced the formulation by fixed predicted class 9 Z'mm (or even I x) then be inf p(ū x.m) 2 8). it would not be truthful anymore to say that the prediction was made with the threshold
1
Σ (u(z)))ww
Mj Ray (K,))R)
Eys= E,- E₁
One difficulty is that this dynamic thresholding only works as long as the model adheres to á S r' for all possible values of 7'. Thus, if was fixed during training, this would not work. As such, P2P is trained be adaptable to any value by randomly sampling 24[0.5,, during training for each instance. Additionally, as input to the model such that it can adapt its masking to the desired level of sparsity. At inference time, we find the desired by increasing its value stepwise until the condition in Equation (4) fulfilled. A natural interpretation of this procedure that the classifier queries for more information until is certain enough make prediction.
By specifying the covariance this way, we ensure the pos- itive semi-defimiteness of the covariance; see Appendix A for the simple proof. To avoid floating point issues and overfitting, we add small regularizer the norm ||E|| An additional advantage of learning the mask over regions instead of pixels is that it becomes computationally and memory-effiientt to compute the covariance matrix for cap- turing these feature relationships. At inference time, the relationships captured in the learnable, region-wise embed- dings Ej can be interpreted, for example, by clustering. In this work, for ease of visualization, we the dimension- ality of the embeddings to and directly visualize them by interpreting them RGB-colored image.
provide the sampled
Loss Function To summarize, P2P learns masking over perceptually meaningful regions. Embedded within this procedure is the capability of capturing the relationship of these regions via a learnable embedding. At inference time, the proposed approach performs dynamic selection the sparsity level by querying more features until certain enough make a prediction. The final loss function that being optimized during training is as follows:
Dynamic Thresholding Fixing the masking threshold constant overlooks the fact that the information required to make prediction can vary across instances. Previous methods have largely bypassed this issue by relying the regularization strength hyperparameter, which leaves some leeway in the masking amount across images. In contrast, P2P addresses this problem explicitly by determining an appropriate masking threshold for each instance inference
log p(y| pm)) - A 1p log(1 - p) + A2I (5)

---

From Pixels to Perception
4. Experimental Setup
Datasets We evaluate performance and faithfulness on the natural image datasets CIFAR-10 (Krizhevsky al., 2009) with 10 classes, and ImageNet (Russakovsky et al., 2015) with 1000 classes. To additionally compute local- ization, we use Imagenet-9 (Xiao et al., 2021), a subset of ImageNet with 9 coarse- grriseedggdddd classes where object seg- mentations have been made available. Furthermore, we in- troduce COCO-10, a subset of MS COCO (Lin et al., 2014), with the classes (Bed, Car, Cat, Clock, Dog, Person, Sink, Train, TV, Vase). Classes C' were chosen by maximizing the class-unique number of images.
Metrics Evaluating inherent interpretability is famously difficult (Lipton, 2016; 2017), especially in ensuring the faithfulness of explanations to the underlying model compu- tations, implied by the term "inherently" (Rudin, 2019). That being said, we believe an inherently interpretable model is supposed to demonstrate the following statement:
I find good explanations that lead to good performance. (2)
(3)
(1)
To evaluate performance in the context of classification problems, we look at the test aceuracy of the classifier given the masked input. We use this metric because all test sets in our evaluation are Mlass-balanced. (2) Localization: Evaluating the goodness of explanations depends on the type of interpretability. In our context, we compare the selected mask m with the ground-truth segmen- tation of the target object m². While various metries have been proposed for evaluating segmentations (Zhang, 1996; Russakovsky et al., 2015; Choe et al., 2020), the objective in instance-wis wise feature selection, as outlined in Equations (1) and (2), obtain a minimal mask. Therefore, achieving perfect segmentation of the ground-truth object is neither necessary nor desirable. To this end, we evaluate a sim- ple overlap metric, defined as Immm'l [esl where score of 0 indicates overlap, and a score of 1 signifies that the mask successfully captured a minimal mask of the object of interest. To ensure comparability, we set the sparsity level r of all baselines to match that of P2P. For real-world datasets, treating object localization as ground-truth ignores the presence of other cooceurring, predictive features. Con- sequently, we complement our quantitative evaluation with the semi-synthetic BAM datasets in Appendix B that are designed to not have any such shortcuts, as well as a visual inspection of Zm to assess their meaningfulness.
(1) Performance:
C = arrg max mrg min
Σ
1 6
( = 0 E C {c)] LAVE
OEC GEC
Thus, we obtain 1000 training and 100 test images per class and use the object segmentations of MS COCO. Lastly, we use the semi-synthetic BAM from Yang & Kim (2019) whose results are shown in Appendix B to save space, and they support the same conclusions.
Baselines and Implementation Details We compare the performance of P2P to multiple state-of-the-art models and insightful baselines. Blackbox predicts directly on æ, thereby providing an upper bound for performance and in- dicating random performance for localization. Blackbox Pixel takes 3. as input, where m is an evenly-spaced mask that adheres to T, critiquing Equation (1) by showcasing that uninformed masking in pixel space can lead to highly predictive solutions. The existing works that we use as base- lines are DiET (Bhalla et al., 2024), REAL-X (Jethani et al., 2021), RB-AEM (Ganjdanesh et al., 2022). B-cos (Böhle et al., 2022), and COMET (Zhang et al., 2025), all described in Section 2. Notably, COMET learns a continuous-vai valued mask. We use the loss from their code repository, not the manuscript, as the latter significantly reduced performance. As faithfulness ablation, we also introduce COMET-1 which inverts m before passing it to the classifier.
(3) Faithfulness: Jacovi& & Goldberg (2020) state that "In- herent interpretability is a claim until proven otherwise." " To validate this claim, we compute the fidelity of explanations using insertion and deletion (Petsiuk, 2018). For deletion, the most important pixels in a.., determined by selection probabilities, are iteratively set to black. Insertion starts with a dark image, iteratively adding the most important pixels of Im. In both metrics, we compute the fidelity of the predictions with respect to the original prediction on am. i.e. using ý as target rather than y. This is because the faithfulness metric evaluates how well the explanation aligns with the model's prediction, not its accuracy. Hooker et al. (2019) suggest retraining classifiers to address distri- bution shifts from pixel masking, but as all methods here are trained on masked pixels, this is not necessary. To save space, we present deletion results and insertion for select datasets in Appendix B, which reinforce our conclusions and confirm that retraining the classifiers is unnecessary.
For all methods, we use a pre-trained LR-ASPP Mo- bileNetV3 (Howard et al., 2019) as the selector and a pre- trained ViT-Tiny (Touvron et al., 2021) as the classifier backbone. Images are preprocessed 224 resolution. All models are trained using Adam (Kingma & Ba, 2015) with 81 = 0.9, 83 = 0.999, and learning rate 1e-4. We train for 20 epochs on ImageNet and 100 epochs on all other datasets with a batch size of 64. To get superpixels, we use FastLLIC (Kim, 2021) with m = 20 and 100 segments, determined by visual inspection. As described in Section 3, any large A leads to p R3 T, so we set it to 10, and A = 0.01 against overfitting. During training. P2P anneals the sparsity loss and Guubeel-Softaa temperature to prevent degenerate cases. At inference, we set the certainty threshold 8 to 0.8 for ImageNet and 0.99 for all other datasets, and determine active regions by thresholding probabilities at 0.5 instead of sampling. We provide an ablation for in Appendix B.
5

---

From Pixels to Perception
Table Accuracy reported across ten seeds. The hest-perforning method for each dataset is bolded, and the runner-up is underlined.
highlights the least important pixels by inverting masks at sparsity level т, performs equally well. This suggests COMET's predictions faithfulness criterion. It is also noteworthy that Blackbox Pixel performs close to Blackbox despite masking signifi- cant portion the image. This shows that non-informative masks in pixel-space do not impair a predictor's capabilities. Thus, if random masking already approaches optimal perfor- mance, can presume that the pixel-level feature selection problem in Equation (1) fails to enforce the learning of meaningful pixels. Further, we observe that DiET struggles with real-world datasets, as its gradient-basee based mask conver- gence algorithm both computationally slow and unstable for high-dimensional data. Similarly, while REAL-X has desirable theoretical properties, it struggles with real-world data. This is likely due its reliance on frozen classifier pre-trained with random masking, which is unable adapt to the structured masks output by the selector.
Dataset (r)
Model
Accuracy (%)
mask-independent, failing the
Blackbox
95.79 + 0.28
Blackbox Pixel
94.56 + 0.33
COMET-1
95.07 + 0.49
DIET
55.97==13..3
CIFAR-IO (20%)
RB-AEM
78.04 +: 23.44
REAL-X
90.44
B-cos
93.80 + 0.18
COMET
94.35 + 0.49
P2P
94.45 = 0.29
Blackbox
89.36 + 0.64
Blackbox Pixel
86.79 + 0.81
COMET
87.98 + 0.49
DiET
27.38 2.41
COCO-10 (40%)
RB-AEM
72.96 3.03
REAL-X
83.98 + 1.05
B-cos
84.54 + 0.73 0.73
Localization In Table 2, show the localization of all methods. Clearly, P2P's region-based mask locates the object of interest better than the baselines. The runner-up is the recently proposed COMET (Zhang et al., 2025). This indicates that our parts-focused selection mask, as well as the modeling of their relationships, helps the model to learn masks that adhere the object of interest. In Appendix C, visualize the embeddings that capture the relationships. Notice that COMET poor localization does not affect accuracy, indicating it does not rely solely on these regions, That said, real-world datasets may contain co-occurring features missed by this metric. Thus, in Appendix B, we use the semi-synthetic BAM to confirm our conclusions. Also, we provide visualizations of om the end of this section.
COMET
88.43 + 0.71
P2P
89.53 + 0.89
Blackbox
71.22 + 0.15
Blackbox Pixel
70.66 + 0.14
COMET-1
69.70 0.06
DIET
3.08
ImageNet (50%)
RB-AEM
57.76 + 1.65
REAL-X
69.08 0.25
B-cos
-8.34=0..1
COMET
70.90 0.21
P2P
68.70 + 0.19
Blackbox
94.45 = 0.42
Blackbox Pixel
93.89 + 0.50
COMET
94.27 = 0.54
Table 2. Localization across ten seeds. The best best-performing method for each dataset bolded, and the runner-up underlined.
DiET
30.62 + 3.99
ImageNet-9 (30%)
RB-AEM
81.67 + 2.12
Dataset (7)
Model
Localization (%)
REAL-X
89.31 + 0.73
Random
25.47 0.01
B-cos
92.18 + 0.21
COMET-1
22.69 1.66
COMET
94.12 + 0.44
DIET
00.31
P2P
94.42 + 0.30
RB-AEM
24.58 + 3.76
COCO-10 (40%)
REAL-X
33.28 + 1.59
5. Results
B-cos
34.57 +: 0.44
COMET
36.43 + 1.70
In the following paragraphs, we showcase that P2P produces meaningful explanations that lead to good performance.
P2P
47.01 1.16
Random
38.52 + 0.01
COMET
24.23 + 3.59
Test Performance Table 1, we report the accuracy of the proposed approach, P2P, compared to the baselines. Re- markably, P2P achieves performance comparable to the upper-bounding Blackbox while, on average, removing to 80% of the image content. Only COMET matches this performance, but we argue that its predictions are not based on the highlighted regions. Observe that COMET-1, which
DIET
38.5441.44
RB-AEM
32.52 + 1.96
ImageNet-9 (30%)
REAL-X
56.13
B-cos
53.26 0.50
COMET
63.73 2.00
P2P
69.25 + 1.06
6

---

From Pixels to Perception
100
100
100
80
80
80
Ficalit (%)
40
40
40
0
0
20
40
80
80
100
80
80
100
20
40
60
80
100
Percentage Inserted (%)
Percentage Inserted (%)
Percentage Inserted (%)
(a) CIFAR-10
(b) COCO-10
(c) ImageNet-9
DIET
RB-AEM
REAL-X
COMET
Fixed P2P (Matched
P2P
Figure 3. Insertion Fidelity, where the most important pixels of the explanation are iteratively added to black image, measuring how much information is required until the original prediction is recovered. The faster, Le steeper the curve, the better. Results are reported as averages and standard deviations across ten seeds.
Ablation Study: Fixing A key contribution of P2P is its dynamic thresholding based on the classifier's certainty. However, P2P also supports fixed threshold across all samples in case application requires specific level of sparsity. In Figure 4, we present an ablation using a fixed r set either to the average sparsity determined by the dynamic thresholding or to 20% to encourage stronger masking.
From these previous results, can deduce that P2P provides both meaningful explanations and strong performance. The final step demonstrate that P2P is an inherently interpretable feature selection method is to show that its predictions are faithfully based on the explanations. For this, we present the insertion fidelity in Figure 3. Recall that fidelity measures the importance of highlighted pixels with respect to the original prediction, not with respect to the ground-truth, thereby measuring the faithfulness of explamations. We exclude B-cos and Blackbox, as they lack masked training, making this metric unable to distinguish fidelity from oul-ofddistribution behavior (Hooker et al., 2019). In Appendix B, we additionally present the insertion fidelity for the remaining datasets and deletion fidelity for all datasets, confirming our conclusions.
Faithfulness
The fixed variant clearly performs worse in accuracy, which is expected since it lacks the flexibility to adapt to the instance-wise required amount of information. This high- lights the advantage of dynamic thresholding in P2P, as it enables the model to selectively query more information when necessary. leading to more informed predictions. the same time, higher sparsity levels lead improved lo- calization, suggesting that the restrictively selected regions prioritizes the
the most informative. As such, if interpretability and correctness of the mask over accuracy, enforcing stricter sparsity constraint may be beneficial.
The curves show that P2P is superior in the fidelity of its explanations, evidenced by the significantly steeper inser- tion curves compared to the baselines. This indicates that the explanations output by P2P are the actual reasons for the pre- diction that the classifier makes. In contrast, this highlights the weakness of COMET's continuous valued masking that was already discussed in the context of COMET For in- stance for COCO-10 10 Insertion Fidelity with 40%, when the top 40% most important pixels are retained, COMET recovers the original prediction only 60% of the time. This calls into question the faithfulness of its masks, as the high- lighted pixels alone do not faithfully capture the model's decision- makinn darker regions still significantly influence the outcome. also explains COMET and COMET-17 strong predictive performance, it does not limit its input to only 100 7% of the content but continues utilizing the full image, merely adjusting for brightness. On the other hand, we find that P2P benefits from parts-based masking, as its predictions depend strongly on the highlighted regions. This indicates that P2P has learned to extract and utilize the important regions of each image.
100
Accuracy (%)
80
60
40
90
100
Localization (%)
Dynamic
Fixed (Matched
Fixed (T=20%)
BAM Object
BAM Scene
COCO-10
ImageNet-9
Figure 4. Ablation study of P2P with constant In "Matched", we set to the average value of the dynamic P2P, which is 20%, 20%, 40%, 30%, for each dataset, respectively.
7

---

From Pixels to Perception
Cat
Vase
Clock
Person
Original
Blackbox Pixel
DiET
REAL-X
COMET
Fixed P2P
P2P
Figure 5. Masked inputs a.m. of COCO-10 10 for selected methods.
We also see that the P2P variant with fixed r worse than the dynamic P2P for low insertion percentages p but better when > This is expected, as dynamic thresholding results in varying levels of masking across images. Some images receive strong masking, where even 10% of pix- els may be sufficient to reconstruct the prediction, leading to high fidelity. Conversely, images with weaker masking require larger fraction of pixels faithfully explain the prediction. Additionally, not achieve the highest predictive accuracy, its selection probabilities effectively capture the rationale behind its pre- dictions. In contrast, RB-AEM, which predicts the mask's 's mean and expansion parameters, seems heavily reliant on the randomness inherent in the distributions' sampling pro- cess. That is. for sufficiently high variance, this approach behaves similarly Blackbox Pixel, relying more ran- dom sampling than meaningful feature selection.
object. Conversely, Blackbox Pixel, despite masking 60% of pixels, removes little actual information, suggesting that Equation (1) is misspecified. In contrast, P2P's region-based formulation enforces perceptual sparsity while maintaining strong performance. Consistent with the quantitative results, COMET highlights important regions, but its non-binary mask fails remove other information. Darkened pixels may appear less informative humans, but they retain their numerical differences that the model leverages, therefore us- ing information from both highlighted and dimmed regions.
find that while DiET does
6. Conclusion
We introduced P2P, a new instance-wise feature selection method for inherently interpretable classification. P2P learns masks that adhere human perception by enforcing sparsity in the space of semantically meaningful regions. Additionally, we proposed dynamic thresholding mecha- nism that adjusts the sparsity for each image based on the prediction difficulty. Empirically, we showed that P2P satis- fies the key properties of inherent interpretability: selects meaningful features that faithfully lead a strong predic- tive performance. Our qualitative results further showed that by masking perception-adherinn regions instead of indi- vidual pixels, P2P effectively captures the relevant informa- tion in the image, making it significantly easier to interpret the model's 's decisions and gain insights into the task, data, and model reliability. Our findings highlight the versatility of P2P, which we believe to hold significant potential for instance-wise feature selection, paving the way for exciting advancements in the field of inherent interpretability.
Visualizations To round off the analysis, provide vi- sualizations of xm in Figure 5, and set of randomly se- lected xm for each dataset in Appendix C. Quantitatively measuring the perceptual benefit masking semantically meaningful regions instead individual pixels is challeng- ing, it enhances the alignment of the masks with human visual perception. While difficult to express numerically, the visualizations clearly illustrate that P2P's masks result in highly interpretable selections. Crucially, all methods learn their masks without any access to cround-truth seg- mentations. Yet, P2P clearly identifies and focuses on the object of interest. For example, for the clock, the model con- fidently makes prediction without needing to see the entire
8

---

From Pixels to Perception
Acknowledgements
Carballo-Castro, A., Laguna, S., Vandenhirtz, M., and Vogt, J. E. Exploiting interpretable capabilities with concept- enhanced diffusion and prototype networks. arXiv preprint arXiv:2410. 18705, 2024. 2
thank Thomas M. Sutter and Alain Ryser for insightful chats. MV supported by the Swiss State Secretariat for Education, Research, and Innovation (SERI) under contract number MB22.00047.
Chen, C.,, Li, O., Tao, D., Barnett, A., Rudin, C., and Su, K. This looks like that: deep learning for interpretable image recognition. Advances in neural information processing systems, 32, 2019. 2
Impact Statement
This paper presents work whose goal is to advance the field of Machine Learning. There are many potential societal consequences of our work, none which feel must be specifically highlighted here.
Chen, J., Song, L., Wainwright, M., and Jordan, M. Learn- ing explain: An information-theorrti perspective on model interpretation. In International conference ma- chine learning. pp. 883-892 PMLR, 2018. 1, 2,2,
References
Choe, J., Oh, S. Lee, S., Chun, S., Akata, Z., and Shim, Evaluating weakly supervised object localization methods right. In Proceedings of the IEEE/CVF conference computer vision and pattern recognition, pp. 3133-3142, 2020. 5
Achanta, R., Shaji, A., Smith, K., Lucchi, A., Fua, P., and Süsstrunk, S. Slic superpixels compared state-of-the- art superpixel methods. IEEE transactions on pattern analysis and machine intelligence, 34(11):227--2282, 2012. 3
Covert, I. C., Qiu, W., Lu, M., Kim, N. Y., White, N. J., and Lee, S.-I. Learning to maximize mutual information for dynamic feature selection. In International Conference on Machine Learning, pp. 6424 6447. PMLR, 2023. 2
Adebayo, J., Gilmer, J., Muelly, M., Goodfellow, I., Hardt, M., and Kim, B. Sanity checks for saliency maps. Ad- vances in neural information processing systems, 31, 2018. 2
Doshi-Veles and Kim, B. Towards rigorous sci- ence of interpretable machine learning. arXiv preprint arXiv: 2017.
Arya, S., Rao, S., Boehle, M., and Schiele, B. B-cosification: Transforming deep neural networks to be inherently in- terpretable. In 38th Conference on Neural Information Processing Systems, 2024.
Boshi-Velez. and Kim, B. Towards A Rigorous Science of Interpretable Machine Learning, March 2017.
Atchison, and Shen, S. M. Logistie-normal distributions: Some properties and uses. Biometrika, 67(2):611-722, 1980.
Espinosa Zarlenga, M., Barbiero, P., Ciravegna, G., Marra, G., Giannini, F., Diligenti, M., Shams, Z., Precioso, F., Melacci, S., Weller, A., et al. Concept embedding mod- els: Beyond the accuracy eecuracy-explainablli trade-off. In Advances in Neural Information Processing Systems, vol- ume 35, pp. 21400-21413, 2022. 2
Bacon, F. Meditationes sacrae. Excusum impensis Hum- fredi Hooper., 1597. httpt /btcks.oos//////// google. ch/books (at.akrlU-HeyUAAAAAAAAAA
1
Bhalla, U.. Srinivas, S., and Lakkaraju, H. Discriminative feature attributions: bridging post hoc explainability and inherent interpretability. Advances in Neural Information Processing Systems, 36, 2024. 2,
Esser, P., Kulal, S., Blattmann, A., Entezari, R., Müller, J., Saini, H., Levi, Y., Lorenz, D., Sauer, A., Boesel, F,, et al. Scaling rectified flow transformers for high- high-resolution image synthesis. In Forty-first International Conference Machine Learning, 2024.
Biederman, I. pytenittoon-by-coponeeee theory of human image understanding. Psychological review, 94
Fanconi, C., Vandenhirtz, M., Husmann, S., and Vogt, This reads like that: Deep learning for interpretable natu- ral language processing. In The 2023 Conference on Em- pirical Methods in Natural Language Processing, 2023,
(2):115, 1987. 1
Böhle, M., Fritz, M., and Schiele, B. B-cos networks: Align- ment is all we need for interpretability. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 10329-103333 2022. 2,5
2
Fryer, D., Strümke, I., and Nguyen, Shapley values for feature selection: The good, the bad, and the axioms. leee Access, 9: 9:144352-144366 2021.
Böhle, M., Singh, N., Fritz, M., and Schiele, B. B-cos alignment for inherently interpretable cnns and vision transformers. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2024.7
Ganjdanesh, A., Gao, S., and Huang, H. Interpretations steered network pruning via amortized inferred saliency

---

From Pixcls to Perception
maps. In European Conference on Computer Vision, pp. 278-296 296. Springer, 2022. 2,5
Koh, P. W., Nguyen, T., Tang, Y. S., Mussmann, S., Pierson, E., Kim, B., and Liang, P. Concept bot- tleneck models. In III, H. D. and Singh, A. (eds.), Proceedings of the 37th International Conference on Machine Learning, volume 119 of Proceedings of Machine Learning Research, pp. 5338-5348, Virtual, 2020, PMLR. URL https: /tttpp:://rroceee /pppceeeecccgggssee mllr. press/ http//111//kkoh html. 2
Geirhos, R., Jacobsen, J.-H., Michaelis, C., Zemel, R., Bren- del, W., Bethge, M., and Wichmann, A. Shortcut learn- ing in deep neural networks. Nature Machine Intelligence, 2(1 ((11):665-67) 673, 2020. 2
Hooker, S., Erhan, D., Kindermans, P.-.., and Kim, B. A benchmark for interpretability methods in deep neural networks. Advances in neural information processing systems, 32, 2019. 2, 5, 7, 14
Krizhevsky, A., Hinton, G., et al. Learning multiple layers of features from tiny images. 2009.
Laguna, S., Heidenreich, J. N.. Sun, J., Cetin, N., Al- Hazwani, I., Schlegel, U.. Cheng, F., and El-Assady, M. Explimeable: A visual analytics approach for exploring lime. In 2023 Workshop on Visual Analytics in Healtheare (VAHC), pp. 27-33, IEEE, 2023. 2
Howard, A., Sandler, M.. Chu, G., Chen, L.-C., Chen, B., Tan, M., Wang, W., Zhu, Y., Pang, R., Vasudevan, V., et al. Searching for mobilenetv3. In Proceedings of the IEEE/CVF international conference on computer vision, pp. 1314-1324, 2019. 5
LCM team, Barrault, L., Duquenne, P.-A., Elbayad, M., Kozhevnikov, A., Alastruey, B., Andrews, P., Coria, M., Couairon, G., Costa-jusså, M. R., Dale, D., Elsahar, H., Heffernan, K., ao Maria Janeiro, J., Tran, T., Ropers, C., Sánchez, E.,, Roman, R. S., Mourachko, A., Saleem, S., and Schwenk, H. Large Concept Models: Language modeling in a sentence representation space, 2024, URL https: / rx owt.vv.rrrg/b...o. 2.12. 08821. 2
Huang, Q. Song, J., Hu, J., Zhang, H., Wang, Y., and Song, M. On the concept trustworthiness in concept bottleneck models. In Proceedings of the AAAI Conference on Ar- rificial Intelligence, volume 38, pp.
2024.
Jacovi, A. and Goldberg, Y. Towards faithfully interpretable nlp systems: How should we define and evaluate faithful- ness? arXiv preprint arXiv:2004.0366
2020.
Lin, T.-.,, Maire, M., Belongie, S., Hays, J., Perona, P., Ra- manan, D., Dollár, P., and Zitnick C. L. Microsoft coco: Common objects in context. In Computer Vision-ECCV 2014: 13th European Conference, Zurich, Switzerland, September 6-12, 2014, Proceedings, Part V 13, pp. 740- 755. Springer, 2014. 14
Jang, E., Gu, S., and Poole, B. Categorical reparame- terization with gumbel-softmax. In 5th International Conference on Learning Representations, ICLR 2017, Toulon, France, April 24-26, 2017, Conference Truck Proceedings. OpenReview.nnt, 2017. URL https: //openreview.co/////// net ftr.m/oruummm///// (or.ormmmmmmddeee335eee..... net.:/iwwwnneeeeeeeeeeeeeeeeeeee
Lipton, Z. C. The Mythos of Model Interpretability. Com- munications of the ACM, (10))):3 43, June 2016. doi: 10.448550/axiiv1166..5333
Jethani, N., Sudarshan, M., Aphinyanaahongs, Y., and Ran- ganath, R. Have we learned explain?: How inter- pretability methods can learn to encode predictions in their interpretations. In International Conference on Arti- ficial Intelligence and Statistics, pp. 1459-1467. PMLR, 2021. 2, 5
2.
Lipton, Z. C. The doctor just won't accept that! arXiv preprint 171 1.0007,, 2017.
Löwe, S., Lippe, P., Locatello. F., and Welling, M. Rotat- ing features for object discovery. Advances in Neural
Kim, A. Fast slie: Efficient superpixel segmentation, 2021. URL https: /tap:://gi.. /gi thub. cwm/Algy///// hat///fftsssss/ssl Accessed: 2025-01-25. 5
Information Processing Systems, 36, 2024. 18
Lundberg, S. A unified approach to interpreting model predictions. arXiv preprint arXiv:1705.07877 2017. 1,2
Kim, B., Rudin, C., and Shah, J. A. The bayesian case model: A generative approach for case-based reasoning and prototype classification. Advances in neural informa-
Ma, C., Donnelly, J., Liu, W., Vosoughi, S., Rudin, C., and Chen, C. Interpretable image classification with adaptive prototype-based vision transformers. In The Thirtt-eighth Annual Conference on Neural Information Processing Systems, 2024. 2
tion processing systems, 27, 2014. 2
Kingma, D. P. and Ba, J. Adam: A method for stochas- tic optimization. In Bengio, Y. and LeCun, Y. (eds.), 3rd International Conference on Learning Represen- tations, ICLR 2015, San Diego CA, USA, May 7-9, 2015, Conference Track Proceedings, 2015. URL http: abs /1/..111...00 6980.
Maddison, C. J., Mnih, A., and Tch, Y. W. The concrete distribution: A continuous relaxation of discrete random variables. In 5th International Conference on Learning Representations, ICLR 2017 Toulon, France, April 24-26,
a/a //arxiv. wwwg/bsg/////oooooeoo.ooo..eease
10

---

From Pixels to Perception
2017, Conference Track Proceedings. OpenReview.net, 2017. URL https::/p//........ http:/ppwpeeeeeeeee.teeee id-81j8515g1.3
Rudin, C. Stop explaining black box machine learning models for high stakes decisions and use interpretable models instead. Nature machine intelligence, 1(5):206- 215, 2019. 1,
Mareinkevičs, R. and Vogt, J. E. Interpretable and ex- plainable machine learning: a methods- overview with concrete examples. Wiley Interdisciplimary Reviews: Data Mining and Knowledge Discovery, 13(3):e1493, 2023. 1
Russakovsky, O., Deng, J., Su, H., Krause, J., Satheesh, S., Ma, S., Huang, Z., Karpathy, A., Khosla, A., Bern- stein, M., et al. Imagenet large scale visual recognition challenge. International journal of computer vision, 1 115: 211-252, 2015. 5
Marcinkevičs, R., Laguna, S., Vandenhirtz, M., and Vogt, E. Beyond concept bottleneck models: How to make black boxes intervenable? In Advances in neural infor- mation processing systems, volume 2024. 2
Saeys, Y., Inza, I., and Larranaga, P. A review of feature selection techniques in bioinformatics. bioinformatics, 23(19):507-22517 2007.
Miller, G. A. The magical number seven, plus or minus two:
Selvaraju, R. R., Cogswell, M.. Das, A., Vedantam, R., Parikh, D., and Batra, D. Grad-cam: Visual explana- tions from deep networks via gradient-based localization. In Proceedings of the IEEE international conference on computer vision, pp. 618-626, 2017. 2
Some limits on our capacity for processing information. Psychological review, 63(2):81, 1956.
Monteiro, M., Le Folgoc, L., Coelho de Castro, D., Pawlowski, N., Marques, B., Kamnitsas, K., van der Wilk, M., and Glocker, B. Stochastic segmentation networks: Modelling spatially correlated aleatoric uncertainty. In Advances in neural information processing systems, vol- ume 33, pp. 12756-12767, 2020. 3
Shrikumar, A., Greenside, P., and Kundaje, A. Learning important features through propagating activation differ- ences. In International conference on machine learning, pp. 3145-3153, PMIR, 2017.
Neubert, P. and Protzel, Compact watershed and preemp- tive slic: On improving trade-offs of superpixel segmen- tation algorithms. In 2014 22nd international conference on pattern recognition, pp. 996-1001. IEEE, 2014. 17
Springenberg, J. T., Dosovitskiy, A., Brox, T., and Ried- miller, M. Striving for simplicity: The all convolutional net. arXiv preprint arXiv:1412.6800 2014.
Oosterhuis, H.,
Lyu, L., and Anand, Local feature selection without label feature leakage for inter- pretable machine learning predictions. arXiv preprint arXiv:2407. 11778, 2024.7 2
Sundararajan, M.. Taly, A., and Yan, Q. Axiomatic attribu- tion for deep networks. In International conference on machine learning, pp. 3319-3322 3328. PMLR, 2017. 2
Tadist, K., Najah, S., Nikolov, N. S., Mrabti, F,, and Zahi, A. Feature selection methods and genomic big data: systematic review. Journal of Big Data, 6(1):1-24, 2019,
Palmer, S. E. Hierarchical structure in perceptual repre- sentation. Cognitive psychology, 9(4):441477, 1977.
1
2
Petsiuk, V. Rise: Randomized input sampling for explanation of black-box models. arXiv preprint arXiv:1806.07421, 2018. 5
Taghanaki, S. A., Havaci. M., Berthier, T., Dutil, F,, Di Jorio, L., Hamarneh, G., and Bengio, Y. Infomask: Masked variational latent representation to localize chest disease. Medical Image Computing and Computer Assisted Antervention-MICA. 2019: 22nd International Confer- ence, Shenghen, China, October 13-17, 2019, Proceed- ings, Part VI 22, pp. 739-747, Springer, 2019. 17
Remeseiro, B. and Bolon-Caneda V. A review of feature selection methods medical applications. Computers in biology and medicine, 1 112:103375, 2019. 2
Ribeiro, M. T., Singh, S., and Guestrin, C. wwh should i trust you?" explaining the predictions of any classifier. In Proceedings of the 22nd ACM SIGKDD international conference on knowledge discovery and data mining, pp- 1135-1144 2016. 1,2
Tibshirani, R. Regression shrinkage and selection via the lasso. Journal of the Royal Statistical Society Series B: Statistical Methodology, 58(1):267-22) 1996. 1,2
Touvron, H., Cord, M., Douze, M., Massa, F., Sablayrolles, A., and Jégou, H. Training data-efficient image transform- ers & distillation through attention. In International con- ference on machine learning, pp. 10347-10357. PMLR, 2021.
Ross, W. Metaphysics. Publishing Platform, 2016. ISBN 9781534926288. ?id=
CreateSpace Independent
URL https:/ /books...../oo.ooo.............. google..oo..ooooooooooooooo.. chwb.okcbbboeeoe.oo aVQbVWEACAAJ.

---

From Pixels to Perception
Vandenhirtz, M., Laguna, S., Marcinkevits, R., and Vogt, H. Stochastic concept bottleneck models. In The Thirty- eighth Annual Conference Neural Information Pro- cessing Systems, 2024. 3,3
Xiao, K. Y., Engstrom, L., Ilyas, A., and Madry, A. Noise signal: The role of image backgrounds in object recog- nition. In International Conference Learning Repre- sentations, 2021. 5
Yang. M. and Kim, B. Benchmarking attribution meth- ods with relative feature importance. arXiv preprint 1rXfv::907.00701. 2019. 5, 14
Yoon, J., Jordon, J., and Van der Schaar, M. Invase: Instance- wise variable selection using neural networks. In Inter- national conference learning representations, 2018.
Yuan, M. and Lin, Model selection and estimation regression with grouped variables. Journal of the Royal Statistical Society Series B: Statistical Methodology, 68 (1):49-67, 67, 2006. 1,2
Zhang, X., Lee, D., and Wang. S. Comprehensive attribu- tion: Inherently explainable vision model with feature detector. European Conference on Computer Vision, pp. 196-213. Springer, 2025. 1, 6,
Zhang. Y.-J. A survey on evaluation methods for image seg- mentation. Pattern recognition, 29(8):335--1366 1996.
Zhou, B., Lapedriza, A., Khosla, A., Oliva, A., and Tor- ralba, A. Places: 10 million image database for scene recognition. IEEE transactions on pattern analysis and machine intelligence, 405-)4445,, 2017. 14
12

---

From Pixels to Perception
A. Positive Semi-Definiteness of Covariance Matrix
Here, we present short proof that shows the positive semi-definiteness of covariance matrix whose entries are defined as dot products of embeddings. The covariance matrix Σ E PDD defined such that each entry computed as dot product of learnable part-specific embeddings:
Σz = Ej Es,
Step 1: Symmetry of Σ
The dot product operation ensures that Zijk Ekj for all j, k, since:
Eyk= Ej Ex Ex E; Ekj-
Thus, E is symmetric.
Step 2: Positive Semi-Definitee Deiinieeinnss quadratic form of E is given by:
To show that E is positive semi-definite, consider an arbitrary vector z E RD. The
2. Σx
3 3jth
j=1 k=1
Substituting E) Ej- Eg, this becomes:
Σz
2,2h EEj- Ex).
j=1
Rewriting using vector notation:
D Σ EE
Σz
Σ
Ex
/=1
Let u = Ep= Ej Then:
Σz u |u|| 0.
Since the quadratic form is non-negative for all RR,, Σ is positive semi-definite
13

---

From Pixels to Perception
Table Test Accuracy and I Localization reported averages and standard deviations across seeds. The best-performing methad for
each dataset is bolded, and the runner-up is underlined.
Dataset (r)
Model
Accuracy (%)
Localization (%)
Blackbox
89.55 + 0.48
22.94 = 0.00
Blackbox Pixel
87.56 + 0.54
22.94 = 0.00
COMET
89.58 + 0.34
28.32 = 10.98
DiET
31.27 + 14.46
37.78 + 11.36
BAM Object (20%)
RB-AEM
75.93 + 1.87
18.68 = 1.71
REAL-X
82.62 0.84
71.19 = 1.52
B-cos
86.47 0.18
48.23 = 0.72
COMET
89.33 + 0.47
82.88 = 0.32
P2P
88.92 + 0.50
83.02 + 0.75
Blackbox
93.12 0.27
77.06 + 0.00
Blackbox Pixel
92.18 0.42
77.06 0.00
COMET-1
92.07 + 0.47
77.85 + 10.56
DIET
55.79
76.63 16.98
BAM Scene (20%)
RB-AEM
80.03 + 1.37
82.02 = 0.93
REAL-X
85.49 + 1.02
95.15 = 1.32
B-cos
91.67 = 0.39
92.99 = 0.19
COMET
90.94 + 0.74
98.49 = 0.63
P2P
91.93 0.45
98.18 0.27
B. Additional Results
BAM Datasets Beyond the datasets in Section 5, we also evaluate all methods on the semi-synthetic BAM Scene and BAM Object (Yang & Kim, 2019). These two datasets are creating by cropping an object from MS COCO (Lin et al., 2014) and inserting it in scenes from Places (Zhou al., 2017). The datasets differ by the choice of whether the object the scene is the target. Notably, with this setup, there are other predictive features, apart from the object scene of interest. As such, in contrast to the results in Section 5, the localization metric perfectly captures the ground-truth pixels m that can be useful for the prediction. In Table 3, we provide the accuracy and localization of all methods on these datasets. Clearly, P2P also excels in this controlled setup, showcasing that it reliably determines the object or scene of interest.
Insertion Fidelity In addition to the visualizations in Section 5, we present the insertion fidelity of the remaining datasets in Figure 6. These results support the conclusions from Figure 3 by again showcasing that P2P's performance highly depends on the pixels that are deemed important by the selector. In contrast, other methods, such as COMET, have a much less steep curve, indicating that they rely on the full image to make their prediction.
Deletion Fidelity In this paragraph, show the deletion fidelity of all methods. With similar idea insertion fidelity, this metric starts from F... and iteratively masks the most important pixels. The stronger the decline, the better, it indicates that the removal of the most important pixels makes the model change its prediction, thereby showing that its prediction is based on these pixels. Similar to the insertion fidelity, the metric measures how often the model changes its predictions, rather than computing the accuracy with respect to the ground truth. In Figure 7, we present the deletion fidelity. Similar to the results in Figure 3, we see that P2P has a very steep curve, indicating that the removal of its most important pixels do have an strong effect on its prediction. This supports the conclusions that P2P is faithful, inherently interpretable classifier.
These results also justify the direct of the insertion and deletion fidelity as faithfulness evaluation. That is, ROAR (Hooker et al., 2019) cautions against directly computing insertion and deletion fidelity due to potential distribution shifts when removing pixels. However, since all models in our study are trained with masking, they remain within distribution even when pixels are removed. P2P's strong performance in both insertion and deletion fidelity further confirms that its fidelity scores are due to faithfulness rather than -ut-of-distribution effects. duitt---dit issues were responsible for a sharp drop, we would expect strong deletion fidelity but poor insertion fidelity. Likewise, if poor-perfore proorperforiig baselines in insertion fidelity suffered from distribution shifts, they should perform well deletion fidelity - yet this is not the case for RB-AEM
14

---

From Pixels to Perception
and COMET. These results allow the conclusion that the observed fidelity results genuinely reflect the faithfulness of the methods rather than being skewed by distribution shifts.
100
100
100
80
Fideltyy (%)
60
80
60
20
20
20
20
40
80
100
20
40
60
100
Percentage Inserted (96)
Percentage Inserted (%)
Percentage Inserted (1)
(a) ImageNet
(b) BAM Object
(c) BAM Scene
DIET
RB-AEM
REAL-X
COMET
Fixed P2P (Matched T)
P2P
Figure Insertion Fidelity, where the most important pixels the explanation 7.m are iteratively added to a black image, measuring how much information is required until the original prediction is recovered. The faster, i.e. the steeper the curve, the better. Results reported as averages and standard deviations across ten seeds.
(a) CIFAR-10
(b) COCO-10
(c) ImageNet-9
100
100
100
80
80
80
Fidelity (%)
40
40
40
0
0
20
40
60
80
100
0
20
40
60
80
100
20
40
60
80
100
Percentage Deleted (%)
Percentage Deleted (%)
Percentage Deleted (%)
100
100
100
80
80
80
Fideliy (%)
60
60
40
40
40
0
01
20
40
60
80
100
20
40
60
80
100
20
40
60
80
100
Percentage Deleted (%)
Percentage Deleted (%)
Percentage Deleled (%)
(d) ImageNet
(e) BAM Object
(f) BAM Scene
DIET
RB-AEM
REAL-X
COMET
Fixed P2P (Matched T)
P2P
Figure 7. Deletion Fidelity, where the most important pixels of the explanation are iteratively removed, measuring how much information needs to he removed until the prediction changes. The steeper the curve, better. Results are reported as averages and standard deviations across ten seeds.
Ablation Study: Certainty Threshold 5 In Table present ablation of different certainty thresholds for P2P. We see that the dynamic thresholding adjusts based the chosen certainty of the prediction. The less certainty is required, the sparser the mask will be. As such, it makes sense that the accuracy decreases with a lower d. Simultaneously. localization improves, as the most important pixels are more likely to be part of the object of interest. Notably, even for extremely strong masking, P2P's predictive performance does not drop to an extremely low level, but it seems even with very little information, the model can perform meaningful predictions.
15

---

From Pixels to Perception
Table 4. Ablation study of P2P for varying certainty thresholds 6. We measure accuracy and localization, well the average number of masked pixels p.
Dataset
5
Accuracy (%)
Localization (%)
p (%)
0.80
84.82
92.23
10.16
0.90
86.70
90.30
12.31
BAM Object
0.95
87.83
88.13
14.69
0.99
88.92
83.02
20.19
0.80
85.18
99,45
12.19
0.90
88.23
99.25
14.77
BAM Scene
0.95
90.08
98.97
17.42
0.99
91.93
98.18
23.46
0.80
85.66
13.20
0.90
89.69
15.56
CIFAR-10
0.95
92.05
17.97
0.99
94.45
23.51
0.80
82.91
54.99
21.44
0.90
86.61
52.91
26.51
COCO-10
0.95
88.15
51.19
31.26
0.99
89.53
47.01
41.71
0.80
68.70
48.31
0.90
69.82
56.85
ImageNet
0.95
70.12
63.04
0.99
70.23
72.99
0.80
87.13
77.97
15.22
0.90
91.02
75.77
18.91
ImageNet-9
0.95
92.79
73.73
22.42
0.99
94.42
69.25
30.19
16

---

From Pixels to Perception
Ablation Study: Superpixel Algorithm Superpixels central part of our method. Thus, we provide ablation study for the choice of algorithm. In preliminary experiments, observed that the choice of SLIC's hyperparameters do not have strong effect performance, as long the number of segments chosen is reasonable (i.e. >20). In Table 5, we explore the effect of the specific superpixel algorithm on P2P's performance by replacing the SLIC superpixel algorithm with the Watershed superpixel algorithm (Neubert & Protzel, 2014), choosing the hyperparameters by ensuring similar number of segments. We see that there are no strong dependencies on the specific choice of superpixel algorithm, further strengthening the generality of P2P.
Table 5. Accuracy and localization performance of P2P using different region proposal algorithms.
Dataset
Method
Accuracy (%)
Localization (%)
ImageNet-9
P2P (Watershed)
93.95
67.70
P2P (SLIC)
94.42
69.25
COCO-10
P2P (Watershed)
90.05
46.95
P2P (SLIC)
89.53
47.01
Ablation Study: InfoMask Here, we additionally compare P2P to InfoMask (Taghanaki et al., 2019) on a subset of datasets. This method is a information bottleneck inspiired approach that learns a masking on pixel level, regularized by a kl-divergence. Some notable differences are that InfoMask operates directly on pixel level and uses 'semi-hard" masking, where each masking probability is either or in (0,1). We present the results applying InfoMask the same setup as P2P in Table 6. The mask of InfoMask is learned pixel-basis, which encourages That is, in these complex datasets, InfoMask selects pixels nearly randomly, relying the fact that reduction in resolution does not effectively reduce information from the image. On the other hand, P2P selects only sparse subset of features that are relevant for the prediction.
behave similarly to Blackbox Pixel.
Table 6. Comparison of P2P and InfoMask in terms of accuracy and localization performance.
Dataset
Method
Accuracy (%)
Localization (%)
ImageNet-9
InfoMask
94.69
38.55
P2P
94.42
69.25
COCO-10
InfoMask
89.20
26.02
P2P
89.53
47.01
17

---

From Pixels to Perception
C. Further Qualitative Examples
Covariance Learning The learned, part-relationship capturing covariance not only useful for the learning process, can also provide interpretability into the model's internal understanding of the image. In Figure show examples of the 3-dimensional embedding used compute the pairwise covariances. These embeddings characterize each part, and due the design choice of 3-dimensional embedding. can visualize them colors. Note that one could use more dimensions and visualize part similarities by preprocessing them with a clustering algorithm similar to Löwe et al. (2024). Note that the colors themselves do not contain any meaning, as the embedding is not constraint in such way, however, one can interpret the similarity in color between two parts, as their relationship. In the provided examples, we see that P2P learns that the background buildings are two separate entities, while the street is also different. In the right picture, we show that P2P does not only rely on color cues for its similarities, as can correctly separate the foreground leaf from the rest of the leaves in the background.
Figure 8. Visualization of the part-wise embeddings, used compute the covariance, colors. Left the partitioned input, middle is the masked input, and on the right, we show the part embeddings.
Randomly Sampled Masks We believe that one of the best ways to assess and understand the goodness of an inter- pretability method is by looking its visualizations. In order give the reader an unbiased picture, we visualize randomly selected n. the test set of each dataset. We urge the reader while analyzing these figures, remember that, in contrast almost all figures that are presented in the main parts of any publication, these images are not cherry-picked but randomly selected. In order to emphasize the masking aspect of P2P, we show the pictures for the low certainty threshold & = 0.8.
In Figures 9 to 14, we present the randomly sampled images. It evident that P2P consistently captures the objects of interest. We argue that images where the model applies weak masking often correspond to cases where the object of interest difficult to identify. This supports the use of dynamic thresholding, as it makes sense to visualize big parts of the image in these challenging examples. Note that for BAM Scene, the background is what determines the label. Also, note that for ImageNet, the masking amount is significantly less, the certainty threshold of 0.8 is harder obtain for this dataset with 1000 potential classes. From interpretability standpoint, these visualizations help identify potential shortcuts the model may have learned. This is significant ad advantage of our inherently interpretable method. Unlike standard black-box models, the provided visualizations enable us to detect these correlations, assess whether they are spurious or meaningful, and refine the model iteratively to enhance its robustness. Some examples of possible correlations that could be investigated would be the following: In COCO-10, teddy bears and pillows might be an indicator for the class bed. Flowers might be used to classify vases. Street lights and signs might indicate the label car. For ImageNet-9, froctal-facing human might imply the class fish. We argue that for pixel- bxsel instance- wise feature selectors it would be much harder arrive at such hypotheses, the pixelated nature of the masks make it more unclear, what exactly the classifier uses for its prediction.
18

---

From Pixels to Perception
Backpack
Bird
Dog
Elephant
COLA
Kite
Pizza
Stop Sign
STOP
H
STOP
TOP
Toilet
Truck
Zebra
Figure Visualization of randomly sampled images overlayed with their partitions, well the masked input am for BAM Object.
19

---

From Pixels to Perception
Bamboo For-
est
Bedroom
Bowling
Alley
Bus Interior
Cockpit
Corn Field
STOP
Laundromat
Running Track
STOP
Runway
Ski Slope
Figure Visualization of randomll sampled images overlayed with their partitions, as masked input æ.. BAM Scene.
20

---

From Pixels to Perception
Airplane
Automobile
Bird
Cat
Deer
Dog
Frog
Horse
Ship
Truck
Figure 11. Visualization of randomly sampled images overlayed with their partitions, well the masked input æ. for CIFAR-I0.
21

---

From Pixels to Perception
Bed
Car
Cat
Clock
Dog
Person
Sink
Train
TV
Vase
Figure 12. Visualization of randomly sampled images overlayed with their partitions, well the masked input æ. for COCO-10.
22

---

From Pixels to Perception
Bird
Carnivore
Dog
Fish
Insect
Musical Instrument
Primate
Reptile
Wheeled Ve-
hicle
Figure 13. Visualization of randomly sompled images overlayed with their partitions, well as the masked input o.. for ImageNet-9,
23

---

From Pixels to Perception
African Grey Parrot
Brittany Spaniel
Eskimo Dog
Macaw
Rock Python
Agama
Caldron
Fire Salamander
Malamute
Samoyed
Alligator Lizard
Bay Retriever
Frilled Lizard
Malinois
Scottish Decrhound
American Alligator
Clumber
Frilled Lizard
Mousetrap
Sorrel
Balance Beam
Common Newt
Garter Snake
Mud Turtle
Spoonbill
Balloon
Cougar
Gazelle
Night Snake
Three- Toed Sloth
Bighorn
Dowitcher
Giant Panda
Orangutan
Tick
Boa Constrictor
Egyptian Cat
Groenendael
Pomeranian
Water Snake
Boa Constrictor
English Springer
Labrador Retriever
Ram
Weimaraner
Figure Visualization randomly sompled images overlayed with their partitions, as well as masked input a. for ImageNet.
24


Reviews:
[{"review_id": "kkDnAVuEKP", "reviewer": "Reviewer_KrMz", "summary": "The goal of this method is to improve the interpretability of machine learning models. This work proposes a new approach to inherent interpretability by sparsifying the input images for model predictions. To achieve this, the method masks semantically defined pixel regions instead of individual pixels and employs dynamic thresholding to determine the necessary level of sparsity during inference. It is evaluated across multiple datasets, successfully producing human-understandable predictions while retaining the predictive performance of blackbox models.", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We thank the reviewer for their review and for their positive feedback. As there are no open questions, we keep our rebuttal brief.\n\nWe thank the reviewer for their comment on Figure 2 and will improve the clarity of the Figure in the camera-ready version of the paper.\nPlease let us know if there are any other questions that could further convince the reviewer of our work, and we will be happy to address them.", "claims_and_evidence": "The paper claims the following:\n1. Novel semantic region-based approach: The paper builds on the COMET approach by using regions instead of pixels. I believe this claim is valid, as it is a novel approach in this field.\n2. Dynamic thresholding that adjusts sparsity: They propose a method for dynamic thresholding by uniformly sampling threshold parameters during training and then using Equation 4 as a threshold parameter for an instance during inference.\n3. Thorough empirical assessment: The paper includes extensive experiments. They compare closely to the blackbox approach, as shown in Table 1, and demonstrate effective localization, as presented in Table 2. The study also features robust ablation studies and visual comparisons.", "methods_and_evaluation_criteria": "Yes, I believe that both the method and the evaluation criteria make sense overall.", "theoretical_claims": "I don't think there are any formal theoretical claims in this paper.", "experimental_designs_or_analyses": "Yes, I believe that the experimental design is well-structured and effectively evaluates the paper. It compares with blackbox models and other relevant methods, and it includes experiments on localization, ablation studies, and visual comparisons.", "relation_to_broader_scientific_literature": "The paper's key contribution is closely related to inherent interpretability literature. They build on methods like COMET.", "essential_references_not_discussed": "I'm not very familiar with this literature in depth. However, based on the related works section, it seems the paper provides a broad introduction to the topic.", "other_strengths_and_weaknesses": "I personally find the method easy to understand and well written. However, I believe that Figure 2 could be improved with better captions for the freeze and unfreeze parts.", "other_comments_or_suggestions": "None", "questions_for_authors": "None", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "VEhx7Tqb63", "reviewer": "Reviewer_ZT81", "summary": "This paper presents P2P (Pixels to Perception), an inherently interpretable image-classification model that performs instance-wise feature selection using grouped feature sparsification at the superpixel level rather than individual pixels. The authors argue that sparsifying at the pixel level can lead to non-human-interpretable explanations, whereas P2P enforces structured sparsification in perceptually meaningful regions.\n\nKey Contributions:\n- Superpixel-Based Feature Selection: Instead of selecting individual pixels, P2P groups pixels into superpixels (e.g., using SLIC) and learns a binary mask at the region level.\n- Instance-Wise Adaptability: The model dynamically determines the sparsity level per instance rather than applying a fixed threshold across all images.\n- Part-Object Relationship Modeling: A logit-normal distribution models relationships between different superpixels to better capture object structures.\n- Faithful and Interpretable Predictions: The model ensures that only the selected regions contribute to classification, avoiding reliance on dimmed but still informative pixels (as seen in COMET).\n\nKey Results:\n- Comparable accuracy to black-box models while removing up to 80% of the image content.\n- Stronger object localization than existing methods (REAL-X, RB-AEM, COMET, B-cos).\n- High faithfulness as demonstrated by insertion/deletion tests.\n- Evaluations on CIFAR-10, ImageNet, COCO-10, and BAM datasets.\n\nThe proposed approach ensures more human-understandable predictions by selecting perceptually meaningful regions instead of arbitrary pixels.", "questions": "", "limitations": "", "rating": 4, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Dear reviewer, Thank you for your thorough review and positive feedback on our work! As there are no questions, we will keep our response brief.\n\nP2P is computationally efficient, as the computational overhead of the logit-normal covariance modeling is negligible compared to the rest of the architecture.\nAs such, P2P is as fast as Real-X and faster than COMET, which has two classifiers. The computation of the superpixels is done on CPU during dataloading and as such does not introduce a computational overhead. \n\nFor an ablation on the superpixel algorithm used, we refer to our rebuttal to Reviewer kwsZ or Reviewer px8r, showing that P2P does not rely on the specific choice of superpixel algorithm.\n\nPlease let us know if any questions pop up that you would like us to address.", "claims_and_evidence": "The paper makes several claims, all of which are well-supported:\n\nClaim: Superpixel-based feature selection is more interpretable than pixel-wise selection.\nEvidence: P2P’s masks align with object boundaries better than pixel-level baselines. This is demonstrated via visualization and quantitative comparisons against segmentation ground truth.\n\nClaim: P2P achieves high classification accuracy despite removing large portions of the image.\nEvidence: P2P maintains accuracy close to black-box models, outperforming sparsity-based baselines like DiET and RB-AEM.\n\nClaim: P2P produces faithful feature selections.\nEvidence: The insertion/deletion experiments confirm that only the revealed pixels influence predictions, unlike COMET, where inverted masks still perform well.\n\nClaim: P2P dynamically adapts the level of sparsity per instance.\nEvidence: The ablation study shows that fixed sparsity thresholds reduce accuracy, whereas dynamic sparsity maintains performance with better localization.\n\nOverall, the paper provides clear empirical evidence for its claims using well-structured experiments.", "methods_and_evaluation_criteria": "Methods:\n- Uses SLIC superpixels to partition the image into perceptual regions.\n- Learns a logit-normal probability distribution to model relationships among superpixels.\n- Implements dynamic thresholding to control sparsity per instance.\n\nEvaluation Criteria:\n- Classification Accuracy – How well the model classifies images after feature selection.\n- Localization – The overlap of selected regions with ground-truth object masks.\n- Faithfulness – Whether removing the selected regions prevents correct classification.\n\nThe evaluation metrics align well with the problem, and results are reported across multiple datasets. Computational overhead is not analyzed in-depth.", "theoretical_claims": "The paper includes a mathematical formulation of instance-wise grouped feature selection and a derivation proving that the logit-normal covariance matrix is positive semi-definite. The proof (Appendix A) is correct and straightforward, ensuring valid covariance modeling. While the derivations are correct, more discussion on the computational efficiency of logit-normal covariance modeling would be beneficial.", "experimental_designs_or_analyses": "The experiments are rigorous and well-structured:\n- Datasets: CIFAR-10, ImageNet, COCO-10, BAM (semi-synthetic).\n- Baselines: COMET, REAL-X, RB-AEM, DiET, B-cos.\n- Metrics: Classification accuracy, localization overlap, insertion/deletion for faithfulness.\n- Ablation studies: Fixed vs. dynamic sparsity, alternative selection models.\n\nWhile accuracy and faithfulness are well-evaluated, the computational efficiency of P2P compared to baselines is not deeply analyzed.", "relation_to_broader_scientific_literature": "The paper is well-grounded in the literature, drawing connections to:\n- Feature selection (LASSO, instance-wise sparsity methods like REAL-X, DiET).\n- Explainability in deep learning (Grad-CAM, concept-based models).\n- Human perception studies (Gestalt principles, Biederman’s recognition-by-components theory).\n\nThe paper does not compare against vision transformers (ViTs), which are increasingly used for interpretability.", "essential_references_not_discussed": "The references are generally comprehensive, but a few areas could be expanded:\n- Vision Transformer (ViT)-based explainability approaches.\n- Graph-based feature selection methods, which also model part-whole relationships.", "other_strengths_and_weaknesses": "Strengths\n- Perceptually meaningful selection – Superpixel-level masks align better with human understanding.\n- Faithfulness – The model truly depends on the selected regions.\n- Dynamic thresholding – Adapts sparsity to instance complexity.\n- Strong empirical validation – Evaluated on multiple datasets with rigorous baselines.\n\nWeaknesses\n- Computational complexity – Training/inference efficiency is not analyzed.\n- Superpixel quality dependency – Performance might degrade with suboptimal superpixel partitioning.\n- Limited discussion of transformer-based explainability – No comparison to ViTs.", "other_comments_or_suggestions": "n/a", "questions_for_authors": "n/a", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "sLc2DD4Oti", "reviewer": "Reviewer_kwsZ", "summary": "The paper proposes a method that learns a masking function that is able to semantically separating important information from background noise. As a part of this, the authors introduce a dynamic threshold based on classification probabilities to determine the level of sparsity for the instance. The authors evaluate their method for classification and localization benchmarks, where the method perform well.", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We thank the reviewer for the feedback and questions! Below is our point-by-point response.\n\n> Ablating the choice of super pixel algorithm, since its just assumed as a fixed component without investigating the effect of it\n\nWe agree that the superpixels are a central part of our method, thus, warranting an ablation analysis. In some preliminary experiments, we observed that the choice of SLIC's hyperparameter do not have a strong effect on performance, as long as the number of segments chosen is reasonable (i.e. >20).  \nAs part of this rebuttal, we have explored the effect of the specific superpixel algorithm on P2P's performance by replacing the SLIC superpixel algorithm with the Watershed superpixel algorithm, choosing the hyperparameters by ensuring similar number of segments.\n\n| Dataset    | Method            | Accuracy (\\%) | Localization (\\%) |\n| ---------- | ----------------- | ------------- | ----------------- |\n| ImageNet-9 | P2P (Watershed) | 93.95         | 67.70             |\n|            | P2P (SLIC)        | 94.42         | 69.25             |\n| COCO-10    | P2P (Watershed) | 90.05         | 46.95             |\n|            | P2P (SLIC)        | 89.53         | 47.01             |\n\nWe see that there are no strong dependency on the specific choice of superpixel algorithm, further strengthening the generality of P2P. \nWe will include these results in the camera-ready version of the paper.\n\n> Explain why their method does not outperform COMET for the largest and most diverse benchmark ImageNet\n\nWe thank the reviewer for posing this question, as it allows us to improve our explanation of why we believe that COMET does not faithfully make a prediction based on the highlighted pixels, thus rendering its interpretability questionable:\nCOMET learns a continuous-valued mask, thereby highlighting some pixels, while darkening others. Note that darkening an image does not destroy any numerical information that the classifier can use. In order to be an inherently interpretable instance-wise feature selection model, we argue that COMET's classification should rely on the highlighted pixels. In our work, we investigate whether this holds in two ways. \n1. By evaluating Insertion Fidelity in Figure 3 of the submission: We start from a black image and iteratively add the (according to the mask) most important pixels. If the model relied on these pixels, we would expect that it's predictive performance quickly improves. Notice that this is not the case, as COMET is one of the worst-performing methods on this metric.\n2. By evaluation of COMET^-1: To check whether COMET bases its prediction only on highlighted pixels, we introduce COMET^-1. Here, COMET highlights the unimportant pixels and then makes a prediction based on this masked image. If COMET would base it's prediction only the highlighted features, then, this variant should perform very badly, as it highlights unimportant pixels. However, COMET^-1 has a high accuracy. This indicates that COMET does not make a prediction based on the highlighted features only, but also uses the non-highlighted pixels.\n\nCombined, this suggests that COMET's classification does not rely on only the highlighted areas of the image, but on the full image. This indicates that COMET is not an inherently interpretable feature selector.\n\nTo summarize and answer your question: COMET performs well on ImageNet because in contrast to P2P, it does not select a subset of features but still uses the full image to make a prediction.\nWe will improve our explanation in the camera-ready version of the paper.\n\n> Experiment with continuous masking vs. binary masking as opposed to simply assuming one is better than the other\n\nAs we argued and seen previously for COMET, we strongly believe that continuous masking does not effectively remove information. Thus, any model with continuous masking is, in our eyes, not a real instance-wise feature selector. Continuous-valued masking has a high risk of spurious correlations[1], where the model uses these darker pixels, and the user does not realize this. As such, we believe continuous masking is a risk for interpretability, where the goal is for the human user to understand the model's decision making. We want to avoid the possibility of P2P having this non-interpretable hard-to-detect behavior. \n\n[1] Geirhos, Robert, et al. \"Shortcut learning in deep neural networks.\" _Nature Machine Intelligence_ 2.11 (2020): 665-673.", "claims_and_evidence": "The claim of SLIC being the optimal superpoint algorithm is not further ablated. Therefore, it is difficult to assess of this selection is optimal.", "methods_and_evaluation_criteria": "- The method section is well structured\n- The Equations before and after Eq 3 lack numbering\n- The intuition behind using a dynamic threshold is reasonable and a good design choice\n- Since the dymanic thresholding takes into account a certainty measure (class prediction softmax „probability“), a look at alternative uncertainty quantification methods from active learning could be interesting. Examples for epistemic uncertainty scores could be [1] and [2]\n- The choice of benchmark datasets is suitable for the task. Especially, IN-9 with background changes is an interesting choice\n\n[1] Gal, Yarin, and Zoubin Ghahramani. \"Dropout as a bayesian approximation: Representing model uncertainty in deep learning.\" international conference on machine learning. PMLR, 2016.\n[2] Rahaman, Rahul. \"Uncertainty quantification and deep ensembles.\" Advances in neural information processing systems 34 (2021): 20063-20075.", "theoretical_claims": "- The proof for semi-definiteness in the appendix is sound", "experimental_designs_or_analyses": "- Distortion (continuous values) vs. Removal (binary): The authors simply make the assumption that this design choice is the superior. I think an evaluation the difference in performance between both alternatives would be an important ablation\n- The method relies on SLIC for super pixels. Since this is an off-the-shelf component of the method, an ablation of using alternative super pixel algorithms and their impact on performance is essential\n- The method shows only minimal gains for classification over the next best competing method, COMET, and being outperformed on the largest, most extensive benchmark ImageNet-1K. The gains for localizations are larger though.", "relation_to_broader_scientific_literature": "In contrast to the best competing method COMET, the proposed method focuses on relevant foreground regions. There is an inverted version of COMET though, which focuses on the background, but the proposed method mostly outperforms ist", "essential_references_not_discussed": "None", "other_strengths_and_weaknesses": "My comments fit well into the previous sections", "other_comments_or_suggestions": "- In the introduction, the acronym P2P is just introduced without explaining what it means. One can guess from the title, but it should be introduced anyhow", "questions_for_authors": "Generally, the paper is well written and interesting. I would appreciate appreciate if the authors would further improve their work by:\n- Ablating the choice of super pixel algorithm, since its just assumed as a fixed component without investigating the effect of it\n- Explain why their method does not outperform COMET for the largest and most diverse benchmark ImageNet\n- Experiment with continuous masking vs. binary masking as opposed to simply assuming one is better than the other\n\nSince my experience with this subfield is limited, I will also take into account the comments from other reviewers for my final judgement", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "v10L17awx4", "reviewer": "Reviewer_px8r", "summary": "This paper introduces **P2P (From Pixels to Perception)**, an instance-wise feature selection method aimed at improving interpretability by selecting **grouped semantic regions** instead of individual pixels. While interpretability is a key challenge, the approach **lacks novelty** and does not sufficiently differentiate from existing **masking-based feature selection methods**.", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Dear reviewer, we thank you for your comments and feedback. Please find below our answers to the open points.\n\n> How does P2P compare to InfoMask?\n\nWe thank the reviewer for pointing out this baseline. InfoMask [1] is a nice information bottleneck-inspired approach that learns a masking on pixel level, regularized by a kl-divergence. \nSome notable differences are that 1. InfoMask operates directly on pixel level, and 2. InfoMask uses 'semi-hard' masking, where each masking probability is either 0, or in (0,1). \nWe present the results of applying InfoMask in the same setup as P2P below. For our rebuttal, we focus on ImageNet-9, and COCO-10 over 3 seeds. We will include the results in the camera-ready version of the paper.\n\n| Dataset    | Method   | Accuracy (\\%) | Localization (\\%) |\n| ---------- | -------- | ------------- | ----------------- |\n| ImageNet-9 | InfoMask | 94.69         | 38.55             |\n|            | P2P      | 94.42         | 69.25             |\n| COCO-10    | InfoMask | 89.20         | 26.02             |\n|            | P2P      | 89.53         | 47.01             |\nAdditionally, in https://anonymous.4open.science/r/P2P/figures_rebuttal/, we provide the Insertion curves. While InfoMask, similar to P2P, achieves near-optimal performance, we see that in the other two evaluation axes, P2P outperforms InfoMask. \n\nWe argue that this difference arises because the mask of InfoMask is learned on pixel-basis, which encourages it to behave similarly to Blackbox Pixel. That is, in these complex datasets, InfoMask selects pixels nearly randomly, relying on the fact that a reduction in resolution does not effectively reduce information from the image. On the other hand, P2P selects only a sparse subset of features, that are relevant for the prediction. To support this point, we add a visualization of InfoMask's learned mask, for an example where it works well, in the repository above. Notice the reliance on the noisy sampling.\n\n\n> Why assume feature removal always improves interpretability? Have you tested cases where removal distorts model decisions?\n\nWe agree that by removing features, we effectively reduce the high-dimensional decision boundary to a lower-dimensional one. This can lead to distortions due to the compression, whereby the target loss tries to minimize these distortions. We argue that a lower-dimensional decision boundary improves interpretability, as less complexity is generally easier to understand for humans. This is supported by research in cognitive psychology, such as [2,3]. Naturally, there are cases where the model decision changes, once P2P is used, compared to a black-box classifier. But even for some of these cases, it is understandable (thanks to P2P's masking), and quite frankly amusing, why the misclassification occurred. In https://anonymous.4open.science/r/P2P/figures_rebuttal/, we provide a selection of failure cases, where the added interpretability helps in understanding why the model failed.\n\n> How do superpixels impact interpretability?\n\nWe argue that superpixels aid the model in defining a meaningful feature basis, where the selected features are human-interpretable, in contrast to learning a mask on pixel level.\nAs part of this rebuttal, we have explored the effect of the specific superpixel algorithm on P2P's performance by replacing the SLIC superpixel algorithm with the Watershed superpixel algorithm, choosing the hyperparameters by ensuring similar number of segments.\n\n| Dataset    | Method            | Accuracy (\\%) | Localization (\\%) |\n| ---------- | ----------------- | ------------- | ----------------- |\n| ImageNet-9 | P2P (Watershed) | 93.95         | 67.70             |\n|            | P2P (SLIC)        | 94.42         | 69.25             |\n| COCO-10    | P2P (Watershed) | 90.05         | 46.95             |\n|            | P2P (SLIC)        | 89.53         | 47.01             |\n\nWe see that there are no strong dependency on the specific choice of superpixel algorithm, further strengthening the generality of P2P. \nWe will include these results in the camera-ready version of the paper.\n\n\n[1] Taghanaki, S.A. _et al._ (2019). InfoMask: Masked Variational Latent Representation to Localize Chest Disease. In: Shen, D., _et al._ Medical Image Computing and Computer Assisted Intervention – MICCAI 2019. MICCAI 2019. Lecture Notes in Computer Science(), vol 11769. Springer, Cham.\n\n[2] Miller, George A. \"The magical number seven, plus or minus two: Some limits on our capacity for processing information.\" _Psychological review_ 63.2 (1956): 81.\n\n[3] Sweller, John. \"Cognitive load during problem solving: Effects on learning.\" _Cognitive science_ 12.2 (1988): 257-285.", "claims_and_evidence": "The paper claims that P2P enhances interpretability by **selecting minimal, meaningful features**, but this is not strongly supported. Prior work, such as **InfoMask (Asgari et al., 2019)**, has already explored similar instance-wise masking techniques. The lack of direct comparisons makes it difficult to assess whether P2P truly provides an improvement.", "methods_and_evaluation_criteria": "P2P relies on **SLIC Superpixels** to group features, but this may not always produce **semantically meaningful regions** for highly structured images. Additionally, the assumption that **removing features improves interpretability** is not always valid, as selective removal can distort decision boundaries.", "theoretical_claims": "The method does not introduce significant theoretical advancements beyond **existing feature selection approaches**. The assumption that **less information leads to better interpretability** should be tested more rigorously.", "experimental_designs_or_analyses": "The experiments **lack comparisons to strong baselines** like **InfoMask** or other **instance-wise feature selection methods**. Without these, it is unclear whether P2P offers meaningful improvements over prior techniques.", "relation_to_broader_scientific_literature": "The paper is related to **interpretable machine learning and feature selection**.", "essential_references_not_discussed": "**InfoMask (Asgari et al., 2019)** – A key prior work on instance-wise masking-based feature selection.", "other_strengths_and_weaknesses": "No", "other_comments_or_suggestions": "_+_ Focuses on interpretability, an important ML challenge.\n\n_-_ Lacks novelty and questionable assumptions about feature removal.", "questions_for_authors": "1. How does P2P compare to InfoMask? \n2. Why assume feature removal always improves interpretability? Have you tested cases where removal distorts model decisions?\n3. How do superpixels impact interpretability?", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Meta-review: This paper introduces P2P (Pixels to Perception), a novel image-classification model that offers inherent interpretability by selecting features at the superpixel level, rather than individual pixels. P2P incorporates dynamic thresholding, allowing it to adaptively determine the optimal level of sparsity during inference. The authors evaluate P2P across various datasets, demonstrating its ability to generate human-understandable predictions while maintaining predictive performance comparable to black-box models.

All reviewers had a positive opinion of the paper after the rebuttal. They noted that the perceptually meaningful selection of superpixel-level masks aligns better with human understanding and praised the strong empirical validation on various benchmarks. Based on this, I support acceptance of the paper.
Decision: Accept (poster)