LucidPPN: Unambiguous Prototypical Parts Network for User-centric Interpretable Computer Vision
Authors: Mateusz Pach
Koryna Lewandowska
Jacek Tabor
Bartosz Michał Zieliński
Dawid Damian Rymarczyk
Venue: ICLR 2025 Poster (2025)
Keywords: xai
interpretability
prototypical parts
Primary area: interpretability and explainable AI
TL;DR: Enhancing the interpretability of prototypical parts networks by aligning prototypical parts with object parts and separating color and non-color visual features through a novel LucidPPN architecture.

Abstract: Prototypical parts networks combine the power of deep learning with the explainability of case-based reasoning to make accurate, interpretable decisions. They follow the this looks like that reasoning, representing each prototypical part with patches from training images. However, a single image patch comprises multiple visual features, such as color, shape, and texture, making it difficult for users to identify which feature is important to the model.

To reduce this ambiguity, we introduce the Lucid Prototypical Parts Network (LucidPPN), a novel prototypical parts network that separates color prototypes from other visual features. Our method employs two reasoning branches: one for non-color visual features, processing grayscale images, and another focusing solely on color information. This separation allows us to clarify whether the model's decisions are based on color, shape, or texture. Additionally, LucidPPN identifies prototypical parts corresponding to semantic parts of classified objects, making comparisons between data classes more intuitive, e.g., when two bird species might differ primarily in belly color.

Our experiments demonstrate that the two branches are complementary and together achieve results comparable to baseline methods. More importantly, LucidPPN generates less ambiguous prototypical parts, enhancing user understanding.

Published as a conference paper at ICLR 2025
LUCIDPPN: UNAMBIGUOUS PROTOTYPICAL PARTS NETWORK FOR USER-CENTRIC INTERPRETABLE
COMPUTER VISION
Mateusz Pach- Koryna Lewandowska Jacek Tabor
Bartosz Zielinski Dawid Rymarczyk!.t. FFaculty of Mathematics and Computer Science, Jagiellonian University ?Department of Cognitive Neuroscience and Neuroergonomics, Jagiellonian University edu.pl
SDDEAS NCBR "Ardigen SA
ABSTRACT
Prototypical parts networks combine the power of deep learning with the explainability of case-based reasoning to make accurate, interpretable de- cisions. They follow the this looks like that reasoning, representing each prototypical part with patches from training images. However, single image patch comprises multiple visual features, such as color, shape, and texture, making it difficult for users to identify which feature is important the model. To reduce this ambiguity, we introduce the Lucid Prototyp- ical Parts Network (LucidPPN), novel prototypical parts network that separates color prototypes from other visual features. Our method emn ploys two reasoning branches; one for non-color visual features, processing grayscale images, and another focusing solely on color information. This separation allows us to clarify whether the model's decisions are based on color, shape, or texture. Additionally, LucidPPN identifies prototypical parts corresponding to semantic parts of classified objects, making compar- isons between data classes more intuitive, e.g., when two bird species might differ primarily in belly color. Our experiments demonstrate that the two branches are complementary and together achieve results comparable to bascline methods. More importantly, LucidPPN generates less ambiguous prototypical parts, enhancing user understanding.
INTRODUCTION
Increased adoption of deep neural networks across critical fields, such as healthcare (Rymar- ezyk all 2022b), and autonomous driving (Wu et al. 2017). shows the need develop models in which decisions are interpretable, ensuring accounlabillty and transparency in decision-making processes (Rudin, 2019 Rudin al. 2022 One promising approach is based on prototypical parts \Chen et al. 2019; Nauta et al., 2023 Rymarczyk et al. 2021 2022d), which integrate the power of deep learning with interpretability, particularly in fine-grained image classification tasks. During training, these models learn visual concepts characteristic for cach class, called Prototypical Parts (PPs). In inference, predictions are made by identifying the PPs of distinct classes within image. This way, the user is provided with explanations in the form of "This looks like that".
The primary benefit of Pasebased methods over post hoc approaches their ability to in- corporate explanations into the prediction process (Chen et al., 2019) directly. Nevertheless, a significant. challenge with these methods lies in the ambiguity of prototypical parts, vi- sualized using five to ten nearest patches. Each patch encodes a range of visual features, including colog"] shape, texture, and contrast (Nauta et al. (021a), making it difficult for users to identity which of them are relevant. This issuc is compounded by the fact that neu- ral networks are generally biased towards texture (Geirhos et al. 2019) and color Hosseini
et all 2018 whereas humans are typically biased towards shape Lannaau et al.) 1988
Wee follow the color definition from the research of (Berga et al.
2020 Khan al.
2012

---

Published as a conference paper at ICLR 2025
Existing metbois
This does look like that. but cunmot zell you why
LacidPeN (ours)
look like that BECAUSE although the shape und testure similar cofor differs
&
2-1
Figure 1: Our novel prototypical parts-based model, LucidPPN, enables the disentangling of color information from the prototypical parts. This capability allows to examine more closely the differences between an image patch and patches representing a prototypical part. As shown in the image, prototypical part of a bird's head from different classes, shows a high resemblance in shape and texture but differs in color. Such detailed analysis was not possible with previous prototypical parts-based approaches.
model can visualize that the head of bird, compared to the
Therefore, recent works have attempted to solve this problem using various strategies. Some works propose reduce the ambiguity of prototypical parts by visualizing them through larger number of patches (Ma et al. 2024 Nauta et al. 2023). However, it does not solve the problem with various visual Teatures encoded in each patch. Other approaches tend to solve this problem by quantifving the appearance of specific visual features (Nautal et all, 2021a) or concepts (Wan et al. 2024) on prototypical parts. However, they generate ambiguous statements such as "color is important", leading to further questions (c.g. about which color) that complicate understanding (Ma ot al., 2024 Xu-Darme et al., 2023
Motivated by the challenge of decoding the erucial visual attributes of prototypical parts, we introduce the Lucid Prototypical Parts Network (LucidPPN). uniquely divides the model into two branches: the first focuses on identifying visual features of texture and shape corresponding to specific object parts (e.g. heads, tails, wings for birds), while the second is dedicated solely to color. It allows us to disentangle color features from the prototypical parts and present pairs of a simplified gray prototypical part and corresponding color (see Figure [] The second advantage of LucidPPN is that the successive prototypes in each class correspond to the same object parts (c.g., the first prototypes are heads, the second prototypes legs, etc.). Altogether, it enabled presented in Figure 2 more intuitive and less ambiguous according to our user studics.
introduce novel type of visualization
Extensive experiments demonstrate that LucidPPN achieves results competitive with cur- rent PPs-based models while successfully disentangling and fusing color information. Addi- tionally, using LucidPPN, feature, as demonstrated on the Stanford Cars dataset (Krause et al. 2013). Finally, study showed that participants, guided by LucidPPN explanations, more accurately identified the ground truth compared to those using PIP-Net.
can identify tasks where color information is an unimportant
Our contributions can be summarized as follows:
We introduce LucidPPN, a novel architecture based on PPs, which disentangles color features from the PPs in inference. Consequently, thanks to LucidPPN know the relevance of the color and shape with texture in the decision process
We propose mechanism that ensures successive prototypes within each class sistently correspond to the same object parts.
We introduce a more intuitive lype of visualization incorporating the assumption
about the fine-grained classification..
We conduct comprehensive examination demonstrating the usability and limita- tions of LucidPPN. Specifically, we highlight scenarios where color in information may not be pivotal or even confuses the model in fineggrained image classification.
SSe the discussion in paragraph Color Impact in Section 5.
2

---

Published as a conference paper at ICLR 2025
Evidence for Lazuli Bunting
Part prototypes of Lazuli Bunting
ScOre
0.9
1.0
0.90
1.0 &
1.0
1.00
0.86
0.9
0.9
0.81
0.8
0.72
Our novel type of visualization utilizes the fact that the successive prototypes each class LucidPPN correspond to the samee object parts. That is why we use a schematic drawing of a bird to show the location of the specific prototypical parts. Moreover, LucidPPN disentangles color features from the prototypical parts to present pairs of a simplified gray prototypical part and a corresponding color. The aggregated resemblance is obtained by multiplying the resemblance to the prototypical part and the resemblance to the corresponding color.
Figure 2:
2 RELATED WORKS
Ante-hoc methods for XAI. Self-explainable models (ante-hoc) aim to make the de- cision process more transparent by providing the explanation together with the prodiction, and they have attracted significant attention (Alvarez Melis & Jaakkola 2018 Böhle et al. 2022 Brendel Bethge 2019). Much of this atlention has locused on enhancing the concept of prolotypical parts introduced in ProtoPNet. (Chen el al. 2019) to represent the activa- including TesNet (Wang which exploit orthogonal- 2021). Proto'Tree (Nauta
tion patterns of networks. Several extensions have been proposed,
et all. 2021) and Deformable ProtoPNet I (Donnelly et al. 2022).
ity in prototype construction. ProtoPShare (Rymarczyk et al.
et all, 2021b), ProtKNN (Ukai al., 2022), and ProtoPool [Rymarczyk et al.) 2022d
reduce the number of prototypes used in classification. Other methods consider hicrarchi- cal classification with prototypes (Hase et. al., 2019), prototypical part transformatioo (Li
el all, 2018), and knowledge distillation techniques from protolypes (Keswani et al. 2022 ).
Prototype-base solutions have been widely adopted in various applications such as medical
imaging (Afnan et al. 2021 Barnett et al. 2021 Kim et al. 2021 Rylaayyyyrckkk et al. 2022b
time-series analysis |Gee al. 2019), graph classification Rymarczyk et al. 2023a Zhang
et all, 2022), semantic segmentation (Sacha et al. 2023) and class incremental learning
marczyk al. 2023b
However, prototypical parts still need to be improved, espocially regarding the understand- ability and clarity of the underlying fcatures responsible for the prediction (Kim et al. parts
2022). Issues such as spatial misalignment of prototypical parts (Carmichael et al. 2024 Sacha el al. 2024h and imprecise visualization techniques (Gautam
al. 2023 Xu- Darme
el all 2023 have been identified. There are also post-hoc explainers analy yzing visual lea- tures such as color, shape, and textures (Nauta al. 2021a). and approaches using multiple image patches to visualize the prototypical parts (Ma et al. 2024 Nauta et al. 2023). In this work, we address the ambiguity of prototypical parts by presenting a dedicated archi- tecture, LucidPPN, that detects separate sets of prototypes for shapes with textures and another set for colors. This approach aims to enhance the interpretabilitty and clarity of the interpretations.
Usage of low-level vision features for image classification. Multiple approaches to extracting features based texture (Armi & Fekri-Ershad 2019 Haralick al.) 1973)
shape (Khan el al. 2012 Minggiang el al. 2008). and color CCen el al.) 2010 Kobayashi & Otsu, 2009) have been proposed before the deep learning era, based on the knowledge about human perception (Fan et al.) (2017). Similar features are trained by shallow layers of deep networks, which can be visualized with methods such as (Zeiler) 2014 Springenberg et all 2014). However, while those methods effectively illustrate low-level features, they struggle with deeper layers, where the visualized concepts entangle multiple visual features and lead to ambiguous explanations. Similar behavior can be observed in recent eX eXplainable
AI (XAI) methods [Basaj et al.) 2021 Laina et al. 2022 Rymarczyk et al.] 2022a Zicliński
& Górszczak, 2021 Nauta et al. 2021a). By using two branches, one for color and one for remaining visual teatures, our method explicitly disentangles these visual features, reducing
ambiguity of explanations based high-level concepts.
3

---

Published as a conference paper at ICLR 2025
3 METHOD
3.1 PRELIMINARIES
Problem formulation. Our objective is to train fine-grained classification model based on prototypical parts, which accurately predicts one of M subtly differentiating classes. We use N image-label pairs yo)....)...... (IN, [N)} C lIx {I,..., M a training set to obtain the model returning highly accurate prodictions and lucid explanations. For this, separate color from other visual features at the input and process them through two network branches with separate sets of PPs.
PDiscoNet. PDiscoNet (van der Klis et al. 2023) generates segmentation masks of object parts, used in training of LucidPPN to align K successive prototypical parts of cach class with K successive object parts. We docided to use it instead of human annotators because it is more efficient and cost-effective. However, it can be replaced with any method of object part segmentattio due to the modularity of our approach.
PeiscoNet model frisco utilizes a convolutional neural network (CNN) to generate a feature map ZDisco [zaj], E (RDDnsco) ) X XW from given image Each of D x W Wisco vectors from such feature map is then compared to trainable vectors q² RDDocco representing K object parts and background, using similarity based on Euclidean distance
exp(-Ilag - (²(
t = >K+ exp(-11=1s 112)
(1)
for i = L,..,, W Disco and j = I,..., Disco, and k E {l,.... K + .l. an attention map T = [tlyles E R Xi Disco for each object part and background. Such attention maps are multiplied by feature map Disco and averaged to obtain one vector per object part. Those vectors are passed to the classification part of PDiscoNet, which involves learnable modulation vectors and a linear classifier.
This way, we obtain
A vital observation that the maps T continuously split the image into regions corre- sponding to discovered object parts thanks to well-conceived set of loss functions added to the usual cross-entropy. They assure the distinctiveness, consistency, and presence of the semantic regions. Yet, the only annotations used in training are the class labels.
In the subsequent sections, ignore the PDiscoNet predictions Piscos using only the attention maps T which we will call segmentation masks from now on.
3.2 LUCIDPPN
3.2.1 ARCHITECTURE
LucidPPN is a deep architecture, presented in Figure 3 consisting of two branches: one for revealing information about shape and texture (ShapeTexNed) and the second dedicated to color (ColorNet). That is why ShapeTex.Ne operates on grayscaled input, while ColorNet uses aggregated information about the color.
SeapeTex.eet. A grayscaled version of image obtained by converting its channels æ (1,9,b) to xs = (w, w, w), where 0.299r 0.587g + 0.114b. This formula approximates human perception of brightness (Pratt 2013) and is default grayscale method used in computational libraries, such as Pyrch Paszke et al. 2019).
Grayscaled image æy is fed to convolutional neural network backbone fs,- For this purpose, we adapt the ConwNeXt-tiny (Liu et al. 2022) without classification head and with increased stride at the two last convolutional layers to increase the resolution of the feature map, like in PIP-Net (Nauta et al.) 2023).. As an output of fs., we obtain a matrix of dimension (D x H x W), which is projected to dimension K A x H W using x convolution layer fs.. (where K is the number of object parts and M is the number of classes) so that each prototype has its channel. Then, apply the sigmoid. As result, we obtain Shape TexNel feature map defined as
reshaped to the size of K x M x x W on which we
7s [= kmm (Ss. (Ss, (fs))) fs(2s) E HR& y
4

---

Published as a conference paper at ICLR 2025
Inputxx
Grayscaled impul.xg Ks
Downscaled inpul
ShapePeaNet
ColorNet
feature map Zg
feature map Ze
Information fuston
feature map Z
Prototypical
part
MaxPool2D
Object part
resemblance R2
Object part
+
Mean
Class
0.1
0.9
prediction
Class
Class B
Class
Figure LucidPPN architecture consists of two branches: ColorNet and Shape TexNet that encode color and shape with texture in feature maps Zo and Zs, respectively. Thanks to a special type of training each channel a feature map corresponds to similarity to specific object part of a given class. In this image, green and orange correspond to two object parts: head and belly, and red and blue correspond to classes and B. Therefore, each feature map consists of four channels for head of A, belly of A, head of B, and belly of B. Corresponding channels from both branches are multiplied to obtain feature map Z which is then pooled with maximum to obtain the resemblance of prototypical parts fusion and aggregated through mean to obtain final logits.
Thus, we link each map 2 to a unique prototypical part of an object part & for class m, from which we compute ShapeTezNel TexNel resemblance Rg = km Jk.m [0, 1] K where
km MaxPool2DD(s5"))
[0, 1IM by taking the mean over Finally, we obtain Shape TeopeTexNee TeaNet predictions Ps = 1 the resemblance all parts of a specific class
PS
km S
(4)
K
k=1
ColorNet. To obtain aggregated information about color, x downscaled through bilincar interpolation to H x W resolution, marked as Zc. Then, Te passed to convolutional neural network fc, composed of six with ReLU activations, except the last layer after which we apply sigmoid. This way, we process each input pixel of se separately, taking into account only its color. As result, obtain ColorNet feature map
imput of ColorNet, image
convolutional layers
Zc = [==
(=)) fe(rc) E (R
Analogously to ShapeTexNe each dimension in the feature map is related to unique prototypical part an object part in class m. Hence, as before, we calculate ColorNet resemblance Re l E [0, K where
C MaxPool2DD:
(6)

---

Published as a conference paper at ICLR 2025
I
Object
Grayscalc
Downscale
part I
ShapeTexNet f
PDiscoNet fxte
ColorNet
Object
part
feature map 2
segmentation masks
feature map Ze
Class A
Class B
Stop gradient
Frozen weights
resembiance Rs
resembiance
Ground truth
label.y
Loss
Figure 4: LucidPPN training schema. We use segmentation masks from PDiscoNel to align the activation of prototypical parts with object parts. Additionally, we enforce the SeaPeT to encode much predictive information possible through the usage of Ls. Lastly, we learn how to classify images through La which is a binary cross-entropy loss.
Information fusion and prediction. To obtain aggregated feature map Za 1= E XKM from both branches, we multiply the Shape TexNet feature map with ColorNet feature map element-wise
kin
km 25
④ 2C
km
(7)
and define aggregated resemblance R = fm kmm E [0,1] as
km A
MaxPool2DD
(8)
The final predictions أ = [6"] Im E [0, 1 for all classes are obtained by averaging km over rlsstrelated parts
r
K
gTT
Am
(9)
K
3.2.2 TRAINING
1=1
As a result of LucidPPN training (presented Figure 4 we aim achieve three primary goals: 1) obtaining a high-accuracy model, 2) ensuring the correspondence of prototypical parts to object parts, 3) and disentangling color information from other visual features. To accomplish these goals, we design three loss functions: 1) prototypical-objec part correspon- dence loss Lp. 2) loss disentangling color from shape with texture Lg, 3) and classification loss LA that contribute to the final loss
L = apa + asLs + GALA,
(10)
where aD, a.s; O are weighting factors whose values are found through hyperparameter scarch. The definition of each loss component is presented in the following paragraphs. Please note that we assume that PDiscoNet was already trained, and we denote ÿj B as one-hot encoding of y.
Correspondence of prototypical parts to object parts To ensure that each pro- totypical part assigned
given class corresponds distinct object parts, we define the prototypical-ojject part correspondence loss L.D This function leverages segmenta- tion masks & from NDiscoNet to align the activations of prototypical parts, represented by the Shape TexNet feature map Zs with the locations of object parts. Hence, the activations from the aggregated feature map ZA will be aligned with these object parts. It defined as
Lp
MBCE Z , TT).
(11)
where MBCE(u, v) is defined as the mean binary cross-entropy loss between two maps u E [0, xxW
MBCE(U, v) HW i= 1=
1
Σ Σ
BCE(Wij viy),
(12)
and y is the ground truth class. We align only the maps corresponding to y because the prototypical parts assigned to other classes should not be highly activated.

---

Published as a conference paper at ICLR 2025
Disentangling color from other visual infromation. To maximize the usage of in- formation about shape and texture during the classification with prototypical parts, we maximize the accuracy of the Shope TexNet through the usage of binary cross-entropy classification loss function on Shape TexNet resemblances values
M
K
Ls
1 KM
Σ m=
BCE km 9m))
(13)
Classification loss. Lastly, to ensure the high accuracy of the model and to combine information from ColorNet and ShapeTexNet, we employ binary cross-entropy on aggregated resemblances as our classification loss
M
K
LA
BCE(rA" km (Tm))
(14)
KM
3.3 PREDICTION INTERPRETATION
LucidPPN adopts the definition of prototypical parts from PIP-Net (Nauta al., 2023 where cach prototypical part is represented by ten patches, typically activated by ten colored images from the training set. However, in LucidPPN, the visualization must demonstrate how each protolypical part is disentangled into color and shape with texture features. That why we propose a method lo present the disentangled visual features a prototypical part by combining five grayscale patches, color bar, and five colored patches. The grayscale and colored patches are selected from the training images with the highest Shape TexNet resemblance and aggregated resemblance, respectively. The color bar is created by sampling RGB color values from the ten colored patches with the highest aggregated resemblance and projecting them using t-SNE (Van der Maaten & Hinton 2008). Morcover, in contrast to PIP-Net, LucidPPN creates prototypical parts corresponcingg to the same object parts in all classes. Therefore, we can use the information about the specific object part location to enrich the explainability.
Local (prediction) interpretation. Figure 2 demonstrates how LucidPPN classifies specific sample x into class by examining the prototypical parts assigned to ÿ that are disentangled into color and other visual features. The views are enhanced with pointers to regions of highest aggregated resemblance, clearly associated with the object parts.
Comparison explanation. Users may wish to inspect and compare local explanations for multiple classes. LucidPPN facilitates this comparison by allowing users to compare prototypical parts of corresponding object parts, making the process intuitive, as shown in pplementary Figure g
Class (global) characteristic. Disentangled protolypical parts corresponding object parts reveal the patterns the model uses to classify a given class. This enables the identi- fication of texture and shape features, as well as colors (see Sup. Figure 21). that describe class without the need to analyze the final-layer connections, unlike other prototypical part-based approaches (Chen et al. 2019 Donnelly et al. 2022
Ma et al., 2024 Nauta
et al. 2023 Rymarczyk et al. 2021 2022c
4 EXPERIMENTAL SETUP
Datasets. We train and evaluate our model on four datasets: CUB-200-2011 (CUB) with 200 bird species Waah et al. 2011), Stanford Cars (CARS) with 196 car models Kranse et all 2013), Stanford Dogs (DOGS) with 120 breeds of dogs (Khosla et al. 2011] and Oxford 102 Flower ('LOWER) with 102 kinds of flowers (Nilsback & Zisserman 2008 More details on image preprocessing are in the Supplement.
Implementation details. Trainings are repeated times. We made the code public. The size of Shape TexNet feature map is 768 28 x 28. The channel number of ColorNet's con- volutional layers 20, 50, 150, 200, 600, K-M. The values loss weights are found through hyperparameter search (aD 1.4, as = 1.0, a = 1.0). Details are in the Supplement.
Metrics During the evaluation, we report the top-1 accuracy classification score. Addi- tionally, we measure the quality of prototypical parts alignment with object parts by cal- culating intersection-over-uiii (IoU). In PDiscoNet, the segmentation map the size of
7

---

Published as a conference paper at ICLR 2025
Table 1: Comparison of accuracy of PPs-based models on 4 datasets. LucidPPN achieves competitive results to all methods, and SOTA on 2 datasets. Note that, LucidPPN is trained with K 12, and 8_ means that the model did not converge during training when using the code provided by the authors.
CUB
CARS
DOGS
FLOWER
ProtoPNet [Chen et al.
2019
79.2
86.1
77.4=0.2
92.1+0.3
ProtoTree Naula et al. 202T6
82.2 1.7 0.7
86.6 1 0.2
ProtoPShare Hymarczyk cl al. 2021
74.7
86.4
74.1
90.3.10.2
ProtoPool (R Rymnarczy el al. 2022c
85.5 0.1
88.9 1 0.1
71.710.2
92.7100.1
PIP-Net
Nauta et al.
2023
84.3
88.2 0.5
80.8 + 0.4
91.8 + 0.5
LucidPPN
81.5 -: 0.4
91.6 + 0.2
79.4 = 0.4
95.0 + 0.3
the input image, while our activation map is the size of the latent space. Hence, we are downsizing the segmentation map to 26 26 resolution to match its dimensions with the activation map before calculating the ToU between the corresponding patches both maps. User studics. Using ClickWorker System we run two user studics to compare the quality of patch-based prototypes and the influence of disentangled resemblance scores provided by LucidPPN. For the first study, we collect the lesting examples from CUB which are correctly classified by PIP-Net, single branch CNNT and LucidPPN. These are joined with information about the two most probable classes per model and associated prototypical parts. .Nintyy workers (30 per method) answer the survey, which consists of 10 questions. They are asked to predict the model's decision based on the evidence for the top two output classes without the numerical scores. This approach mimics the user study presented in HIVE (Kim al. 2022) and is also inspired by the study performed in (Ma et al.) 2024h. In the second study, we also collect images from CUB. This time we join them with prototypical parts of the correctly predicted class and one other class. Each of the forty workers answers 10 questions in which he/she rates from 1 (least) to (most) to assess the influence of the color features on the model's prediction. The users give ratings based on LucidPPN prototypical parts visualization, with or without included numerical resemblance scores. More details and the survey templates in the Supplement.
5 RESULTS
In this section, we show the effec- liveness of LucidPPN, the influence of the color disentanglement in the processing on the model's perfor- mance, and the results related to the interpretability of learned prototyp- ical parts based on the user study. Comparison to other PPs- based models. In Table [ we compare the classification quality of LucidPPN and other PPs-based methods. We present the mean ac-
Table 2: Comparison of the accuracy of ShapeTeNNet to LucidPPN. Integrating color with other visual fea- tures proves advantagcous for datasets containing objects found in nature. However, for the CARS dataset, adding color information docs not enhance the model's performance. This is because color is not a significant. feature when classifying vehicles, as the same car model can appear in various colors.
CUB
CARS
DOGS
FLOWER
Shape LucidPPN
80.4
91.7
78.6
93.6
81.8
91.7
78.9
95.3
euracy and standard deviation. We report best performing LucidPPN, which in the case of all datasets was trained with fixed K = 12. Our LucidPPN achicves the highest accuracy for CARS and FLOWER datasets, and competitive results on CUB and DOGS.
Color impact. The influence of ColorNet on LucidPPN predictions is shown in Table 2 We compare the accuracy of Shape TexNet with the LucidPPN predictions. The information fusion enhances the results the CUB, DOGS, and FLOWER. However, it does not affect performance on the CARS. This can be attributed to the characteristics of the CARS dataset, where vehicles of the same model can differ in colors, indicating that color not
httpp://wwee https://ww..clccewwrrerrrr com/
For ablation analysis we also report results of a single branch CNN which has the same archi- tecture as Shape TexNet and receives colored images input. Its local interpretation is visualized similarly to LucidPPN, but without the gray patches and color har as presented in Sup. Figure 13
8

---

Published as a conference paper at ICLR 2025
critical for this task. This contrasts with the fine-grained classification of natural objects, such as birds and flowers, where color plays a significant role.
In Table 3 we show the results of experiments aiming to analyze how the model is susceplible to the change of the color on the image. We report the accuracy of PIP-Net, Shape and LucidPPN on original and hue-perturbed images from the CUB dataset. One can notice that PIP-Net is highly dependent on color information and its score drops by over 37% after perturbation. At the same time Shape TezNet is immune to this transformation, while LucidPPN loses approximately 12.5% accuracy because of it. To alter hue we randomly rotate hue values in the HSV color space. After rotation, adjust the luminosity of each pixel by proportionately scaling its RGB channels. This step is key to modifying the hue without changing the brightness perceived by humans.
User studies. Statistics from the user study assessing the lucidity of explanations gener- ated by LucidPPN, single branch, and PIP- Net are in Figure D and Supplementary Ta- ble [ We report the mean user accuracy with a standard deviation and p-values. Users bas- ing their responses on LucidPPN explanations score significantly better than both PIP-Net and random guess baselines. Additionally, we conclude that most of the accuracy in this user study can be attributed to the PDiscoNet part supervision as single branch scores similarly to LucidPPN, without statistically significant difference. While both of our explanation vari- ants with prototypical parts corresponding to the same object parts prove to be more intu- itive for users, we also want highlight the advantage of using full LucidPPN over single branch. To this end, in Supplementary Figure 7 and Supplementary Table G we show the outcomes of the study evaluating the user's ability to recognize the importance of color fea- tures in LucidPPN's decisions. Users without information about resemblance values struggle in this task achieving the same performance as if they answered at random. In contrast, users provided with the resemblances in LucidPPN visualizations score 23% better. Note that neither single branch nor PIP-Net gives the disentangled resemblance values. In both studies, perform one-sided t-test and one-sample t-test to compare methods against each other and 50% accuracy, respectively. More details can be found in the Supplement.
Table 3: Robustness of the model to changes in image color. When the hue value is perturbed, the accuracy of PIP- Net drops significantly. In contrast, the ac- curacy drop for LucidPPN is only half as much, and for
none.
Original Hue-perturbed
PIP-Net TexNel LucidPPN 81.9
83.9
53.0
80.3
80.3
71.7
ABLATION AND ANALYSIS
In this section, we examine how LucidPPN's perfor- mance is impacted by object part supervision and the weights of the loss function components.
100
p-value
Accuracy [%)
Influence of part supervision on the perfor- mance of LucidPPN. One of the features of Lu- cidPPN is object part supervision based on PDis- coNet. To check its influence on the PPs-based model without disentanglement, in Table $ we compare the accuracy of a single branch to LucidPPN and PIP- Net. The single branch scores better than both mod- els. The disentanglement in LucidPPN causes a small (<6%) or negligible drop in accuracy while offering more insights from the model.
50
Randam
PIP-Net
LucidPPN
Single branch
Figure 5: User study results show that users based on LucidPPN ex- planations outperform those with explanations from PIP-Net to sta- tistically significant degree.
Loss weighting. In Figure i we investigate the im- pact of the loss weight ap, which is responsible for prototypical-object parts alignment, on training outcomes. In this analysis, the weights of the other losses are fixed at ag = ac = 1. We evaluate the accuracy and intersection- over-union (IoU) between the highest activated ShapeTeNNe feature map and PDiscoNet's segmentation masks for each object part. The results show that increasing an enhances the ToU, but after a certain point, it gradually reduces accuracy. Notably, omitting the
9

---

Published as a conference paper at ICLR 2025
loss ap (part supervision) significantly diminishes the network's classification performance and makes the learned prototypical parts collapse into a single, most descriptive one as pre- sented in Supplementary Figure 10. While other works address this issue by adding novel regularization losses (Nauta et al. 2023 Wang et al. 2021), these solutions fail to ensure consistency of the considered parts across different classes.
We provide additional results in the Supplementary Materials. They include ablations on the LucidPPN's backbone, the size of a ColorNet, type of input color representation, number of object parts K, and different training schodules for LucidPPN's branches. Also, show examples of PIPNet failures mitigated by LucidPPN, and we discuss the reasons for introdug Lg.
CONCLUSIONS
In this work, we propose LucidPPN, an inherently interpretable model that uses prototypical parts to disentangle color from other visual features in its explanations. Our extensive results demonstrate the effectiveness of our method, and user studies confirm that our explanations are less ambiguous than those from PIP-Nct. In futurr rescarch, we aim to further refine the model architecture to separately process shape and texture features, as well as analyze different visualization stralegies of disentanglernent and their recognition ion by the users. Additionally, we plan to explore the human perception syslem in greater depth to inform the design of the next generation of interpretable neural network architectures.
Table 4: Accuracy of PIP-Net, LucidPPN, and a single branch CNN supervised by PDiscoNet.
CUB
CARS
DOGS
FLOWER
PIP-Net LucidPPN
84.3
88.2
80.8
91.8
81.5
91.6
79.4
95.0
single branch
86.6
91.9
82.7
95.6
Limitations. Our work faces a significant con- straint: while our designed mechanism adeptly disen- tangles color information from input images, it can- not currently extract other crucial visual features such as texture, shape, and contrast. This high- lights a broader challenge within the field: the ab- sence of universal mechanism capable of encom- passing diverse visual attributes. Furthermore, our approach inherits limitations from other Paseb architcctures, including issues such as spatial mis- aligument (Sacha et al. 2024). the non-obvious in- terpretation of PPs (Ma et al.) 2024] and those of 2023). The latter could be PIP-Net (Nauta et al. addressed with uextual descriptions of concepts dis- covered by PPs. Lastly, LucidPPN increases the transparency of the decision made by the deep neu- ral networks however it still has a performance gap to black-box models, or even to those offering some insights into the model reasoning process such PDiscoNet (van der Klis ct 2023 This shall be under further investigation to fill this performance gap if possible.
-40
Accuracy [9)]
Jo0 [%)
76
25
10
0
dp
Figure 6: Influence of the weight of prototypicaloojjett part corre- spondence loss on accuracy and Intersection-ove--Uniioo
(IoU). An increase of ap improves loU but at a certain point gradually reduces ac- curacy.
Broader Impact. Our work advances the field of interpretability, a crucial component for trustworthy AT systems, where users have the right to understand the decisions made by these systems (Kaminski) 2021 Tabassi 2023). LucidPPN enhances the quality of explana- tions derived from PPs- based neural networks, which are among the most promising tech- niques for ante- hoc interpretability methods. Consequently, it can facilitate the derivation of scientific insights and the creation of better human-AI interfaces for complex, high-stakes applications.
Additionally, LucidPPN provides visual characteristics for PPs, which are espocially ben- eficial in domains lacking standardized semantic textual descriptions of concepts. This is particularly useful fields such as medicine, where it aids in analyzing radiology and histopathology images.
10

---

Published as a conference paper at ICLR 2025
REPRODUCIBILITY STATEMENT
We ensured that our experiments are reproducible by thoroughly describing them in Sec- tion and the Supplement. Additionally, the Supplementary Materials include the code used to perform the experiments, along with README. md file providing further instructions.
ACKNOWLEDGEMENTS
The work of M. Pach, K. Lewandowska and B. Zieliński work was funded by National Centre of Science (Poland) grant no. 2022/47 /t//B//SS/////// The work of D. Rymar- czyk was funded by National Centre of Science (Poland) grant no. 202//4/NN/STTT ////5////////14// The work of J. Tabor was funded by National Centre of Science (Poland) grant no. BBST6/01137.
We gratefully acknowledge Polish high-performance computing infrastructure PLGrid (HPC Centers: ACK Cyfronet. AGH) for providing computer facilities and support within com- putational grant no. (CG/2023/016556 purchased with funds from the Priority Research Area (Artificial Intelligence Computing Center Core Facility) under the Strategic Programme Excellence Initiative at Jagiellonian University.
Some experiments were performed
We are grateful to Jakub Pach and Tomasz Pach for their assistance in composing images for the survey according to our developed template.
REFERENCES
Michael Anis Mihdi Afnan, Yanhe Liu, Vincent Conitzer, Cynthia Rudin, Abhishek Mishra, Julian Savulescu, and Masoud Afnan. Interpretable, not black-box, artificial intelligence should be used for embryo selection. Human Reproduction Open, 2021.
David Alvarez Melis and Tommi Jaakkola. Towards robust interpretabilityy with self- explaining neural networks. In S. Bengio, Н. Wallach, Н. Larochelle, K. Grauman, N. Cesa-Bianchi, and R. Garnett (eds.), Advances in Neural Information Processing Sys- tems, volume 31. Curran Associates, Inc., 2018.
Laleh Armi and Shervan Fekri-Ershad. Texture image analysis and texture classification methods-a review. arXiv preprint arXiv:194..00 2019.
Alina Jade Barnett, Fides Regina Schwartz, Chaofan Tao, Chaofan Chen, Yinhao Ren, Joseph Y Lo, and Cynthia Rudin. A case-based interpretable deep learning model for classification of mass lesions in digital mammography. Nature Machine Intelligence, 3 (12):1061-1070, 2021.
Dominika Basaj, Witold Oleszkiewicz, Igor Sieradzki, Michal Górszczak, B Rychalska, T Trzcinski, and B Zielinski. Explaining self-supervised image representations with visual probing. In International Joint Conference on Artificial Intelligence, 2021.
David Berga, Marc Masana, and Joost Van de Weijer. Disentanglemmntt of color and shape representations for continual learning. arXiv preprinl urXiv:2007.06333 2020.
Moritz Böhle, Mario Fritz, and Bernt Schiele. B-cos networks: Alignment is all we need for interpretabilivy. In Proceedings of the IEEE/CV Conference on Computer Vision and Pallern Recognition, pp. 10329-10338, 2022.
Wieland Brendel and Matthias Bethge. Approximating CNNs with bag-of-local-features models works surprisingly well imagennt. In International Conference on Learning Representations, 2019. URL htttps://ppereeiies
Zachariah Carmichael, Suhas Lohit, Anoop Cherian, Michael Jones, and Walter J Scheirer. exxel-grounded prototypical part networks. In Proceedings of the IEEE/CVF Winter Conference Applications of Computer Vision, pp. 4768 4478,,77, 2024.
11

---

Published as a conference paper at ICLR 2025
Chaofan Chen, Oscar Li, Daniel Tao, Alina Barnett, Cynthia Rudin, and Jonathan K Su. This looks like that: deep learning for interpretable image recognition. Advonces in newral information processing systems, 32, 2019.
Wei-Ta Chen, Wei-Chuan Liu, and Ming-Syan Chen. Adaptive color feature extraction based on image color distributions. IEEE Transactions on image processing, 19(8):2005-2016, 2010.
Jon Donnelly, Alina Jade Barnett, and Chaofan Chen. Deformable protopnet: An inter- pretable image classifier using deformable prototypes. In Procesdings of the IEEE/CVF conference on computer vision and pattern recognation, pp. (0265-10275, 2022.
Shaojing Fan, Tsong Ng, Bryan Lee Koenig, Jonathan Samuel Herberg, Ming Jiang, Zhiqi Shen, and Qi Zhao. Image visual realism: From human perception to machine computation. IEEE transactions on poltern analysis and machine intelligence, 40(9): 2180-2193 2017.
Srishti Gautam, Marina M-C Höhnc, Stine Hansen, Robert Jenssen, and Michael Kampffmeyer. This looks more like that: Enhancing self-explaining models by proto- typical relevance propagation. Pattern Recogrition, 136:109172, 2023.
Alan H Gee, Diego Garcia-Qlano, Joydeep Ghosh, and David Paydarfar. Explaining deep classification of time-scries data with learned prototypes. In CEUR workshop proceedings, volume 2429, pp. 15. NIH Public Access, 2019.
Robert Geirhos, Patricia Rubisch, Claudio Michaelis, Matthias Bethge, Felix A Wichmann, and Wicland Brendel. Imagenct-trained cmns are biased towards texture; increasing shape bias improves accuracy and robustness. 7th International Conference on Learning Repre- sentations, ICLR, 2019.
Robert M Haralick, Karthikeyan Shanmugam, and Its' Hak Dinstein. Textural features for image classification. IEEE Transactions systems, man, and cybernetics, (6):610-621, 1973.
Peter Hase, Chaofan Chen, Oscar Li, and Cynthia Rudin. Interpretable image recogni- tion with hierarchical prototypes. In Proceedings of the AAAI Conference on Human Computation and Crowdsourcing, volume 7, pp. 32-40, 2019.
Ju He, Shuo Yang, Shaokang Yang, Adam Kortylewski, Xiaoding Yuan, Jie-Neng Chen, Shuai Liu, Cheng Yang, Qihang Yu, and Alan Yuille. Partimagenet: A large, high-quality dataset of parts. In European Conference on Computer Vision, pp. 128 145. Springer, 2022.
Hossein Hosseini, Baicen Xiao, Mayoore Jaiswal, and Radha Poovendran. Assessing shape bias property of convolutional neural networks. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops, pp. 1923 1931, 2018
Qihan Huang, Mengqi Xue, Wenqi Huang, Haofei Zhang, Jie Song, Yongcheng Jing, and Mingli Song. Evaluation and improvement of interpretability for self-explainable part- prototype networks. In Proceedings of the LEEE/CVE International Conference on Com- puter Vision, pp. 2011 10200220002202 2023.
Margot E. Kaminski. The right to explanation, explained. In Research Handbook on Infor- mation Lave and Governance, pp. 278 299. Edward Elgar Publishing, 2021.
Monish Keswani, Sriranjani Ramakrishnan, Nishant Reddy, and Vinceth Balasubrama- nian. Proto2proto: Can you recognize the car, the way do? In Proceedings of the TEEE/CVI Conference on Compuler Vision and Paltern Recognátion, pp. 10233 10243, 2022.
Fahad Shahbaz Khan, Joost Van de Weijer, and Maria Vanrell. Modulating shape features by color attention for object recognition. International Journal of Computer Vision, 98: 49 64, 2012.
12

---

Published as a conference paper at ICLR 2025
Aditya Khosla, Nityananda Jayadevaprakash, Bangpeng Yao, and Fei-Fei Li. Novel dataset for fine-graing aine-grinned image categorization: Stanford dogs. In Proc. CVPR workshop on fine- grained visual categorization (FGVC), volume 2. Citeseer, 2011.
Eunji Kim, Siwon Kim, Minji Seo, and Sungroh Yoon. Xprotonet: Diagnosis in chest radio- graphy with global and local explanations. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognation, pp. 15719-15728, 2021.
Sunnie SY Kim, Nicole Meister, Vikram Ramaswamy, Ruth Fong, and Olga Russakovsky. Hive: Evaluating the human interpretability of visual explanations. In European Confer- ence on Computer Vision, pp. 280-298, 298. Springer, 2022.
Takumi Kobayashi and Nobuyuki Otsu. Color image feature extraction using color index local auto-correlations. In 2009 JEEE International Conference on Acoustics, Speech and Signal Processing, pp. 1057-1060. IEEE, 2009.
Jonathan Krause, Michael Stark, Jia Deng, and Li Fei-Fei. 3d object representations for fine- grained categorization. In Proceedings of the IEEE international conference on computer vision workshops, pp. 554-561, 2013.
Iro Laina, Yuki M Asano, and Andrea Vedaldi. Measuring the interpretability of unsuper- vised representations via quantized reverse probing. ICLR, 2022.
Barbara Landau, Linda B Smith, and Susan S Jones. The importance of shape in early lexical learning. Cognilive developmend, 3(3):299-3)) 321, 1988.
Oscar Li, Hao Liu, Chaofan Chen, and Cynthia Rudin. Deep learning for case-based reason- ing through prototypes: neural network that explains its predictions. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 32, 2018.
Zhuang Liu, Hanzi Mao, Chao- Yuan Wu, Christoph Feichtenhofer. Trevor Darrell, and Saining Xic. A convnet for the 2020s. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp. 11976 11986, 2022.
Ily Loshchilov and Frank Hutter. Decoupled weight decay regularization. arXiv preprint arXiv: 111117111110111 2017.
Chiyu Ma, Brandon Zhao, Chaofan Chen, and Cynthia Rudin. This looks like those: Illu minating prototypical concepts using multiple visualizations. Advances in Neural Infor- mation Processing Systems, 36, 2024.
Yang Mingqiang, Kpalma Kidiyo, Ronsin Joseph, et al. A survey of shape feature extraction techniques. Pattern recognition, 15(7):43 -90, 2008.
Meike Nauta, Annemarie Jutte, Jesper Provoost, and Christin Seifert. This looks like that, because... explaining prototypes for interpretable image recognition. In Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 441-456.
Springer, 2021a.
Meike Nauta, Ron Van Bree, and Christin Seifert. Neural prototype trees for interpretable fine-grained image recognition. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp. 14933-14943, 2021b.
Meike Nauta, Jörg Schlötterer, Maurice van Keulen, and Christin Seifert. Pip-net: Patch- basod intuitive prototypes for interpretable image classification. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 2744-2753, 2023.
Maria-Elena Nilsback and Andrew Zisserman. Automated flower classification over large number of classes. In 2008 Siath Indian conference on computer vision, graphics & image processing, pp. 222-729. IEF., 2008
13

---

Published as a conference paper at ICLR 2025
Adam Paszke, Sam Gross, Francisco Massa, Adam Lerer, James Bradbury, Gregory Chanan, Trevor Killeen, Zeming Lin, Natalia Gimelshein, Luca Antiga, et al. Pytorch: An im- perative style, tigh-peeffrmance deep learning library. Advances in neural information processing systems, 32, 2019.
William K Pratt. Introduction to digital image processing. CRC press, 2013.
Cynthia Rudin. Stop explaining black box machine learning models for high stakes decisions and use interpretable models instead. Nature machine intelligence, 165):206-216 2019.
Cynthia Rudin, Chaofan Chen, Zhi Chen, Haiyang Huang, Lesia Semenova, and Chudi Zhong. Interpretable machine learning: Fundamental principles and 10 grand challenges. Statistic Surveys, 16:1 2022.
Dawid Rymarczyk, Lukasz Struski, Jacek Tabor, and Bartosz Zielińiski. Protopshare: Pro- totypical parts sharing for similarity discovery in interpretable image classification. In Proceedings of the 27th ACM SIGKDD Conference on Knowledge Discovery & Dala Min- ing, pp. 1420- 100-14300 1430, 2021.
Dawid Rymarczyk, Adriana Borowa, Anna Bracha, Maurycy Chronowski, Wojeiech Ozimek, and Bartosz Zieliński. Comparison of supervised and self-supervised deep representations trained on histological images. In MEDINFO 2021: One World, One Health Global Partnership for Digital Innovation, pp. 1052-10000 1053. IOS Press, 2022a.
Dawid Rymarczyk, Adam Pardyl, Jaroslaw Kraus, Aneta Kaczyúska, Marek Skomorowski, and Bartosz Zielinski. Protomil: multiple instance learning with prototypical parts for whole-slide image classification. In Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 421-436. Springer, 2022b.
Dawid Rymarczyk, Lukasz Struski, Michat Górszczak, Koryna Lewandowska, Jacek Tabor, and Bartosz Zicliński. Interpretable image classification with differentiable prototypes assignment. In Proceedings of the European Conference on Computer Vision (ECCV), 2022c.
Dawid Rymarczyk, Lukasz Struski, Michal Górszezak, Koryna Lewandowska, Jacek Tabor, and Bartosz Zicliński. Interpretable image classification with differentiable prototypes assignment. In European Conference on Computer Vision, pp. 351 368. Springer, 2022d.
Dawid Rymarezyk, Daniel Dobrowolski, and Tomasz Danel. Progrest: Prototypical graph regression soft trees for molecular property prediction. STAM International Conference on Data Mining, 2023a.
Dawid Rymarczyk, Joost van Weijer, Bartosz Zielinski, and Bartlomiej Twardowski. Icicle: Interpretable class incremental contimual learning. In Proceedings of the LEEE/CVE International Conference on Computer Vision, pp. 1887-1898, 2023b.
Mikolaj Sacha, Bartosz Jura, Dawid Rymarezyk, Lukasz Struski, Jacek Tabor, and Bartosz Zielińisk.. Interpretability benchmark for evaluating spatial misalignment of prototypical parts explanations. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 38, pp. 21563 21573, 2024.
Mikolaj Sacha, Dawid Rymarczyk, Lukasz Struski, Jacek Tabor, and Bartosz Zieliński. Pro- toseg: Interpretable semantic segmentation with prototypical parts. In Winter Conference on Applications of Compuler Vision (WACV), 2023.
Jost Tobias Springenberg, Alexcy Dosovitskiy, Thomas Brox, and Martin Riedmiller. Striy- ing for simplicity: The all convolutional not. In International Conference on Learning Represendations (ICLR), 2014.
Elham Tabassi. Artificial intelligence risk management framework (ai rmf 1.0). 2023.
Yuki Ukai, Tsubasa Hirakawa, Takayoshi Yamashita, and Hironobu Fujiyoshi. This looks like it rather than that: Protoknn for similarity-based classifiers. In The Eleventh International Conference Learning Representations, 2022.
14

---

Published as a conference paper at ICLR 2025
Robert van der Klis, Stephan Alaniz, Massimiliano Mancini, Cassio F Dantas, Dino lenco, Zeynep Akata, and Diego Marcos. Pdisconet: Semantically consistent part discovery for fine- grained recognition. In Proceedings of the IEEE/CVE International Conference Computer Vision, pp. 1866-1876, 2023.
Laurens Van der Maaten and GeofTrey Hinton. Visualizing data using t-sne. Journal of machine learning research, 9(11), 2008.
Catherine Wah, Steve Branson, Peter Welinder, Pictro Perona, and Serge Belongic. The caltech-ucsd birds-200-201 dataset. 2011.
Qiyang Wan, Ruiping Wang, and Xilin Chen. Interpretable object recognition by semantic prototype analysis. In Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 800-809, 2024.
Jiaqi Wang et al. Interpretable image recognition by constructing transparent embedding space. In Proceedings of the IEEE/CVI /VF International Conference on Computer Vision, pp. 895-904, 2021.
Bichen Wu, Forrest landola, Peter H Jin, and Kurt Keutzer. Squeezedet: Unified, small, low power fully convolutional neural networks for real-time object detection for autonomous driving. In Proceedings of the IEEE conference on computer vision and pattern recognition workshops, pp. 129-137, 2017.
Nan Wu, Stanislaw Jastrzebski, Kyunghyun Cho, and Krzysatof J Geras. Characterizing and overcoming the greedy nature of learning in multi-modal deep neural networks. In International Conference on Machine Learning, pp. 24043 24055. PMLR, 2022.
Romain Xu-Darme, Georges Quénot, Zakaria Chihani, and Marie- Cerie-Christtne Rousset. Sanity checks for patch visualisation in prototype-based image classification. Tn Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3690 3695, 2023.
MD Zeiler. Visualizing and understanding convolutional networks. In European conference on computer vision/au Xiv, volume 1311, 2014.
Zaixi Zhang, Qi Liu, Hao Wang, Chengqiang Lu, and Cheekong Lee. Protgnn: Towards self-explaining graph neural networks. 2022.
Bartosz Zieliáski and Michal Górszczak. What pushes self-supervised image representations away? In International Conference on Newral Information Processing, pp. 514 521. Springer, 2021.
15

---

Published as a conference paper at ICLR 2025
SUPPLEMENT FOR LUCIDPPN: UNAMBIGUOUS PROTOTYPICAL PARTS NETWORK FOR USER-CENTRIC INTERPRETABLE COMPUTER VISION
MORE DETAILS ON DATA PREPROCESSING
In training, we apply transformations as follows: Resize(size=22449) TAWideNoColor () MandontoorzzonaaPPiip RandomBesizeddroppeiiee (222, 224), scale= (0.95, .5, 1.), where TAWideNioColor() is the same variation of TrivialAugment augmentation as in PIP-Net. Additionally, the image entering the ShapeTezzVet is normalized with Normalize (mean=0.444) std=0. 269) after being converted to grayscale. At test time and when finding the prototypical parts patches, we only apply Resize (size=224) followed by grayscaling and normalization in case of ShapeTezzNe input. The CUB images used for training and evaluation are first cropped the bounding boxes similarly other PP-based methods.
We do not modify any parameters in PDiscoNet. CUB setlings are used for datasets not trained in the PDiscoNet paper. For efficiency, we generated and saved the segmentation masks to avoid inferencing PDiscoNet during LucidPPN training.
MORE DETAILS ON EXPERIMENTAL SETUP
The networks (Shape TerNet and ColorNet) are optimized together in minibatches of size 64 for 40 epochs using AdamW [Loshchilov & Hutter 2017) optimizer with beta values of 0.9 and 0.999, epsilon of 10-8 and weight decay of O. The learning rate of Shape TenNet parameters is initialized to 0.002 and lowered to 0.0002 after 15 epochs. The learning rate of the ColorNet is fixed at 0.002. We freeze the weights of ShapeTerNet backbone for the first 15 epochs warm-up stage similar other PPs-based approaches (Chen al. 2019 Nauta et al. 2023 Rymarczyk et al. 2022c).
MORE DETAILS ON COMPUTING RESOURCES
We ran our experiments on an internal cluster and a local cloud provider, a single GPU, it was either NVIDIA A100 40GB NVIDIA H100 80GB. The node we ran the experiments on has 40GB of RAM and an &-core CPU. The model on average trains for hours.
MORE DETAILS ON USER STUDIES WITH EXEMPLARY SURVEYS.
Each worker answering short 10-question survey was paid 1.50 euros. Questions between users may differ as they are randomly composed. Participants are gender- balanced and have ages from 18 to 60.
User study on quality of prototypical parts. For PIP-Net, we randomly select sam- ples with K' = 4,3,2, in the proportion of 3: 2: based the frequency of occurrence as PIP-Net doesn't 't have the same number of prototypical parts assigned to data classes. The LucidPPN pieces of evidence for classes in the same samples always show four prototypical parts as we use model trained with K = 4 here.
Example surveys for LucidPPN, PIP-Net, and single branch are presented in Figures 25 to 37, 38 to 50, and 51 to 63, respectively.
User study on the importance of disentangled visual features. Because we focus on the influence of the color features, we use visualizations with a random single object and prototypical part to let the user focus on the influence of the color. When gathering samples for the survey, we make sure that for nearly half of them color was important for the correct prediction, and for half of them, it was not. We define that the color was important, when LucidPPN was correct, but ShapeTexNel was wrong. And, we define that color was unimportant both outputs were correct.
Example surveys for LucidPPN with color feature scores and without them are presented in Figures 64 to 76, and 77 to 89, respectively.
16

---

Published as a conference paper at ICLR 2025
DETAILED RESULTS OF THE USER STUDY
In Tables and we present detailed results of the user studics. We also visualize the results of the user study on the importance of scores in the Figure
Table 5: User study results indicate that users based LucidPPN explanations outperform those with explanations from PIP-Net to a statistically significant degree.
Mean Acc. [%]
p-value PIP-Net
Std.
random
LucidPPN
PIP-Net
60.0 18.1
0..02
LucidPPN
67.9 16.9
2.13 10
0.014
single branch
69.9 12.8
1.11 10-*
0.008
0.299
Table Details of the user study about assesing the importance of color.
Mean Acc. [%]
p-value without resemblances
- Std.
random
without resemblances with resemblances (TaucidPPN)
49.50 + 11.3
0.577
60.87 + 19.9
0.012
0.016
100
p-value 0.005
Accuracy [9%]
75
Random
Witbout maunbbance
With reambl Latter LaciPPeN
Figure 7: User study shows that disentangled resemblance scores enable users to better understand the relevance of color model's decisions.
COMPARISON EXPLANATION EXAMPLE
We show how our model can generate explanations by comparison of two potential classes in Figure g
COLOR REPRESENTATION
We have performed ablation study to evaluate how different color representations of the imput Ie inffucnce the model's results. Instcad directly downsizing the RGB image, we first transformed it into HSV space, replaced the S and V values with the Hue value, and then downsized the image. In other words, the input to the net was an image composed of the Hue channel repeated three times. The results demoonstraee that LucidPPN with this imput still outperforms ShapeTexNet, with performance similar to the basic LucidPPN as presented in the Table 7
COLORNET SIZE
Since the architecture of ColorNet may significantly impact LucidPPN's performance, we conducted ablation study the architecture's size. Table 8 presents the accuracy com- parison for CUB, All layers are 1 x convolutions followed by ReLU, except the last layer, which is followed by a sigmoid activation. The results indicate that using at least two layers
17

---

Published as a conference paper at ICLR 2025
Evidence for Green Violetear
Green Violetears seen in training
prototypes of Green Violeter
00
0.50
Classified image
0.01 Vs 2.53
Final scores
0.01
0:01
Bay -be-steeaddd Warblers seen in training
Part prototypes Bay- breasted Warbler
0.1
0.1
Bay-breadn
Bay-breasted Warbler
E Seideeen for
Figure 8: Comparison explanation example. Best viewed in landscape orientation.
to introduce non-linge con-linearity is beneficial. Additional layers have smaller impact but can be added ensure sufficient expressiveness of the network.
QUALITATIVE EXAMPLES OF FAILURE CASES WITHOUT DISENTANGLEMENT THAT ARE IMPROVED THROUGH LUCIDPPN
The main goal of the disentanglement is not to improve the accuracy but to provide better understanding of the model's reasoning based on color and shape with texture information. The explanations containing Shape TenNet, ColorNel, and aggregated resemblances intro-
18

---

Published as a conference paper at ICLR 2025
Table 7: Accuracy of LucidPPN when ColorNet receives RGB values vs. only hue value. Shape TexNet added for comparison.
ColorNet input
CUB
CARS
DOGS
FLOWER
SeapeTezNel
80.1 1 0.2 91.7 + 0.1 79.0 0.3 93.6 1 0.3
RGB Only Hue
81.5 1 0.4
91.6 + 0.2
7.44
95.0 0.3 94.1 1 0.5
LucidPPN
81.1 + 0.4
91.6 = 0.2
79.5
Table 8: LucidPPN's accuracy for CUB vs the size of ColorNet.
Number of layers
Hidden dimennsoon
Accuracy [%]]
1
80.3
2
(600) (50, 200, 600)
81.4
4
81.2
6 (20, 50, 150, 200, 600)
81.5
duced in this work offer this additional information. Such an insight was missing in the previous prototypical parts models (Chen et al. 2019 Nauta et al. 2023).
Nevertheless, disentanglement can enhance accuracy in scenarios where shape and texture are the primary decision factors, with color serving to refine decisions that are difficult to make based on other features alone. Figure 9 illustrates examples from CARS and CUB where LucidPPN adheres to this principle, whereas the single branch CNN does not. Table 9 shows how often single branch CNN with colored input misclassifies color-altered images compared to the LucidPPN, and vice versa, for two specific data classes scenarios:
1. For test images of the typically red Lamborghini Aventador, which were converted to green and yellow via hue rotation, the single branch CNN mistakenly classified these altered images either the typically green Lamborghini Gallardo the usually yellow Lamborghini Diablo 24 times, despite the noticeable differences in shape (e.g., headlights and bumpers). In contrast, LucidPPN correctly classified these altered images. LucidPPN only made such a mistake 3 times, while the single branch CNN did not.
2. For test images of the red Cardinal, which were similarly converted to yellow and indigo, the single branch CNN misclassified the Cardinal as any other bird 34 times. LucidPPN made this mistake only once, while the single branch CNN was correct in all other instances.
Table 9: Number found examples for which LucidPPN and single branch outperformed each other when asked to predict class in the color altered images.
Lamborghini Avendator
Candinal
LucidPPN correct but single brunch wrong
24
34
LucidPPN wrong but single branch correct
PROTOTYPICAL PARTS EXAMPLES TRAINED WITHOUT PART SUPERVISION
are presented in Figure 10
19

---

Published as a conference paper at ICLR 2025
Wrong class guessed by single branch
Original image
Altered images
Predictions
single branch:
Lamborghini
Gallardo
LucidPPN:
Lamborghini
Aventador
single branch:
Lamborghini
Diablo
Correct label: Lamborghöni
LucidPPN:
Lamborghini
Aventador
Aventador
single brunch:
Blue-winged
Warbler
LucidPPN:
Cardinal
single branch:
Indigo
Bunting
Correct label: Cardinal
LucidPPN: Cardinal
Figure 9: Examples of images with altered colors that change the prediction of a single branch CNN include Lamborghini Aventador (top) and Cardinal (bottom). Both are incorrectly classified by the single branch CNN, while LucidPPN with color disentangling classifies them correctly.
NUMBER OF PARTS
In Figure I1 we show the impact of choosing a different number of parts K. Lu- cidPPN achives high results for all tested K, however it noticcable that incrcasing K improves classification. Especially on CARS, our method seems to strongly benefit from choosing K 2 4. The reasonably high scores for all K allow for a choice between sparse explanations and higher accuracy.
NEED FOR Ls
Many prototypical-partssbasdd models, such ProtoPNet (Chen et al. 2019). Pro- toPool [Rymarczyk et al.] 2022d), and PIP-Net (Nauta et al., 2023). involve complex training schemes with warm-up and pretraining phases. Initially, believed that Shape- TerNet should be pretedined before training ColorNet, given that Shape TexNel processes more complex data. However, the ablation study presented Figure shows ShapeTezzee (or delaying the training of ColorNet) is either unnecessary or may even negatively influence color-based explanations.
showw that warming
During the initial development of LucidPPN, we used Ls to guide the learning of Shape- TexNet during its warm-up phase. Once we observed that Shape did not require a warm-up, we switched to jointly using Ls and LA in training. We found that remov- ing Ls negatively impacted LucidPPN's performance which is presented in Figure 6 in the manuscript. Consequently, we retained Ls, as it provides essential guidance for ShapeTexNet to effectively extract important features from its complex input.
20

---

Published as a conference paper at ICLR 2025
With PDiscoNet supervision
Without PDiscoNet supervision
Yellow- headed Blackbird
Blue Jay
Figure 10: Examples from two classes demonstrate that prototypes learned without PDis- coNet supervision focus on single object part, in contrast. to those more diverse learned by LucidPPN.
FLOWER
CARS
DOGS
100
Apcunacy [36]
90
85
75
12
Figure 11: Influence of the number of object parts K on LucidPPN accuracy. Increasing the number of parts improves the accuracy of the model. Note that each dataset is plotted in a unique color.
This need for stronger guidance aligns with observations in multimodal learning, where separate learning of different modality branches maximizes the information extracted from each modality (Wu et al. 2022). Here, we can think of each branch as different modalities. Alternatively, using weighted average could yield similar accuracy, but it would complicate the final prediction. This approach would necessitate analyzing the contribution of each logit vector separately and understanding their aggrogation, with potentially different weights for cach dataset, making the process less transparent for the user.
START OF COLOR NETWORK TRAINING
natural to ask whether delaying the start of ColorNet optimization could improve LucidPPN. In Figure we report the accuracy and color sparsity after delaying the training of ColorNet. The change in classification quality is negligible. However, we observe a drop in color sparsity, indicating that ColorNet is less focused relevant colors. It is important to note that despite the delay, the number training epochs for ColorNet remains constant for comparability.
21

---

backbones have an advantage over ImageNet Published as a conference paper at ICLR 2025
Color spursity [%)
Accuracy
50-
10
20
30
Delay [spochs]
Figure 12: Influence of a delay when ColorNet starts train on LucidPPN 's accuracy and color sparsity. While this delay does not negatively affect accuracy, it results in lower color sparsity. This means that the network is concentrating on single color when processing the PP.
LUCIPPPN WITH DIFFERENT BACKBONES
We evaluate LucidPPN on an additional ResNet50 backbone and compare the results to other models and backbones in Table 10 LucidPPN with ResNet50 backbone performs worse than the one with ConvNeXt-ting, which similar PIP-Net. When comparing the results, note that resulting in a few points higher accuracy.
Table 10:
Accuracy for different PP-based methods and backbones. The asterisk means that the used backbone was pretrained on the iNaturalist dataset instead of ImageNet.
CUB
CARS
DOGS
FLOWER
ProtoPNet ProtoPShare
ResNet34
79.2
86.1
ResNet34
74.7
86.4
Proto
ResNet50
82.2 + 0.7*
86.6 0.2
ProtoPool
ResNet50
85.5 + 0.1"
88.9
ResNet50
82.0 0.3"
0.3
PIP-Net
ConvNNeXt-tiny
84.3 0.2
88.2 -: 0.5
80.8 + 0.4
91.8 1 0.5
ResNet50
75.5
89.0 0.3
70.8 0.2
89.5 0.4
LucidPPN
ConvNeXt--inny
$1.500.4
91.6 + 0.2
79.4 0.4
95.0 0.3
DISCUSSION ON PATCHES OF THE COLOR INFO AND THE PATCH OF GRAY-SCALED INPUT ALIGNED THE LATENT SPACE
Using a convolutional backbone, we assume a spatial correspondence between the latent map from Shape and the input, similar the approach in ProtoPNet |Chen al, 2019). As we downsize the colorful image to match the height and width of the activation map, the input dimensions for ColorNet are maintained consistently. ColorNet employs 1 convolutions to encode color information, ensuring the latent map has the same dimensions as both the downsized input and the Shape TexNet activation map.
Given the use of x convolutions on the downsized image and a convolutional backbone for the full-rcsolution image, to the (i.j) position on the other. Finally, we extract color information from the latent representation at the same location where the prototypical part is most active, ensuring alignment between the color and shape features.
assume that the (i.j) position
map corresponds
LOCAL INTERPRETNTION OF SINGLE BRANCH
An example of prediction interpretation of single branch is presented in Figure 13
22

---

Published as a conference paper at ICLR 2025
Evidence for Lazuli Bunting
Part prototypes
I score
0.90
1.00
0.86
0.81
0.72
Figure 13: An example local interpretation of single brunch for Lazuli Bunting.
Object
Grayscale Input
part
Object purt 2
SetpeTexNNtt
PDiscoNet frines
Class A
Class B
feature map /s
segmentation masks T
Frozen weights
Loss
Figure 14: The image illustrates the object-part correspondence loss, which is applied solely to the outputs Shape TexNel and PDiscoNet. First, we identify object parts through PDiscoNet (e.g,, first on the head, second the wing). Next, align the corresponding prototypical parts with the object parts identified by the segmentation results through LD loss.
OBJECT-PART CORRESPONDENCE.
On Figure 14 we present how object-part correspondence LD loss works.
CONSISTENCY AND STABILITY OF PROTOTYPICAL PARTS
One way evaluate the quality of prototypical parts is to measure their consistency and stability Huang et al. (023)) In Table 11 present the results of those metrics. The results show that LucidPPN achieves state-of-the-art results on stability without any ad- ditional loss components, and comparable to other metrics when it to stability. This improvement, likely due to the enhanced object-part correspondence enabled by its prototypical parts.
COMPUTATIONAL COSTS
In Table 12 we provide computational costs training LucidPPN when compared to other prototypica-ppatts parts-basetdd architectures.
GENERALIZATION TO NOT FINE-GRAINED DATASET
To assess whether LucidPPN generalizes to broader classification tasks (beyond fne-grained datasets), we present results on PartImageNet He et al. (2022). On this dataset, LucidPPN achieves an accuracy of 84.1%, outperforming PIPNet, which achieves 82.8%.
COMPARISON OF EXPLANATION VISUALIZATIONS
In Figures 15
16
and
19 we compare the decision explanations generated by
different. me
23

---

Published as a conference paper at ICLR 2025
Table 11: Results of LucidPPN on consistency and stability metrics from the work of Hunng et all (023). The results indicate that LucidPPN is more robust than other prototypical- parts- based approaches and achieves state-of-the-art results for Consistency while still ro- maining competitive in Stability.
Method
Consistency Stability
ProtoPNet
28.3
56.7
ProtoTrre
16.4
23.2
ProtoPool
35.7
58.4
TesNet
48.6
60.0
Deformable ProtoPNet
44.2
53.5
Huang et al. (2023)
70.6
72.1
LucidPPN (our)
71.2
66.3
Table 12: Computational costs of prololypical--parts--aaee methods. One can observe that training of LucidPPN requires fewer hours and less RAM memory than PIP-Net, but more GFLOPs. Generally, LucidPPN and PIPNet require more RAM memory than ProtoPNet and ProtoPool, however they converge faster.
Method
Training time GFLOPs for batch of data Avg. Training Memory Usage
ProtoPNet
3h
586
4.9GB
ProtoPool
18h
658
14.4GB
PIP-Net
3h
354
41.5GB
LucidPPN (our)
2h
475
22.9GB
FAITHFULLNESS OF PATCH VISUALIZATIONS
LucidPPN introduces a key difference in the definition of prototypical parts compared to PIPNet. While PIPNet employs Softmax across channels in the latent feature map, Lu- cidPPN uses the sigmoid activation function. The sigmoid function allows each channel's activation to be learned independently, not influenced by the relative activations channels. At the same time, Softmax normalization can distort activations by emphasizing values that are only relatively high compared to others, even if they are low in absolute terms. Therefore, using the sigmoid function instead of Softmax, one the image patches selected for visualization are faithful because such patches should have resemblance score close 1. In Figure 200 we provide a distribution of the sigmoid function values obtained for patches used in prototype visualization. For LucidPPN trained on the CUB dataset (blue curve), 61.04% of those patches have values above 0.9, which indicates that prototype visualizations are relatively faithful. Moreover, higher faithfulness can be obtained when training with an additional loss component Lş that punishes the model the sigmoid function value for a given prototype is smaller than for all samples in the batch:
other
easily verify if
L.c KM k=1m=1
Σ
max km A,b)
DEB
where B is the number of samples in batch and km is the value of km for sample b in the batch. As we observe in Figure 20 (green and yellow curves), the distribution of sigmoid function values moves right, with increasing weight og of Lc- However, it also comes with small decrease in accuracy.
PRUNING PROTOTYPES LESS FAFTHFUL VISUALIZATIONS
To increase the faithfulness LucidPPN, we analyze the effects pruning the prototypes
with less faithful representation (those with resemblance scores 0.9). As shown in Ta- ble 13 LucidPPN accuracy after pruning drops only by around 2% (from 81.6% to 79.3%). However, interestingly, the accuracy stays the same for Lc 0.05. It suggests that combi-
24

---

Published as a conference paper at ICLR 2025
nation of using loss Le and applying the pruning allows to enforce high resemblance scores (>0.9) of visualized patches without sacrificing on the accuracy.
REASON BEHIND USING THE BINARY CROSS ENTROPY WITH SIGMOID INSTEAD THE CROSS ENTROPY WITH SOFTMAX
The intuition behind Binary Cross Entropy (BCE) usage is rooted from multilabel classifi- cation. To some degree, ShapeTexNet operates in a multilabel setting from the prototypical parts perspective, they may match multiple classes. Hence, to enable multiple classes having high similarity to the same prototypical parts, we use sigmoid instead of softmax when computing the feature maps. This necessitates shift from pross-Entropy (CE) to Binary Cross-Entropy (BCE) because CE would then solely maximize the activation of the correct class while ignoring crucial signals from negative classes. Another reason behind our choice is to make it easier to verify the faithfulness of visualizations, as the sigmoid function allows each channel's activation be learned independently, not influenced by the relative activations of other channels. While, Softmax normalization used with CE can distort acti- vations by emphasizing values that are only relatively high compared others, even they are low in absolute terins.
Lazuli Bunting
Part prototypes Lazult Bunting
score
0.9
LO
0.90
0.86
0.9
0.9
0,81
0.8
&
0.9
0.72
Figure 15: Local interpretation visualization in LucidPPN
Learned Port Protolypes
Closs weight Weight similarity
1.m=0.9
8.5"0.9 7.7
sim-10
4.5
4.5'1.0 4.5
sim=08
+.2
3.2'0.8 2.6
Prediction score for class Lazuli Bunting
14.8
Figure 16: Local interpretation visualization in PIP-Net
NVO DYStin ALTUER
Text Image
Figure 17: Local interpretation visualization in ProtoTTree
GLOBAL CHARACTERISTICS EXAMPLES
We present global characteristics for different datasets in Figures 21
22
25

---

Published as a conference paper at ICLR 2025
Why is that Ford Preestar Minivan 2007?
Image
Prototypical part
Activation map Similarity and weigh
4.08 + 0.89 3.63
3.87 + 0.83 3.21
SUM: 21.64
Figure 18: Local interpretation visualization ProtoPool
Why classfied red-bellied woodpecker!
being red-hellied woodpocker:
Original image
Protatype
Training image where prototypo comes from
Activation map
Similarity Class
Points
(box showing part
looks like prototype)
6.499 1.180 7.669
4.392 1.127 = 4.950
3.890 1.108 4.310
Total points to red-bellied woodpecker: 32,736
Figure 19: Local interpretation visualization in ProtoPNet
26

---

Published as a conference paper at ICLR 2025
Percentage of prototypes
ac 0 (Acc: 81.6)
80
ac = 0.05 (Acc: 81.6)
ac 0.1 (Acc: 79.2)
60
40
20
0
0.50
0.55
0.60
0.65
0.70
0.75
0.80
0.85
0.90
0.95
1.00
Aggregated resemblance score
Figure 20: Distribution of the aggregated resemblance scores for different. weights of cluster loss Lc- For LucidPPN (ac = 0) trained on CUB dataset, 61.04% of patches representing prototypes have aggregated resemblance score above 0.9, which indicates that prototype visualizations are relatively faithful. Moreover, when training with additional loss function Lc, we obtain 95.33% patches with values above 0.9, with only small drop accuracy.
Table 13: Accuracy before and after pruning the prototypes with less faithful visualizations. The results show that the combination of training with loss Le and pruning can enforce faithfullness of visualizations without the loss in performance.
ac
Before pruning After pruning
81.6
79.3
0.05
81.6
81.6
0.1
79.2
79.2
27

---

Published as a conference paper at ICLR 2025
Blackberry Lily
& & & &
Ostcosper- mum
Black-eyed
Susan
Wild Pansy
&
Figure 21: An example showcasing global characteristics of four classes the FLOWER dataset, using prototypical parts from LucidPPN trained with K = 2. This visualization demonstrates the ability to detect differences between data classes. For instance, the os- teospermum and black-eyed susan exhibit more variation in color, while the blackberry lilly and wild pansay classes differ in texture and shapc.
28

---

Published as a conference paper at ICLR 2025
&
&
&
&
&
&
&
&
&
&
2
&
&
&
&
&
& &
Figure 22: Selected global characteristies for LucidPPN trained on CUB with 4
29

---

Published as a conference paper at ICLR 2025
&
&
&
&
&
& & &
&
&
&
&
&
&
&
&
&
&
Figure 23: Selected global characteristics for LucidPPN trained on CARS with K = 4
30

---

Published as a conference paper at ICLR 2025
&
3
&
&
&
&
&
&
&
&
&
&
&
&
& & &
& & &
&
& &
Figure 24: Selected global characteristics for LucidPPN trained on DOGS with K 4
31

---

Published as a conference paper at ICLR 2025
&
&
Figure 25: Selected global characteristics for LucidPPN trained on FLOWER with K = 2
32

---

Published as a conference paper at ICLR 2025
Given bird photo model predicts based protstype has learned from previcssly photos Specifially each protetype, model identifies photo that prototype similarity Greater determines dissification bird
given photo
species by
species you is predicted
Random guessing will you 50% accuracy. You will receive a reward only your per formancce is reasonably beyond the random chance.
Interface explanation
An image being classified by the model
Each row shows prototype of bird part composed of two components: shape with texture
Prototypes' location
color
Evidence for species A
Part protecypes specieer d
Evidence species
Part protetypes of species
Figure 26: Page of survey for LucidPPN
33

---

Published as a conference paper at ICLR 2025
Example
Evidence for species A
Part prototypes of specier A
&
&
&
Evidence species B
Part prototypes of specier &
&
&
&
Rased on glven evidenos, the model would the photo shows...
Correct
Explonation: Evidence for species is more similrr the image
species
Next
Figure 27: Page 2 of survey for LucidPPN
34

---

Published as a conference paper at ICLR 2025
Example
Evidence for species A
Part prototypes of specier A
&
&
&
Evidence species B
Part prototypes of specier &
&
&
&
Rased glven evidenos, the model would the photo shows...
Correct
Explanation: Evidence for species is more simitrr the image
species
Next
Figure 28: Page 3 of survey for LucidPPN
35

---

Published as a conference paper at ICLR 2025
Question
Evidence for species A
Part prototypes of species A
&
Evidence for species
Part prototypes of species B
Rosed on given evidence, the model would say the photo shows...
species
Figure 29: Page 4 of survey for LucidPPN
36

---

Published as a conference paper at ICLR 2025
Question
Evidence for species A
Part prototypes of species A
&
Evidence for species
Part prototypes of species B
&
Rosed on given evidence, the model would say the photo shows...
species
Previous
Figure 30: Page 5 of survey for LucidPPN
37

---

Published as a conference paper at ICLR 2025
Question
Evidence for species A
Part prototypes of species A
&
Evidence for species
Part prototypes of species B
Rosed on given evilence, the model would say the photo shows...
species
Figure 31: Page 6 of survey for LucidPPN
38

---

Published as a conference paper at ICLR 2025
Question
Evidence for species A
Part prototypes of species A
&
Evidence for species
Part prototypes of species B
Rosed on given evidence, the model would say the photo shows...
species
Figure 32: Page 7 of survey for LucidPPN
39

---

Published as a conference paper at ICLR 2025
Question
Evidence for species A
Part prototypes of species A
&
Evidence for species
Part prototypes of species B
Rosed on given evilence, the model would say the photo shows...
species
Figure 33: Page 8 of survey for LucidPPN
40

---

Published as a conference paper at ICLR 2025
Question
Evidence for species A
Part prototypes of species A
&
Evidence for species
Part prototypes of species B
&
Rosed on given evilence, the model would say the photo shows...
species
Figure 34: Page 9 of survey for LucidPPN
41

---

Published as a conference paper at ICLR 2025
Question
Evidence for species A
Part prototypes of species A
&
&
Evidence for species
Part prototypes of species B
Rosed on given evidence, the model would say the photo shows..
species
Figure 35: Page 10 of survey for LucidPPN
42

---

Published as a conference paper at ICLR 2025
Question
Evidence for species A
Part prototypes of species A
&
Evidence for species
Part prototypes of species B
Rosed on given evidence, the model would say the photo shows...
species
Figure 36: Page 11 of survey for LucidPPN
43

---

Published as a conference paper at ICLR 2025
Question
Evidence for species A
Part prototypes of species A
&
Evidence for species
Part prototypes of species B
Rosed on given evidence, the model would say the photo shows...
species
Figure 37: Page 12 of survey for LucidPPN
44

---

Published as a conference paper at ICLR 2025
Question
Evidence for species A
Part prototypes of species A
&
Evidence for species
Part prototypes of species B
&
Rosed on given evilence, the model would say the photo shows...
species
Previous
jobs
Figure 38: Page 13 of survey for LucidPPN
45

---

Published as a conference paper at ICLR 2025
photo species protsoypes provoussy Specificaly prototype ieetiiies region speciti species.
prototype siilaitt. simlenty
For given photo we show evidence bird species found by model. Your task choose species you think is predicted by the model based that evidence.
Random guessing will 50% accuracy. You will receive reward only your performance is reasonably beyond the random chance.
Interface explanation
An image being classified by the model
Each row shows prototype of bird part
Evidence for species A
Prototypes
Evidence for species
Prototypas
Figure 39: Page of survey for PIP-Nct
46

---

Published as a conference paper at ICLR 2025
Example 1.
Evidence for species A
Learned Part Prototypes
Evidence for species B
Learned Part Prototypes
given
apecies
Explonation: Exidence i more similar the
species
Previous
Next
Figure 40: Page of survey for PIP-Net
47

---

Published as a conference paper at ICLR 2025
Example
Evidence for species A
Learned Part Prototypes
Evidence for species B
Learned Part Prototypes
themo
species
Explenation: Exidence specias i more similar
species
Previous
Next
Figure 41: Page of survey for PIP-Net
48

---

Published as a conference paper at ICLR 2025
Question
Evidence for species A
Learned Prototypes
Evidence for species B
Learned Part Prototypes
Suoed on given exidence, the model would say the photo shows...
species
species
Previous
Next
Figure 42: Page of survey for PIP-Net
49

---

Published as a conference paper at ICLR 2025
Question 1,
Evidence for species A
Learned Prototypes
Evidence for species B
Learned Part Prototypes
Scoed on given exidence, the model would say the photo shows...
species
species
Previous
Next
Figure 43: Page of survey for PIP-Net
50

---

Published as a conference paper at ICLR 2025
Question 2.
Evidence for species A
Learned Prototypes
Evidence for species B
Learned Part Prototypes
Sused on given exidence, the model would say the photo shows...
species
species
Previous
Next
Figure 44: Page of survey for PIP-Net
51

---

Published as a conference paper at ICLR 2025
Question 3.
Evidence for species A
Learned Prototypes
Evidence for species B
Learned Part Prototypes
Sused om given evidence, the model would say the photo shows....
species
species
Previous
Next
Figure 45: Page of survey for PIP-Net
52

---

Published as a conference paper at ICLR 2025
Question
Evidence for species A
Learned Prototypes
Evidence for species B
Learned Part Prototypes
Suoed on given exidence, the model would say the photo shows...
species
species
Previous
Next
Figure 46: Page of survey for PIP-Net
53

---

Published as a conference paper at ICLR 2025
Question
Evidence for species A
Learned Prototypes
Evidence for species B
Learned Part Prototypes
Sused on given exidence, the model would say the photo shows...
species
species
Previous
Next
Figure 47: Page 9 of survey for PIP-Net
54

---

Published as a conference paper at ICLR 2025
Question
Evidence for species A
Learned Prototypes
Evidence for species B
Learned Part Prototypes
Suoed on given exidence, the model would say the photo shows....
species
species
Previous
Next
Figure 48: Page 10 of survey for PIP-Net
55

---

Published as a conference paper at ICLR 2025
Question 7.
Evidence for species A
Learned Prototypes
Evidence for species B
Learned Part Prototypes
Scoed on given exidence, the model would say the photo shows....
species
species
Previous
Next
Figure 49: Page 11 of survey for PIP-Net
56

---

Published as a conference paper at ICLR 2025
Question
Evidence for species A
Learned Prototypes
Evidence for species B
Learned Part Prototypes
Scoed on given exidence, the model would say the photo shows....
species
species
Previous
Next
Figure 50: Page 12 of survey for PIP-Net
57

---

Published as a conference paper at ICLR 2025
Question
Evidence for species A
Learned Prototypes
Evidence for species B
Learned Part Prototypes
Scoed on given exidence, the model would say the photo shows...
species
species
Previous
Send job
Figure 51: Page 13 of survey for PIP-Net
58

---

Published as a conference paper at ICLR 2025
Ghen rd model predects
laamed from prestousty photos Spectiially for each the model idensifies dlesilicatio the speciiic
looks similar procutgpe larity,
a given photo we evidence species by Your taak species you predicted evidence.
Random m guessing gee you 50% accuracy. You will rocetie reward only your performance is ressonallyy beyend the random chance.
Interface explanation
An image being classified by the model
Each row shows
Prototypes' location
a
prototype bird
Evidence for species A
Evidence for species B
Put proketypc of speciie
Figure 52: Page 1 of survey for single branch
59

---

Published as a conference paper at ICLR 2025
Example
Evidence for species A
Part prototypes of species A
Evidence for species B
Part prototypes of species B
evilenoe, model
Apecies
Deplonatar:: Disdense species A simtler
spc0es
Prestous
Next
Figure 53: Page 2 of survey for single branch
60

---

Published as a conference paper at ICLR 2025
Example
Evidence for species A
Part prototypes of species A
Evidence for species B
Part prototypes of species B
evklenoe, model
Desimuttan::rddeene species sintle
spc0es
Prestous
Next
Figure 54: Page 3 of survey for single branch
61

---

Published as a conference paper at ICLR 2025
Evidence for species A
Part prototypes of species A
Evidence for species B
Part prototypes species
Busrd on siion ovidana, modd wou sap the plocto cre......
Previoua
Nest
Figure 55: Page 4 of survey for single branch
62

---

Published as a conference paper at ICLR 2025
Evidence for species A
Part protolypes of species A
Evidence for species B
Part prototypes species
Buord on sivon evidana, modd mou sap the placto cret....
Previoua
Nest
Figure 56: Page 5 of survey for single branch
63

---

Published as a conference paper at ICLR 2025
Question
Evidence for species A
Part protelypes of species A
Evidence for species B
protolypes species
Busid on siion ovidana, modd mau sap the plocto cre.....
Previoua
Nest
Figure 57: Page 6 of survey for single branch
64

---

Published as a conference paper at ICLR 2025
Evidence for species A
Part protolypes of species A
Evidence for species B
Part prototypes species
Busrd on siken ovidana, mode mou sep the placto cre......
Previoua
Nest
Figure 58: Page 7 of survey for single branch
65

---

Published as a conference paper at ICLR 2025
Question
Evidence for species A
Part protolypes of species A
Evidence for species B
Part prototypes species
Busid on siknn ovidana, modd wou sap the placto cre.....
Previoua
Nest
Figure 59: Page 8 of survey for single branch
66

---

Published as a conference paper at ICLR 2025
Evidence for species A
Part protelypes of species A
Evidence for species B
protolypes species
Busrd on siven ovidana, modd mou day the placto cree......
Previoua
Nest
Figure 60: Page 9 of survey for single branch
67

---

Published as a conference paper at ICLR 2025
Evidence for species A
Part protolypes of species A
Evidence for species B
prototypes species
Busrd on skinn ovidana, modd mou sap the plocto cret....
Previoua
Nest
Figure 61: Page 10 of survey for single branch
68

---

Published as a conference paper at ICLR 2025
Evidence for species A
Part protolypes of species A
Evidence for species B
Part protolypes species
Buord on siinn ovidaaa, modd mou sap the plocto cret....
Previoua
Nest
Figure 62: Page 11 of survey for single branch
69

---

Published as a conference paper at ICLR 2025
Question
Evidence for species A
Part protolypes of species A
Evidence for species B
Part protolypes species
Buord on siknn ovidara, modd wou sap the plocto cret.....
Previoua
Nest
Figure 63: Page 12 of survey for single branch
70

---

Published as a conference paper at ICLR 2025
Evidence for species A
Part protolypes of species A
Evidence for species B
protolypes species
Buocd on skin oilanaa modd mou sap the placto cret....
Previcus
Senid juls
Figure 64: Page 13 of survey for single branch
71

---

Published as a conference paper at ICLR 2025
Given lerd photo model predicts the species based has learned fram previously seen photos. Speciically for each prototype, the model iderefies region similaiyy. arly desermines dessificatio books similer the prototype olotpesa
Mcdel first
The deriinn tn riecety an image of beinnging g tnia spectti spertes. frilowing b queddoo,, you ure the twn most probubble species present in the image, according to the model. Based the oxplanation prouided by the model bry answor the followngg question: "In your opinion, what the influence of color on the modelts decision process7"(1 (1 None, Wealk, 3 Don't know, 4 Mederate, S Substantial)
explonation
An image being classified by the model
Both rows show a prototype of a bird part composed of two components: shape with texture
color
Evidence for correct species
Pur peoeaypes of conrect specics
0.90
0.90
0.81
Evidende wrong species
Similarity of prototype components to the image, the higher the more similar
peetotypos wrong specior
&
0.20
0.20
0.04
Next
Figure 65: Page of survey for full LucidPPN (with scores)
72

---

Published as a conference paper at ICLR 2025
Exampie
Evidence for correct species
Part prototypes cornect speciess
&
0.90
0.90
0.81
Shape texture similarity image HIGHER for the correct species. Color was NOT needed.
Evidence for wrong species
Part pretorypes wming species
&
0.20
0.20
0.04
your spition, whoris the influeece color the modalss dedidion precesd)
Gorrect
Explanatio: Shape with tedture similarty of ofinage proscypes was enough to predict the connect cass.
None
Don't know
Moderaae
5- Subxutantii
Previous
Next
Figure 66: Page of survey for full LucidPPN (with scores)
73

---

Published as a conference paper at ICLR 2025
Example
Evidence for correct species
Part prototypes cornect species
&
0.15
1.00
0.15
Shape texture similarity image LOWER the correct species. Color WAS needed.
Evidence for wrong species
Part pretorypes wming species
&
0.29
0.01
0.00
your spition, whoris the influence color the modalss dedition precesd)
Sebetantial
Explanatio: Shape with testure similarty of orinage prosoypes would leed to mosaake and was avcided by co corsidering colfee
None
Don't know
Moderaae
5- Subxutantii
Previous
Next
Figure 67: Page of survey for full LucidPPN (with scores)
74

---

Published as a conference paper at ICLR 2025
Question
Evidence for correct species
Port prosotypes of correct species
0.38
0.99
0.38
&
Evidence for wrong species
Part prototypes wrong species
0.42
0.48
0.20
&
whatis influenee rodat's dediton dedatenproennd proeass?
Weak
3- Don't know
Moderrace
Previous
Next
Figure 68: Page of survey for full LucidPPN (with scores)
75

---

Published as a conference paper at ICLR 2025
Question
Evidence for correct species
Port prototypes of correct species
0.98
1.00
0.98
&
Evidence for wrong species
Part prototypes wrong species
0.41
0.35
0.14
&
whatis influence modatss dediton proeass?
Weak
3- Don't know
Moderrae
Previous
Next
Figure 69: Page of survey for full LucidPPN (with scores)
76

---

Published as a conference paper at ICLR 2025
Question
Evidence for correct species
Port prosotypes of correct species
0.89
0.79
0.70
&
Evidence for wrong species
Part prototypes wrong species
0.35
0.82
0.29
&
whatis influence modats dediton dedatenproennt
Weak
3- Don't know
Moderrac
Previous
Next
Figure 70: Page of survey for full LucidPPN (with scores)
77

---

Published as a conference paper at ICLR 2025
Question
Evidence for correct species
Part prosotypes of correct species
&
0.37
1.00
0.37
Evidence for wrong species
Part prototypes wrong species
&
0.82
0.02
0.02
whatis influence modats dediton
Weak
3- Don't know
Moderrace
Previous
Next
Figure 71: Page of survey for full LucidPPN (with scores)
78

---

Published as a conference paper at ICLR 2025
Question
Evidence for correct species
Part peosotypes of cornect species
0.31
0.85
0.26
Evidence for wrong species
Part prototypes wrong species
&
0.62
0.16
0.10
whatis influence modatss dediton dedatenpreenst
Weak
3- Don't know
Moderrace
Previous
Next
Figure 72: Page of survey for full LucidPPN (with scores)
79

---

Published as a conference paper at ICLR 2025
Question
Evidence for correct species
Part peosotypes of correct species
&
0.96
1.00
0.96
Evidence for wrong species
Part prototypes wrong species
&
0.12
1.00
0.12
whatis influenee rodatss dediton dedatenpreenst
Weak
3- Don't know
Moderrae
Previous
Next
Figure 73: Page of survey for full LucidPPN (with scores)
80

---

Published as a conference paper at ICLR 2025
Question
Evidence for correct species
Port prosotypes of correct species
&
0.99
0.99
0.98
Evidence for wrong species
Part prototypes wrong species
&
0.72
0.97
0.70
whatis influence rodatss dediton procsss?
Weak
3- Don't know
Moderrae
Previous
Next
Figure 74: Page 10 of survey for full LucidPPN (with scores)
81

---

Published as a conference paper at ICLR 2025
Question
Evidence for correct species
Port prosotypes of correct species
0.95
0.74
0.70
&
Evidence for wrong species
Part prototypes wrong species
0.29
0.92
0.27
&
whatis influence rodatss dediton dedatenproendd proeass?
N-None
Weak
3- Don't know
Moderrac
Previous
Next
Figure 75: Page 11 of survey for full LucidPPN (with scores)
82

---

Published as a conference paper at ICLR 2025
Question
Evidence for correct species
Port prosotypes of correct species
0.23
0.46
0.10
&
Evidence for wrong species
Part prototypes wrong species
0.18
0.41
0.07
&
whatis influence modatss dediton dedatenpreenst
Weak
3- Don't know
Moderrac
Previous
Next
Figure 76: Page 12 of survey for full LucidPPN (with scores)
83

---

Published as a conference paper at ICLR 2025
Question
Evidence for correct species
Port prosotypes of correct species
0.01
0.99
0.01
&
Evidence for wrong species
Part prototypes wrong species
0.11
0.02
0.00
&
whatis influenee modal's dediton proeass?
Weak
3- Don't know
Moderrae
Previous
Send Job
Figure 77: Page 13 of survey for full LucidPPN (with scores)
84

---

Published as a conference paper at ICLR 2025
prototype prossoyps learned from similntty. color may predictinn. shape texture ay protoopec),
Specifically prototpe. model idenefles
speciic spectes,
The mocil made a decéiion to classity an image of bird as bellnngngg ig to specific aind specias, in the following question,, you will see the two most probuble bird spediss presontio in the imogt, according to the model, Bosod the oxplonation provided the modol answer the following questionc "In your opinion, what the Influence color the modelts decision process7" None, Weakk Don't know, Mederate, Substentiel)
Interi expleratioon
An image being classified by the model
Both rows show a prototype of a bird part composed of two components: shape with texture
color
Evidence for correct species
Port protstppc corvett specier
Evidence wrong species
Port grototypes of wrong specier
Figure 78: Page of survey for LucidPPN with no scores
85

---

Published as a conference paper at ICLR 2025
Example
Evidence for correct species
prosotypes of species
&
Shape with texture similarity to image HIGHER for the correct species. Color was NOT needed.
Evidence for wrong species
Part protolypes wrong species
&
your optiion whoxss the infueene color the mode's prosenpp
Correct answer Nove
Explanation: Shape with teaoure similarty orarae proserpen was emough to preditt connect
1- Noooee
Weak
Moderrtee
Substontial
Figure 79: Page 2 of survey for LucidPPN with no scores
86

---

Published as a conference paper at ICLR 2025
Example 2.
Evidence for correct species
prosotypes of species
&
Shape with texture similarity to image LOWER for the correct species. Color WAS needed.
Evidence for wrong species
Part protolypes wrong species
&
your optioon, whoxs the influenc color the mode's prosenp
Correct anower Sabstartial
Explanation: Shape with teaoure similartt imagee proseryan would ribitaee and ded by soral dering color.
1- Noooe
Weak
Moderrtee
Substontial
Figure 80: Page 3 of survey for LucidPPN with no scores
87

---

Published as a conference paper at ICLR 2025
Question
Evidence for correct species
prosotypes of species
&
Evidence for wrong species
Part protolypes wrong species
&
opirion,
1- None
Weak
Don't know
Substondial
Figure 81: Page 4 of survey for LucidPPN with no scores
88

---

Published as a conference paper at ICLR 2025
Question
Evidence for correct species
prosotypes of species
&
Evidence for wrong species
Part protolypes wrong species
&
opirion,
None
Weak
Don't know
Substondial
Figure 82: Page 5 of survey for LucidPPN with no scores
89

---

Published as a conference paper at ICLR 2025
Question
Evidence for correct species
prosotypes of species
Evidence for wrong species
Part protolypes of wrong species
&
opition,
1- Nooooe
Weak
Don't know
Substondial
Figure 83: Page 6 of survey for LucidPPN with no scores
90

---

Published as a conference paper at ICLR 2025
Question
Evidence for correct species
prosotapes of species
Evidence for wrong species
Part prototypes of wrong species
opinioos
1- NoNooe
Weak
Don't know
Substondial
Figure 84: Page 7 of survey for LucidPPN with no scores
91

---

Published as a conference paper at ICLR 2025
Question
Evidence for correct species
prosotypes of species
Evidence for wrong species
Part prototypes of wrong species
opiniion
1- None
Weak
Don't know
Substondial
Figure 85: Page 8 of survey for LucidPPN with no scores
92

---

Published as a conference paper at ICLR 2025
Question
Evidence for correct species
prosotapes of species
&
Evidence for wrong species
Part prototypes of wrong speciess
opinion,
1- None
Weak
Don't know
Substondial
Figure 86: Page 9 of survey for LucidPPN with no scores
93

---

Published as a conference paper at ICLR 2025
Question
Evidence for correct species
prosotypes of species
&
Evidence for wrong species
Part protolypes of wrong species
&
opiiions
1- Noooo
Weak
Don't know
Substondial
Figure 87: Page 10 of survey for LucidPPN with no scores
94

---

Published as a conference paper at ICLR 2025
Question
Evidence for correct species
prosotypes of species
&
Evidence for wrong species
Part protolypes of wrong species
&
opition,
1- Noooo
Weak
Don't know
Substondial
Figure 88: Page 11 of survey for LucidPPN with no scores
95

---

Published as a conference paper at ICLR 2025
Question
Evidence for correct species
prosotypes of species
Evidence for wrong species
Part prototypes wrong species
&
opirion,
1- None
Weak
Don't know
Substondial
Figure 89: Page 12 of survey for LucidPPN with no scores
96

---

Published as a conference paper at ICLR 2025
Question
Evidence for correct species
prosotypes of species
&
Evidence for wrong species
Part protolypes wrong species
&
opition,
1- None
Weak
Don't know
Substondial
Previous
Figure 90: Page 13 of survey for LucidPPN with no scores
97


Reviews:
[{"review_id": "Kydg8p3P6R", "reviewer": "Reviewer_Zw4S", "summary": "Summary Of Contributions:\n1.Introduction of LucidPPN: This novel architecture separates color features from other visual components during inference, enabling clearer identification of feature importance in the decision-making process.\n2.Consistent Object-Part Mapping: A mechanism ensures that prototypes within each class consistently correspond to the same object parts, improving interpretability.\n3.Enhanced Visualization Method: A more intuitive visualization type is introduced, optimized for fine-grained classification.\n4.Comprehensive Analysis: The paper provides an in-depth examination of LucidPPN's usefulness and limitations, particularly identifying cases where color may or may not be a critical feature in fine-grained classification.", "questions": "Concerns:\n1.It is recommended to add subheadings to each key step or method description to make it easier for readers to understand and locate the content.\n\n2.Consider further improving the accuracy of LucidPPN to enhance its explainability while maintaining a minimal loss of performance.", "limitations": "", "rating": 6, "confidence": 3, "soundness": 2, "presentation": 2, "contribution": 2, "strengths": "1.The LucidPPN in the paper consists of two branches, one for color and the other for shape/texture, which effectively decouples different features. This method can reduce the ambiguity of traditional prototype networks and enable users to better understand the reasons behind the model's decisions.\n\n2.Compared to existing methods, LucidPPN achieves a more detailed analysis of Prototypical Parts, making it easier for users to understand the features that the model is focusing on.\n\n3.Through user studies, it was proven that the explanations provided by LucidPPN are clearer and easier for users to understand than those of other models such as PIP-Net. This empirical result helps to enhance the persuasiveness of the method.", "weaknesses": "Weakness\n\n1.The Section 3 has a lot of paragraphs but lacks subheadings, making it difficult to follow the logical flow of the different parts.\n\n2.There was no noticeable advantage in accuracy. The model was compared on four datasets in total, and its accuracy was lower than that of PIP-Net on two of the datasets, especially on the CUB dataset, where its accuracy was lower than that of all three methods, and no explanation was given for this gap.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "USxI5SFIHA", "reviewer": "Reviewer_na4o", "summary": "In this paper, the authors proposed a Lucid Prototypical Parts Network (LucidPPN), a novel prototypical parts network that separates color prototypes from other visual features. A LucidPPN has two branches: a ShapeTexNet and a ColorNet. Given an input image, the ShapeTexNet is a convolutional neural network (CNN) that takes a gray-scale version of the image as input and outputs a set of feature maps, and the ColorNet is another CNN that takes a down-sampled version of the image as input and outputs another set of feature maps. Since the last layer of both the ShapeTexNet and the ColorNet is a 1x1 convolutional layer with KM filters, we can interpret the last convolutional layer as a prototype layer with KM prototypes, where K is the number of prototypes per class and M is the number of classes, and the output of the last layer as prototype activation maps. The output feature maps (aka prototype activation maps) from the ShapeTexNet and the ColorNet are fused using element-wise products, and then max-pooled to yield a prototype similarity score for each prototype. The predicted class score is simply an average of the prototype similarity scores over all prototypes of the class. In a LucidPPN, each of the K prototypes in each of the M classes corresponds to consistent image parts (e.g., the first prototype of each class corresponds to head of a bird, etc.). This is achieved by aligning the fused output feature maps (prototype activation maps) with segmentation masks produced by a pre-trained PDiscoNet (an object part segmentation model) using a prototypical-object part correspondence loss. In addition to a loss function to improve the classification accuracy of the entire model, the authors also introduced a loss function to improve the classification accuracy of the ShapeTexNet alone and to disentangle color from other visual features. The authors evaluated their LucidPPN models on 4 commonly used fine-grained classification benchmarks (CUB-200-2011, Stanford Cars, Stanford Dogs, and Oxford Flowers), and found that their LucidPPN models achieved competitive test accuracy compared to other interpretable models. The authors also did a user study to evaluate the influence of disentangling color from other visual attributes on interpretability.", "questions": "- My main concern is that I did not see prototype projections in this work. Without prototype projections, how could you conclusively visualize prototypes using training images? The closest training images to a prototype could still be far away from the prototype in the latent space.\n- During training, are the segmentation masks from PDiscoNet aligned with the ShapeTexNet feature maps or the aggregated feature maps? \n- I am also not clear as to why binary cross entropy is used instead of multi-class cross entropy for training?", "limitations": "", "rating": 8, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "- Originality: The paper introduced a novel idea of disentangling color from shape and texture, so that the visual attribute of each prototype is more clearly defined (compared to prior work).\n- Quality: The authors did show that their LucidPPN could maintain a reasonable accuracy while providing less ambiguous prototypes.\n- Clarity: The paper is clearly written.\n- Significance: Interpretability is a significant area of research in machine learning.", "weaknesses": "- Quality: There seems to be no prototype projection in this work. Without prototype projections, it is unclear if the prototypes can be faithfully visualized using training images (because the closest training images to a prototype could still be far away from the prototype in the latent space).\n- Clarity: Page 6, Lines 314-315. I am confused as to whether you are aligning the segmentation masks from PDiscoNet with prototype activation maps from the ShapeTexNet or the aggregated feature maps.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "Ck1szKWb7j", "reviewer": "Reviewer_9gD3", "summary": "This paper propose to disentangle color prototypes from other visual features in ProtoPNets, by introducing a novel network architecture, named LucidPPN. The proposed method clarifies feature importance and aligns prototypical parts with object semantics, enhancing interpretability. Experiments show that LucidPPN achieves competitive accuracy while producing clearer and less ambiguous explanations for users.", "questions": "My questions are listed in \"Weaknesses\" section.", "limitations": "", "rating": 6, "confidence": 5, "soundness": 4, "presentation": 4, "contribution": 4, "strengths": "* This paper explicitly decouple prototypes into specific semantic types, such as color and shape, whereas existing methods have overlooked this aspect of information. And I believe this paper could serve as a significant inspiration for future research.\n* This paper provides sufficient cases and visualizations to validate the semantic information of the learned prototypes.\n* The paper is well-written and easy to follow.\n* The authors provide code for reproducibility check.", "weaknesses": "[Major]\n\n1. **Quantitative evaluation of the interpretability:** In previous work, Huang et al. [1] have discussed the inconsistency of traditional ProtoPNets. Does this issue exists within the proposed method? Please provide qualitative or quantitative evaluations.\n2. **Experiments:** Please supplement the missing results for baseline methods on datasets like DOGS and FLOWERS in Table 1, as adapting to these datasets, which were not covered in the original papers, seems quite straightforward.\n3. **Experiments:** This paper only implement the proposed method on several CNNs. However, vision Transformers are introduced to the realm of CV for several years, and have also been implemented as the backbone of ProtoPNets [2]. Please provide additional experimental results using ViT [3-4] or even CLIP [5] as the backbone.\n4. **Related Work:** In XAI, introducing human understandable semantics as evidences for prediction has been explored by concept bottleneck models (CBMs) [6]. What is the relationship between the proposed method and CBMs. Can concepts be introduced into the realm of ProtoPNet for higher interpretability?\n\n\n> [1] Huang, Qihan, et al. \"Evaluation and improvement of interpretability for self-explainable part-prototype networks.\" Proceedings of the IEEE/CVF International Conference on Computer Vision. 2023.\n> \n> [2] Xue, Mengqi, et al. \"Protopformer: Concentrating on prototypical parts in vision transformers for interpretable image recognition.\" arXiv preprint arXiv:2208.10431 (2022).\n>\n> [3] Dosovitskiy, Alexey, et al. \"An image is worth 16x16 words: Transformers for image recognition at scale.\" International Conference on Learning Representations. 2021.\n>\n> [4] Touvron, Hugo, et al. \"Training data-efficient image transformers & distillation through attention.\" International conference on machine learning. PMLR, 2021.\n>\n> [5] Radford, Alec, et al. \"Learning transferable visual models from natural language supervision.\" International conference on machine learning. PMLR, 2021.\n>\n> [6] Koh, Pang Wei, et al. \"Concept bottleneck models.\" International conference on machine learning. PMLR, 2020.\n\n[Minor]\n\n1. **Experiments:** What is the computational cost of inference and training? Please provide a comparison with baseline methods, including metrics such as training time, FLOPs, and memory usage.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "FIunVysmJR", "reviewer": "Reviewer_9WXj", "summary": "The manuscript presents the Lucid Prototypical Parts Network (LucidPPN), designed to identify key visual features—specifically color, shape, and texture—based on the prototypical parts networks. The proposed LucidPPN utilizes a non-color branch to process grayscale images alongside a color branch that focuses on color information, thereby clarifying the model's decisions based on these visual attributes. Experimental results demonstrate that the proposed method exhibits advantages over baseline approaches and generates more interpretable prototype parts.", "questions": "(1)\tThe manuscript focuses on interpretability through the lenses of color, shape, and texture. However, other low-level features such as edges, contrast, and spatial frequency are also relevant. Have alternative low-level features also been considered in the analysis?\n\n(2)\tThe datasets utilized in the experiments are relatively small in size. How will the proposed method perform on larger datasets, such as ImageNet? Some insights into performance scalability would be beneficial.\n\n(3)\tThe manuscript primarily presents visualization results for the prototypical parts identified by the proposed method. How do these results compare with other prototypical parts-based models? A comparative analysis would enhance the understanding of the method's effectiveness.\n\n(4)\tIn global feature visualizations, such as Figure 14, the manuscript illustrates the ability of the proposed method to detect shape and color. How does this compare with traditional edge detection operators (e.g., Sobel) for shape extraction and color feature extraction methods (e.g., color histogram)? Additionally, how does it compare with the direct visualizations of shallow layer attention to texture and color using techniques like Grad-CAM?", "limitations": "", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "(1)\tThe methodology is well-structured, with intuitive design in the separation of color and non-color network branches, making it accessible and easy to understand.\n\n(2)\tThe experiments are comprehensive, with a substantial number of visualization results provided in the appendix, enhancing the manuscript's depth.", "weaknesses": "(1)\tWhile analyzing \"color,\" \"shape,\" and \"texture\" offers a valuable perspective, these features have been extensively studied in the field of visual perception. Given that the shallow layers of deep networks are capable of extracting low-level features, the necessity for additional processing and analysis from prototypical parts raises concerns on the novelty and contribution of this work.\n\n(2)\tThe improvements demonstrated by the proposed method appear to be limited because its performance on some instances is lower than that of the compared methods. For example, in Table 1, the proposed method underperforms other prototypical parts networks on some datasets. While color, shape, and texture are indeed significant visual features in interpretability, they may not be sufficiently critical in this context.\n\n(3)\tThe organization of the experimental section appears somewhat unbalanced. While the results and visualizations presented are commendable, an excessive amount of content is relegated to the appendix, which may hinder the reader’s ability to grasp key insights and maintain a coherent narrative.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)