ProtoVAE: A Trustworthy Self-Explainable Prototypical Variational Model
Authors: Srishti Gautam
Ahcene Boubekki
Stine Hansen
Suaiba Amina Salahuddin
Robert Jenssen
Marina MC Höhne
Michael Kampffmeyer
Venue: NeurIPS 2022 Accept (2022)
Keywords: Interpretability
Explainable AI
Self-explaining Models
Deep Neural Networks

Abstract: The need for interpretable models has fostered the development of self-explainable classifiers. Prior approaches are either based on multi-stage optimization schemes, impacting the predictive performance of the model, or produce explanations that are not transparent, trustworthy or do not capture the diversity of the data. To address these shortcomings, we propose ProtoVAE, a variational autoencoder-based framework that learns class-specific prototypes in an end-to-end manner and enforces trustworthiness and diversity by regularizing the representation space and introducing an orthonormality constraint. Finally, the model is designed to be transparent by directly incorporating the prototypes into the decision process. Extensive comparisons with previous self-explainable approaches demonstrate the superiority of ProtoVAE, highlighting its ability to generate trustworthy and diverse explanations, while not degrading predictive performance.

Proto A Trustworthy Self-Explainable Prototypical Variational Model
Srishti Gautam', Ahcene Boubekki', Stine Hansen', Suaiba Amina Salahuddin'
Robert Jenssen', Marina MC Höhne Michael Kampffmeyer'
UIT The Arctic University of Norway
Technical University of Berlin
Abstract
The need for interpretable models has fostered the development of self-explainable classifiers. Prior approaches are either based on multi-stage optimization schemes, impacting the predictive performance of the model, produce explanations that are not transparent, trustworthy or do not capture the diversity of the data. To address these shortcomings, propose ProtoVAE, variational autoencoder- based framework that learns class-specific prototypes in an end-to-end manner and enforces frustworthiness and diversity by regularizing the representation space and introducing an orthonormality constraint. Finally, the model is designed to be transparent by directly incorporating the prototypes into the decision process. Extensive comparisons with previous self-explainable approaches demonstrate the superiority of Proto highlighting its ability to generate trustworthy and diverse explanations, while not degrading predictive performance.
Introduction
Despite the substantial performance of deep learning models in solving various automated real-world problems, lack of transparency still remains crucial point of concern. The black-box nature of these high-accuracy achieving models a roadblock in critical domains such as healthcare 2 law [3 or autonomous driving [4). This has led to the emergence of the field of explainable artificial intelligence (XAI) which aims to justify or explain a model's prediction in order to increase trustworthiness, fairness, and safeness in the application of the complex models henceforward.
Consequently, two lines of research have emerged within XAL On the one hand, there are general methodologies explaining a posteriori black-box models, so-called post-hoc explanation methods 6. While on the other hand, there are models developed to provide explanations along with their predictions 8 9 10] The latter class of models, also known self-explainable models (SEMs), are the focus of this work. Recently, many methods have been developed for quantifying post-hoc explanations (1I). However, there is still a lack of concise definition of what SEMs should encompass, thus a lack of comparability of recent methods [12].
Methodologically. a large number of SEMs follow the approach of concept learning, analogous to prototype or basis feature learning, where a set of class representative features are learned 8 [ In this paper gauge SEMs through the prism of three properties. First and foremost, the prototypes should be visualizable in the input space, and these transparent concepts should directly be employed by glass-box classification model. Many of the existing approaches transparency by using nearest training samples to visualize the prototypes
to imitate prototype g [3] while some flatly
training images prototypes preventing end-to-end optimization and limiting the flexibility of the model [g] [14. Secondly, the prototypes should exhibit both inter-class and intra-class diversity. [ are prone to prototype collapse into single point which necessarily undermines their performance. Finally, SEMs should perform comparable to their black-
Methods failing to ensure this property
36th Conference on Neural Information Processing Systems (NcurIPS 2022).

---

box counterparts while producing robust and faithful explanations. Previous approaches have tendency to achieve exllaxxplaaaiiltty by sacrificing the predictive performance 94 [3].
To address the aforementioned shortcomings of current SEMs, we introduce ProtoVAF, a prototypical self-explainable model based variational autoencoder (VAE) backbone. The architecture and the loss function are designed to produce transparent, diverse, and trustworthy predictions, as well as explanations, while relying on an end-to-end optimization. The predictions are linear combinations of distance-based similarity scores with respect to the prototypes in the feature space. The encoder and decoder are trained prior centered on one of the prototypes. The latter are enjoined to capture diverse characteristics of the data through a class-wise wise orthonormality constrain. Consequently, our learned prototypes are truly transparent global explanations that can be decoded and visualized in the input space. Further, we are able to generate local pixel-wist -iieewiie explanations by back- proppaaaggggnnn relevances from the similarity scores. Empirically, our model corroborates trustworthiness both in terms of performance as well as the quality of its explanations.
mixture of VAES sharing the same network but each with its own Gaussian
Our main contributions can be summarized as follows:
We define three properties for SEMs, based on which we present novel prototypical self- explainable model with a variational auto-encoder backbone, equipped with fully transparent prototypical space.
We are able to learn faithful and diverse global explanations easily visualizable in the input space.
We provide an extensive qualitative and quantitative analysis on five image classification datasets. demonstrating the efficiency and trustworthiness of our proposed method.
2 Predicates for a self-explainable model
For the benefit of an efficient and comprehensible formalization of SEMs, we here define three properties that we consider prerequisites for SEMs.
Definition An SEM is transparent if:
(i) iss concepts are utilized perform the downstream task without leveraging complex black-box model;
(ii) its concepts visualizable in input space.
Definition 2 An SEM is diverse if its concepts represent non-overlapping information in the latent space.
Definition 3 An SEM is trustworthy if:
(i) the performance matches to that of the closest black-box counterpart;
(ii) the explanations are robust, i.e., similar images yield similar explanations. (ii) the explanations represent the real contribution of the input features to the prediction.
Note that these definitions echo properties and axioms found other works. However, the view of such properties is diverse across the literature which leads to failure of encompassing the wide research of SEMs in general. For example, transparency known as 'completeness' in (15) and 'local accuracy" in [16]. In the next section, provide comparison of existing SEMs based on the fulfillment of the proposed predicates.
Categorization of related self-explainable works
Self-explainable models optimize for both explainability and prediction, making the network in- herently interpretable. As our main contribution is a prototypical model, we review and categorise existing prototypical SEMs according to the above-mentioned properties.
SENN R introduces general self seff-explainable neural network designed in stages to behave locally like linear model. The model generates interpretable concepts, to which sample similarities are directly aggregated to produce predictions. This generalized approach has been followed by most of the prototypical and concept- based self-explainabbe methods, and is also mirrored by our approach. SENN, however uses training data to provide interpretation of learned concepts, therefore approximating transparency, unlike our model which by-design has decoder to visualize prototypes.

---

Table 1: Summary of the SEM properties satisfied by the baselines. The optimization scheme is also indicated. The symbol indicates that the concepts cannot be directly visualized in the input space and that the nearest training data serve as ersatz.
Transparency
Diversity
Trustworthiness
Optimization
SENN g
End-to-end
ProtoPNet
Alternating
14
Alternating
STT
End-to-end
[3
End-to-end
Pro
End-to-end
where a prototypical layer is introduced before the final classification layer. For maintaining interpretability. the prototypes are set as the projection of closest training image patches after every few iterations during training. Our method is closely related to ProtoPNet with the distinction of decode-able learned prototypes yielding a smooth and regularized prototypical space, thus allowing more flexibility in the model. TesNet 14 extends ProtoPNet and improves diversity at class level using five loss terms. Similarly to our approach, they distribute the base concepts among the classes and include an orthonormal constraint. However, the basis concepts are still projections from the nearest image patches, which leads to loss in predictive performance, similar to ProtoPNet. SITE 17 generates class prototypes from the input and introduces transformation -equivvriiiii model by constraining the interpretations before and after transformation. Since the prototypes are dynamic and generated for each test image, this method only provides local interpretations and lacks global interpretations. FLINT [13] introduces interpreter model (FLINT-g) in addition to the original predictor model FF..NT-)). Although FLINT- has been introduced by the authors an SEM on its own. Therefore, we focus on FLINTT-, henceforward referred to as FIJNTT. FLINT takes input features of several hidden layers of the predictor to learn dictionary of attributes. However, the interpreter is not able to approximate the predictor model perfectly, therefore losing trustworthiness. Unlike prior approaches, VAE designed to fulfill all three SEM properties. We summarize the discussed methods and their categorization in Table T
ProtoPNet is a representative of line of works 1 18 19 20
framework that learns in parallel the interpretations, it is not
4 Proto VAE
In this section, we introduce PAO, which is designed to obey the aforementioned SEM properties. Specifically, transparency is in-built in the architecture and further enforced along with diversity through the loss function. Also, we describe how our choices ensure the trustworthiness of our method.
4.1 Transparent architecture
In a transparent self-explainable model, the predictions are interpretable functions of concepts visualizable in the input space. To satisfy this property, we rely on an autoencoder- architecture as backbone and linear classifier. In order have consistent, robust, and diverse global explanations. we consider prototypes in greater number than classes. Unlike previous prototypical methods (g] 14, which update the prototypes every few iterations with the embeddings of the closest training images, Proto VAE is trained end-to-end to learn both the prototypes in the feature space and the projection back to the input space. This gives VAE the flexibility to capture more general class characteristics. To further alleviate situations where some of the optimized prototypes are positioned far from the training data in the feature space, possibly causing poor reconstructions and interpretations, we leverage variational autoencoder (VAE). VAEs are known to learn more robust embeddings and thus generate better reconstructions from out-of-distribution samples than simple autoencoders 21 A schematic representation of the network is depicted in Fig.
Details of the operations The downstream task at hand is the classification into K cllasses of the image dataset X {(z,, y.) where x; E IR" is image and the one-hot vector yı E {0, 1]K
3

---

sample
T
emcooerr
decoder
y
KM
prototypes of closs 1
prototypes of closs K
classifier
Schematic representation of the architecture of ProtoVAE. The input image æ encoded by into tuple (J,, or.. A vector z is sampled from (i,, σ which, on one side, is decoded by into the reconstructed input è and, on the other side, is compared to the prototypes Φxy resulting in the similarity scores The latter are passed through the classifier h to get the final prediction ý.
Figure 1:
encodes its label. The network consists of encoder f RR R² x R², decoder g RR RR"
(d p), M prototypes per class + {Φkz k=l::KK j=1. similarity function sim IR" - IR and glass-box linear classifier h IR
R + [0, 1]K An image z; is first transformed by f into a tuple (u,, σ.) = f(x)) which, in the VAE realm, are the parameters of the posterior distribution. feature vector 2; is then sampled from the normal distribution N (p,, σz) and is used twice. On the one hand, it is decoded as Ø; g(z,). On the other hand, it is compared the prototypes. We use the same
similarity function in g and obtain the resulting vector s; E IR.
K' as:
= sim Φb²) = log (1
22 2xi11 +11
Φbgl
with 0 < 1. Finally, is used to compute the predictions: ÿ; h(s.). Moreover, the similarity vector s; captures the distance the prototypes but also indicates the influence each prototype on the prediction.
4.2 Diversity and trustworthiness
Unlike transparency, diversity cannot be achieved solely through the architectural choices. It needs to be further enforced during the optimization. Our architecture implies two loss terms: a classification loss and VoE-loss. Without further regulation, model is left vulnerable to the curse of prototype collapse 14 22 which would undermine the SEM diversity property. We prevent such a situation with a third loss term enforcing orthonormality between prototypes of the same class. The loss function of Proto VAE can thus be stated as follows:
PProtoVAE Lpred Lorth LVAE-
We detail now each term and discuss how they favor diversity and trustworthiness.
Inter-class diversity through classification Although the prototypes are assigned to a class, the classifier is blind that information. Thus, the prediction problem is classic classification that we solve using the cross-entropy loss.
N
Cpeed
CE(h(s:):
(3)
Since h is linear, the loss pushes the embedding of each class to be linearly separable, yielding greater inter-class diversity of the prototypes.
The inter-class diversity is guaranteed by the previous terms. However, without further regularization, the prototypes might collapse to the center of the class, obviating the possibilities offered by the extra prototypes. To prevent such situation and foster intra-class diversity, we enforce the prototype of each class to orthonormal to each other follows:
Intra-class diversity through orthonormalization
K
Lorth
Ias
(4)
k=1

---

where IM is the identity matrix of dimension M x M and the column-vectors of matrix & are the prototypes assigned to class minus their mean, i.e., $ {Φt Øk. J ... M) with ΦR Ei= i=1 Ost. Beyond regularizing the Frobenius norm I|-II of the prototype, this term favors the disentanglement of the captured concepts within each class, which is one way obtain intra-class diversity.
Robust classification and reconstruction through VAE The VAE architecture ensures the ro- bustness of the embedding and of the decoder. In its original form, the VAE loss considers a single standard normal distribution as a prior and is trained to minimize:
| =1 + DKr. (= Ia)),
where La the identity matrix of dimension
Such an objective enjoins the embedding to organize as if generated by single Gaussian distribution, thus making it difficult to split it with the linear classifier h. To help the classifier, we consider instead a mixture of VAEs sharing the same network each with a Gaussian prior centered on one of the prototypes. Since, each prototype has label, only data-points sharing that label are involved in the training of the associated VAE. The loss function of our mixture of VAEs is (derivation in the supplementary material Sec. S2):
K M
S: (k.j) DEL (N (;) (N) Ia)). (6) Ei (k,
LVAB
o 2111
(k)
In addition to training the decoder, this loss enjoins the embedding to gather closely around their class prototypes.
4.3 Visualization of explanations
VAE is designed to have the inherent capability to reconstruct prototypes via the decoder, which trained to approximate the input distribution. Additionally, to generate faithful pixel- local explanation maps, we build upon the concepts of Layer-wise relevance propagation (LRP) 2 which a awdel- XAI method computing relevances based on the contribution of neuron to the
prediction. Following [12), generate explanation maps, where for each prototype, the similarity of an input to the prototype is backpropagated to the input image according to the LRP rules. For an input image 2, the point-wise similarity between the transformed mean vector μa; with prototype Φk is first calculated as:
Yikj day +
with d:ks (11: Ø2y) * (Mz - Økj):
where is the Hadamard element-wist product and r7 > O. The similarity "ikj is then backpropagated through the encoder following LRP composite rule, which known as best practice [24] to compute local explanation maps. Following this, the LRP,, 0,3 rule is applied to the convolutional layers and the
Deep Taylor Decomposition based rule DTD.# 25 is applied to the input features.
Experiments
In this section, we conduct extensive experiments to evaluate VAOls trustworthiness, trans- parency, and ability to capture the diversity in the data. More specifically, we demonstrate the trustworthiness of our model in terms of predictive performance in Sec. |. Qualitative evaluations are then conducted in Sec. 5.2 to verify the diversity and transparency properties, followed by quantitative evaluation of the explanations corroborating its trustworthiness. Additionally, we provide an ablation study for the terms in Eq. and further study the effect of the I.2 norm in Eq, 6 on the prototype reconstructions the supplementary material in Sec. S6.1 and Sec. S6.9, respectively.
Datasets and implementation: We evaluate Proto VAE on 5 datasets, MNIST (26.. FashionMNIST 27 (IMNIST), CIFAR-10, (28). a subset of QuickDraw 22] and SVHN 30 We use small encoder networks with 4 convolution layers for MNIST, fMNIST and CIFAR- 3 for QuickDraw and for SVHN. These convolution layers are followed by linear layers which gives us the tuple (Ms o.) for each image i. The decoder mirrors the encoder's architecture. Similar to 19), we fix
5

---

Table 2: Performance results of PAo compared to other state-of-ther thrt--rrtt methods (measured in accuracy (in %)). The reported numbers are means and standard deviations over runs. Best and statistically non-significantly different results are marked in bold. *Results for SITE are taken from the original paper and thus based on more complex architectures.
Black-box encoder
FLINT B
SENN
*SITE
ProtoPNet 9
Proto ProtoVAE VAE
MNIST
99.2#0.1
99,4=0.1
98.8
98.8
94.740.6
99,4=0.1
IMNIST
0.2
91.5=0.2
88.3+0.3
85.4+00.6
91.9==0.2
CIFAR-10
83.9
79.6=0.6
76.3++0.2
84.0
67.8++0.9
84.6=-011
Draw
86.7+004
82.6+1.4
79.3+0.3
58.7++0.0
87.5
SVHN
0..
90.8+004
91.5+0.4
88.6++0.3
92.2-+0.3
MNIST
IMNIST
SVHN
CIFAR-10
2
2
2
Dress
HODK Autorabble
4
4
4
Sandals
2
7
7
7
Bag
6
6
Ship
Figure 2: Visualization of learned prototypes for different classes for MNIST, IMNIST, SVHN and CIFAR-10.
the prototypes per class, M,, to 5 for MNIST and SVHN and for the other datasets. Further details about the datasets and additional implementation details, such as the detailed architecture and hyperparameters, are provided in the supplementary material Sec. S3 and S4. Our code available at hwwpr///wwoooomtSsssccaaaaaaaaaa
Baselines To ensure fair comparison, we modified the publicly available code of ProtoPNet, FLINT and SENN to use the same backbone network as ProtoVAE and when relevant the same number of prototypes per class used for ProtoVAE. We also provide the results for the predictive performance of SITE as reported in [17], since the code is not publicly available. We also report the performance of our model with a ResNet-18 backbone in the supplementary material Sec. S6.2. Further, we compare Proto VAE using FLINT's encoders as provided in [33 for both FLINT and SENN in Sec. $6.3 in the supplementary material. Finally, we also compare with the black-box counterpart of our model, i.e, a classical feed-forwart CNN based on the same encoder as Proto but followed by a linear classifier and trained end-to-end with the cross-entropy loss. This black-bor encoder model is thus free from all regularization necessary for saff-xxlainnabiity..
5.1 Evaluation of predictive performance
In the Table 2 we can observe that ProtoVAE surpasses other SEMs in terms of predictive performance on all five datasets, which is based on its increased flexibility in the architecture. For ProtoPNet, we observe a gap in performance, which is due to the low number of optimal class- representatives in the actual training data. This creates huge bottleneck at the prototype layer and therefore limits its performance. Further, and more importantly, when compared the true black-box counterpart, Proto VAE achieves no loss in accuracy and is even able to perform better all the datasets. We believe this is due to an efficient over-clustering the latent space with the flexible prototypes, well the natural regularizations achieved through the VAE model. These results strengthens the trustworthiness Proto VAE in terms predictive performance.
5.2 Evaluation of explanations
Qualitative evaluation The demonstrated results in this section strengthen the fulfilment of the transparency property by providing human-understandable explanations for ProtoVAE. We visualize the decoded prototypes for different datasets, which act global explanations for the corresponding

---

2
2
2
2
2
2
Figure 3: UMAP representations for the prototypical space for MNIST (left), decoded prototypes overlayed for classes 2, 4 and 7 (right), and interpolation between prototypes of the same class (2) and between prototypes of different classes (2-7) (bottom).
classes, in Fig. 2 The prototypes for MNIST demonstrate that class 2 consist of 2's with flat bottom line or with rounded bottom lines. For IMNIST, the sandals class consists of both heels and flats. The class prototypes thus directly help visualizing the components of the classes by looking at a fixed number of prototypes per class instead of all the training data. Interestingly, although SVHN often contains multiple digits of different classes in the same training image, our prototypes efficiently capture only one digit representing its class. Moreover, a blurring effect is observed in prototypes which captures more variability and therefore suggests efficient representation of the true "mean" of a subset of a given class, as opposed to other methods O 14 [3] which show the closest training images and therefore sharper. This behavior supports flexibility in the network, therefore enhancing predictive capability along with the ability to provide more faithful explanations. This blurring effect is observed to be more prominent in CIFAR-10 10, which is due to the high complexity in each class in the dataset and can be reduced by using larger number of prototypes per class Additionally, transparent prototypical space, we show UMAP representations of the prototypes and the training data for MNIST in Fig. 3 This visualization further illustrates the inter-class as well the intra-class diversity of the prototypes. Moreover, due to the regularized prototypical space, we are efficiently able to interpolate between prototypes both within a class and between classes, therefore making the latent space fully transparent. In Fig, 3 we interpolate between 2 different prototypes of class '2' and from a prototype of class '2' class '7' prototype.
claim of more
provide more clear visualization of the learned
The local explanability maps for a test image according to the three closest, i.e. most similar, prototypes for both PAO and ProtoPNet are shown in Fig. along with the corresponding similarity scores. As seen, different prototypes of the same class activate different parts of the same test image, which therefore helps in achieving better performance. The ProtoPNet maps are extremely coarse which therefore makes them challenging to interpret. Therefore, we overlay the heatmaps over the input image for ProtoPNet. As observed, the most activated prototypes do not belong to the same class as the test image. This might happen because of ProtoPNet focusing on patches in prototypes, therefore losing contextual information. The closest prototypes shown for the image "apple' belong to class 'lion'. Further, an uninformative training image, which is not seen in the ProtoVAE prototypes, has been selected by ProtoPNet to represent out of prototypes for class 'lion". The remaining 5 prototypes are represented by other same training image. This effect is seen predominantly in ProtoPNet where the prototypes of the same class collapse to one point and are thus represented by the same training image, therefore dissatisfying the diversity property, opposed to Proto The prototypes for class 'lion' for both the models are included in the supplementary material in Sec. S6.4.
As reference to gauge the quality and sharpness of the pictures Fig reconstructions of test images provided Sec. 1..11. We demonstrate this behavior in Sec. S6.8 and show in Sec. S6.7 how local explainability maps can be used gather additional information about pixel-wist wise relevances thereby counterbalaneing blurry prototypes.
7

---

Nearest prototypes
Nearest prototypes
5 W
5 V
5
Apple
Apple
Apple
Pro
5
Apple
Test Image
0,505
0.488
0.476
Test Image
0.234
0.230
0.228
Nearest prototypes
Nearest prototypes
9
Lion
Lion
Lion
ProtoPNet
I
Apple
Test Image
Test Image
1.798
1,719
1.719
1.130
1.130
1.130
Figure 4: Three maximally activated prototypes, the corresponding prototypical activations, and corresponding similarity scores for a test image of class 5 (for MNIST) and apple (for QuickDraw), for both ProtoVA VAE and ProtoPNet models.
Nearest protolypes from different
Nearest protolypes from different
7
I
Ant
Apple
9
Dos
Test Image
0.080
0.076
0.082
Tost Image
0.096
0.098
0.108
Figure 5: Maximally activated prototypes from three random classes, along with the prototypical explanations for MNIST (left) and QuickDraw (right) datasets.
We also show the closest prototypes from different random classes and their corresponding ex- plainability maps demonstrate the behavior of explanations for different class prototypes Fig. [5] Interestingly, the 'dog' image from the QuickDraw dataset resembles an 'ant" prototype for the legs. an "apple" prototype for the face and 'cat' prototype for the ears. This information provided by the local explainability maps thus aligns well with human- uuman-understandabll concepts.
To compare the efficacy of the mapping the input space learned by our decoder, methodologies we show prototypes along with the closest training images for different datasets Fig. The prototypes are observed to be the representative of a subset of the respective class. For example, the prototype shown for class '4' of MNIST is representing the subset '4' with an extended bar, while the 'banana' prototype represents the left facing "banana" subset, and the 'dog' prototype represents the subset of white dogs on darker background.
with training-data projection of prototypes (9]
Finally, in order to demonstrate the scalability of Proto VAE and its applicability on complex higher- resolution real world datasets, we provide analysis on the CelebA dataset [31] in Sec. S6. 10. Note that the less important and fairly diverse features (such as background) appear blurry, while the more important features (skin color, hair color, hair style or age) crisp and clearly visible.
Quantitative evaluation To quantify the trustworthiness of the explanations provided by the proposed model, we calculate the Average Drop (AD) and Average Increase (AI) with respect local explanation maps and similarity scores for all prototypes [2] The AD measures the decrease in similarity scores with respect to each prototype when the 50% least important pixels are removed from the images, while Al estimates the ratio of increasing similarity scores. A low AD and high

---

MNIST4
Neurest training images
IMNIST-Sundals
Nearest training images
4
4
4
QuickDraw-
CIFARIO-Dog
Figure 6: The three closest training images to the learned prototypes for MNIST (class '4'), Quick- Draw (class Baaanaa), ) IMNIST (class 'Sandals') and CIFAR-IO (class 'Dog'), proving our prototypes representing a "real mean" of subset of classes.
Table 3: AD and AI for quantitative evaluation of explanations of ProtoVAE and ProtoPNet. The re- ported numbers are means and standard deviations over runs. Best and statistically con-significantly different results are marked in bold.
MNIST AD
IMNIST AD
CIFAR-I0
QuickDraw
SVHN A1
AD
ProtoPNet
3.4+0.3
0.62:000
7.2404
0.5+00
11.6202
0.5+0.0
2.6:0.11
0.7-0.0
5.40.0
0.7::000
Proto VAE
0.4:0000
0.6:0.00
5.1:0.0
0.800.0
6.622:00
0.7400.0
0.1:00.0
0.9
6.1=0.1
0.7::0.0
AI suggest better performance. These scores computed as follows:
AD = NKM ΣΣΣ
100
K M
max (0, (k:,
50% (k:))) (k
AI=
N
[k.(k.)) < 50%
(1, (1.
NKM
where s) is the similarity score of image i with prototype j of class k (see Eq and 350% (k.j) j) the similarity score after masking the 50% least activated pixels according to the prototypical explanation map of prototype j. Also, [[-]] are the Iverson brackets which take the value 1 if the statement they contain is satisfied and otherwise.
We report the mean and standard deviation for AD and AI computed over 5 random subsets of 1000 test images for Proto VAE and ProtoPNet in Table 3 For the grayscale datasets, MNIST and IMNIST, the masked pixels are replaced by 0. For CIFAR- TO and SVHN, they are replaced by random uniformly sampled values. ProtoVAE achieves considerably lower AD and higher comparable AI for all the datasets. For SVHN, ProtoPNet performs well which we believe is due to the abundance of representative patches in the dataset, thereby improving its explanations.
Finally, we perform relevance ordering test 33 where we start from a random image and monitor the predicted class probabilities while gradually adding percentage of the most relevant pixels to the random image according to the local explanation maps. We take 100 random test images and report the average results of change in predicted class probability for all the prototypes in the model. The rate distortion graphs are shown in Fig. 7 for MNIST, QuickDraw and SVHN. We also include baselines, ordered randomly. Larger area under the curve indicates better performance. As shown, VAE's local explanations able capture relevant information than ProtoPNet for all three datasets. Further, for MNIST, ProtoPNet is performing even worse than Random-ProtoPNer highlighting the lack of trustworthiness in ProtoPNet's explanations.
and Random-Prood ProtooV... where the pixel relevances
Conclusion and Discussion
In this work, we define three properties that act as prerequisites for efficient development of SEMs. namely, transparency, diversity, and trustworthiness. We then introduce ProtoVAE a prototypical self-explainable method, based on variational auto-encoder backbone, which addresses these three properties. Pro incorporates transparent model and enforces diversity and trustworthiness through the loss functions. In addition to providing faithful explanations, Proto is able to achieve better predictive performance than its counterpart black-box models.

---

MNIST
QuickDraw
SVHN
Prdiiceed class pribabiity
0.8
0.8
0.8
0.2
0.2
20 40
60
100%
0
20
40
60
100%
20
40
60
00
100%
Most importart pixels
ProtoVAE
ProtoRNet
fandorn-ProtoVE
Random-ProtaPMet
Figure 7: Relevance ordering test for ProtoPNet and Proto VAE, along with the respective random baselines (Random-Peed ProtoPPee and Random- Prooooo--roooo PAE)) Higher curve suggests better performance of PAO for all 3 datasets of MNIST, QuickDraw and SVHN.
The main limitation of Proto is the fixed number of prototypes. This means that the model has to grasp simple as well as more complex classes with the same number of prototypes. For example, in MNIST, there more variations to be captured by the prototypes in the class '4' than in class '1', I A simple but effective solution is a distance-basee pruning procedure, which will be explored in future works. Another approach in sight is to prior the distribution of the prototypical similarities and prioritize some prototypes by controlling the frequency with which each prototype is used in the predictions. Finally, since global explanations only be good the decoder, one more promising research direction is to leverage more expressive generative models, such as "Very Deep VAEs" (34] and normalizing flows (35] further improve the scalability of the method to more complex datasets.
Acknowledgments and Disclosure of Funding
This work was financially supported by the Research Council of Norway (RCN), through its Centre for Research-based Innovation funding scheme (Visual Intelligence, grant no. 309439), and Consortium Partners. The work further partially funded by RCN FRIPRO grant no. 315029 and RCN IKTPLUSS grant no. 303514. Moreover, the work was partly supported by the German Ministry for Education and Research through the third-party funding project Explaining 4.0 (ref. 011S20055).
References
[1] Alex J DeGrave, Joseph D Janizek, and Su-In Lee. AI for radiographic COVID-19 detection selects shortcuts over signal. Nature Machine Intelligence, 3(7):610-619 619, 2021.
[2] Srishti Gautam, Marina M.-C. Höhne, Stine Hansen, Robert Jenssen, and Michael Kampffmeyer. Demonstrating the risk of imbalanced datasets in chest x-ray image-based diagnostics by prototypical relevance propagation. In 2022 IEEE 19th International Symposium Biomedical Imaging (ISBI), pages 1-5, 2022.
[3] Cynthia Rudin. Stop explaining black box machine learning models for high stakes decisions and use interpretable models instead. Nature Machine Intelligence, (5):206-215, 2019.
[4] Daniel Omeiza, Helena Webb, Marina Jirotka, and Lars Kunze. Explanations in autonomous driving: survey. IEEE Transactions on Intelligent Transportation Systems, pages 1-21, 2021.
[5] Marco Tulio Ribeiro, Sameer Singh, and Carlos Guestrin. "why should i trust you?": Explaining the predictions of any classifier. KDD '16, page 1 1135-1144, 2016.
[6] Ramprasaath R. Selvaraju, Michael Cogswell, Abhishek Das, Ramakrishna Vedantam, Devi Parikh, and Dhruv Batra. Grad-cam: Visual explanations from deep networks via gradient- based localization. 20/7 IEEE International Conference on Computer Vision (ICCV), pages 318-626. 2017.
10

---

[7] Jason Yosinski, Jeff Clune, Anh M Nguyen, Thomas J. Fuchs, and Hod Lipson. Understanding neural networks through deep visualization. ICML Deep Learning Workshop, 2015.
[8] David Alvarez Melis and Tommi Jaakkola. Towards robust interpretability with self-explaining neural networks. In S. Bengio, H. Wallach, H. Larochelle, K. Grauman, N. Cesa- Bianchi, and R. Garnett, editors, Advances in Neural Information Processing Systems, volume 31. Curran Associates, Inc., 2018.
[9] Chaofan Chen, Oscar Li, Daniel Tao, Alina Barnett, Cynthia Rudin, and Jonathan K Su. This looks like that: Deep learning for interpretable image recognition. In H. Wallach, H. Larochelle, A. Beygelzimer, d'Alché-Buus E. Fox, and R. Gameet, editors, Advances in Neural Information Processing Systems, volume 32. Curran Associates, Inc., 2019.
[10] Sascha Saralajew, Lars Holdijk, Maike Rees, Ebubekir Asan, and Thomas Villmann. seepffaattin-b--compome Probabilistic modeling of reasoning over a set of components. In NeurlPS, 2019.
[11] Anna Hedström, Leander Weber, Dilyara Bareeva, Franz Motzkus, Wojciech Samek, Sebastian Lapuschkin, and Marina M.-C. Höhne. Quantus: An explainable ai toolkit for responsible evaluation of neural network explanations. 2022.
[12] Srishti Gautam, Marina M. C. Höhne, Stine Hansen, Robert Jenssen, and Michael Kampffmeyer. This looks more like that: Enhancing self-explaining models by prototypical relevance propaga- tion. arxiv, 2021.
[13] Jayneel Parekh, Pavlo Mozharovskyi, and Florence d'Alché-Bue Buch A framework to learn with interpretation. In M. Ranzato, A. Beygelzimer, Y. Dauphin, P.S. Liang, and J. Wortman Vaughan, editors, Advances in Neural Information Processing Systems, volume 34, pages 24273-24285. Curran Associates, Inc., 2021.
[14] Jiaqi Wang, Huafeng Liu, Xinyue Wang, and Liping Jing. Interpretable image recognition by constructing transparent embedding space. In 2021 IEEE/CVF International Conference on Computer Vision (ICCV), pages 875-884, 2021.
[15] Mukund Sundararajan, Ankur Taly, and Qiqi Yan. Axiomatic attribution for deep networks. In Proceedings of the 34th International Conference Machine Learning- Volume 70, ICML'17, page 5319-3328. JMLR.org, 2017.
[16] Scott M. Lundberg and Su-In Lee. A unified approach to interpreting model predictions. In Proceedings of the 3 International Conference on Neural Information Processing Systems, NIPS 17, page 4768 4777, Red Hook, NY, USA, 2017. Curran Associates Inc.
[17] Yipei Wang and Xiaoqian Wang. Self-interpretable model with transformation equivariant interpretation. InM. M. Ranzato, A. Beygelzimer, Y. Dauphin, P.S. Liang, and J. Wortman Vaughan, editors, Advances in Neural Information Processing Systems, volume 34, pages 2359-2372. Curran Associates, Inc., 2021.
[18] Eunji Kim, Siwon Kim, Minji Seo, and Sungroh Yoon. Xprotonet: Diagnosis in chest radio- graphy with global and local explanations. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 15719-15728, June 2021.
[19] Dawid Rymarezyk, tukasz Struski, Jacek Tabor, and Bartosz Zieliński. Protopshare: Prototypi- cal parts sharing for similarity discovery in interpretable image classification. In Proceedings of the 27th ACM SIGKDD Conference on Knowledge Discovery &amp; Data Mining, KDD '21, page 1420-1430, New York, NY, USA, 2021. Association for Computing Machinery.
[20] Meike Nauta, Ron van Bree, and Christin Seifert. Neural prototype trees for interpretable fine-grained image recognition. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 14933-14943, June 2021.
[21] Alexander Camuto, Matthew Willetts, Stephen Roberts, Chris Holmes, and Tom Rainforth. To- wards a theoretical understanding of the robustness of variational autoencoders. International Conference Artificial Intelligence and Statistics, pages 3565-3573. PMLR, 2021.
11

---

[22] Li Jing, Pascal Vincent, Yann LeCun, and Yuandong Tian. Understanding dimensional collapse contrastive self-supervised learning. arXiv preprint arXiv:2110.09548, 2021.
[23] Sebastian Bach, Alexander Binder, Grégoire Montavon, Frederick Klauschen, Klaus-Robert Müller, and Wojciech Samek. On pixel-wise explanations for non-linear classifier decisions by layer-wise relevance propagation. PLOS ONE, 10(7):1 6, 07 2015.
[24] M. Kohlbrenner, A. Bauer, S. Nakajima, A. Binder, W. Samek, and S. Lapuschkin. Towards best practice explaining neural network decisions with Irp. In IJCNN, pages 1-7, 2020.
[25] Grégoire Montavon, Sebastian Lapuschkin, Alexander Binder, Wojeiech Samek, and Klaus- Robert Müller. Explaining nonlinear classification decisions with deep taylor decomposition. Puttern Recognition, 65:211-222, 2017.
[26] Y. Lecun, L.. Bottou, Y. Bengio, and P. Haffner. Gradient-based learning applied to document recognition. Proceedings of the IEEE, 86(11):2278-2324 1998.
[27] Han Xiao, Kashif Rasul, and Roland Vollgraf. Fashion-mnist: a novel image dataset for henchmarking machine learning algorithms. CoRR, ab//1788.077747, 2017.
[28] Alex Krizhevsky. Learning multiple layers of features from tiny images. 2009.
[29] David Ha and Douglas Eck. A neural representation of sketch drawings. In International Conference on Learning Representations, 2018.
[30] Yuval Netzer, Tao Wang, Adam Coates, Alessandro Bissacco, Bo Wu, and Andrew Y. Ng. Reading digits in natural images with unsupervised feature learning. In NIPS Workshop on Deep Learning and Unsupervised Feature Learning 2011, 2011.
[31] Ziwei Liu, Ping Luo, Xiaogang Wang, and Xiaoou Tang. Deep learning face attributes in the wild. In Proceedings of International Conference on Computer Vision (ICCV), December 2015.
[32] Jeong Ryong Lee, Sewon Kim, Inyong Park, Taejoon Eo, and Dosik Hwang. Relevance-cam: Your model already knows where to look. In Proceedings of the IEEE/CVE Conference on Computer Vision and Pattern Recognition (CVPR), pages 14944-1499 14953, June 2021.
[33] Jan MacDonald, Stephan Wäldchen, Sascha Hauch, and Gitta Kutyniok. rate-distortion framework for explaining neural network decisions. ArXiv, abs/1905.1 1092, 2019.
[34] Rewon Child. Very deep vaes generalize autoregressive models and can outperform them on images. arXiv preprint arXiv:2011.106500 2020.
[35] Danilo Rezende and Shakir Mohamed. Variational inference with normalizing flows. In International conference machine learning, pages 1530-1538. PMLR, 2015,
Checklist
1. For all authors...
(a) Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope? [Yes]
(b) Did you describe the limitations of your work? [Yes] Discussed in section 6
(C) Did you discuss any potential negative societal impacts of your work? [Yes] Included in the supplementary material.
(d) Have you read the ethics review guidelines and ensured that your paper conforms to them?
[Yes]
2. If you are including theoretical results...
(a) Did you state the full assumptions of all theoretical results? [N/A]
(b) Did you include complete proofs of all theoretical results? [N/A]
you ran experiments..
(a) Did you include the code, data, and instructions needed to reproduce the main experimental results (either in the supplemental material or as a URL)? [Yes] Provided the supplementary material.
12

---

(b) Did you specify all the training details (e.g., data splits, hyperparameters, how they were chosen)? [Yes] Details reported the supplementary material.
(c) Did you report error bars (e.g., with respect to the random seed after running experiments multiple times)? [Yes] Mean and standard deviation for accuracy are reported.
(d) Did you include the total amount of compute and the type of resources used (e.g.. type of GPUs, internal cluster, or cloud provider)? [Yes] Included the supplementary material.
4. If you are using existing assets (e.g., code, data, models) curating/releasing new assets...
(a) If your work uses existing assets, did you cite the creators? [Yes]
(b) Did you mention the license of the assets? [Yes] Included in the supplementary material.
(c) Did you include any new assets either in the supplemental material or as URL? [Yes] Provided in the supplementary material.
(d) Did you discuss whether and how consent was obtained from people whose data you're ro using/curating? [N/A] All the datasets used in this work open-source.
(e) Did you discuss whether the data you are using/curating contains personally identifiable information offensive content? [N/A] The datasets used open-source and do not contain personally identifiable information or offensive content.
5. If you used crowdsourcing or conducted research with human subjects...
(a) Did you include the full text of instructions given to participants and screenshots, if applica-
ble? [N/A]
(b) Did you describe any potential participant risks, with links to Institutional Review Board (IRB) approvals, if applicable? [N/A]
(c) Did you include the estimated hourly wage paid to participants and the total amount spent on participant compensation? [N/A]
13


Reviews:
[{"review_id": "D8yPIvZeHoY", "reviewer": "Reviewer_zZon", "summary": "The authors introduce ProtoVAE, a VAE-based framework for interpretable classification. Specifically, the authors propose to learn class-specific prototypes in the VAE's latent space. For a given sample, the distance to those prototypes is then used by a linear layer to classify that sample. Since the authors additionally train a decoder, the learnt prototypes can be visualised in input space, which makes the prototypes and thus the classification pipeline easily interpretable.\n\nTo evaluate their approach, the authors compare to various other self-explainable models with a focus on ProtoPNet and discuss advantages w.r.t. classification accuracy and interpretability of their proposed approach.", "questions": "Please see the list of weaknesses, I would appreciate if the authors could address my concerns.\n\nSpecifically:\n- Can we hope to scale this approach to larger datasets? How would this approach compare to ProtoPNet on, e.g., CUB?\n- How does the interpretability of ProtoVAE compare to ProtoPNet if the prototypes cannot be visualised anymore (e.g., prototypes are already difficult to interpret on CIFAR10)?\n- What do we learn from the LRP-based explanations? The heatmaps do not seem to be easily interpretable and even somewhat conflicting when it comes to highlighting the similarity to the different prototypes (Fig. 4, top row).\n- The authors say that the blurriness of the CIFAR prototypes can be reduced by a larger number of prototypes. Is there experimental evidence for this? What is the impact of the L2 reconstruction loss on this? ", "limitations": "To me, there are no obvious negative societal impacts that the authors would need to address in more detail. However, as expressed in the previous points, I believe the manuscript would benefit significantly from a more detailed discussion of the limitations w.r.t. scalability and current bottlenecks.", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "## Strengths \n- The proposed approach is conceptually simple, but nevertheless yields better accuracy than other self-explainable models on the evaluated datasets and allows for a direct visualisation of the used prototypes in input space. \n- By using a VAE, the proposed approach yields an easily interpretable latent space and the model can interpolate between different input samples.\n- Further, the model allows for both global explanations (the learnt prototypes) as well as local explanations (distances to learnt prototypes of a given sample).\n- The manuscript is generally well-written and easy to read.\n\n\n## Weaknesses\n\n- ProtoPNet was developed for a more complex dataset (larger images, fine-grained classification). In contrast, the proposed approach evaluates only on much simpler datasets — a discussion on potential scalability issues is missing. \n- Continuing the previous point: the interpretability of the proposed model crucially depends on the quality of the decoder. However, on CIFAR10 the prototypes already become much less interpretable (Fig. 6, dog prototype or Fig. 2 CIFAR10 prototypes). Can we even hope for such an approach to be scaled to 'relevant' datasets in terms of interpretability?\n- Apart from the interpretability, the accuracy on CIFAR10 also seems very limited. I would appreciate if the authors could comment on the major limiting factors w.r.t. classification performance. A minor point related to this: the comparison to the black-box encoders is confusing (Table 2). While I understand that the authors introduce this as a relevant comparison in Definition 3 (i), this comparison seems to implicitly suggest that ProtoVAEs outperform black-box models. However, 84.6% accuracy on CIFAR10 is, of course, far from what DNNs can achieve.\n- A core feature of the ProtoPNet is to learn and compare to *local* prototypes (\"This looks like that\"). In contrast, the ProtoVAE is designed to learn full-image prototypes, which limits its flexibility. I would appreciate if the authors could comment on learning local prototypes, especially given that the comparison to ProtoPNet is so central in the manuscript. \n- While I understand the motivation for the local explanations according using LRP, I am unsure about their added benefit w.r.t. the interpretability of the model decision. I would appreciate if the authors could discuss this in more detail, specifically because it seems difficult to parse those explanations: what do we learn e.g., from the heatmaps in Fig. 4 (top row)? (minor: and what is the colour coding exactly? red positive, blue negative?) Further, LRP is a general framework that encompasses many potential rules for backpropagating relevance scores. It is unclear from the text which rules were chosen.\n\n\nMinor:\n- There are other generative classification approaches, maybe most notably normalizing flows (e.g., Mackowiak et al. 2021), that might be relevant for a more complete contextualisation. \n", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"ethics_review_area": ""}, "extra_text": {}}, {"review_id": "ARQKwFCIsDk", "reviewer": "Reviewer_iezD", "summary": "The authors presented a nice piece of work in this paper towards improving deep model explainability through concept learning with prototypes. The proposed method is designed to generate explanations that are transparent, diverse and trustworthy. The predictive performance of the proposed method is better compared to related methods with various digit as well as natural image datasets.", "questions": "Questions: a) VAEs are not well known for reconstruction quality of bigger size images or complex images with more prototypes compared to the 5 datasets used for experimentation in this paper. While this method shows nice results with the said datasets, would this method scale to more complex datasets?\n\nb) In L94, there is a typo at \"incorporates generates\".", "limitations": "The authors have addressed the limitations, but did not comment about any potential negative societal impact of their work.", "rating": 6, "confidence": 3, "soundness": 2, "presentation": 2, "contribution": 2, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strength: a) The proposed self-explainable method is capable of producing explanations that have some desirable properties i.e. transparency, diversity and trustworthiness. The properties are induced in the generated explanations through different loss terms used for model training.\n\nb) The model achieves better accuracy compared to related methods with 5 different datasets. That improves the usability of the proposed method as the explanations can be generated now without sacrificing the predictive performance much.\n\nWeaknesses: a) I think the technique followed here to generate the prototypes should be explained more clearly. Did you simply use the decoder to reconstruct the prototypes?\n\nb) The structure of the prototypes are not clearly understood from fig1 and the number of prototypes as well. The information that every column vector of the matrix (phi_k) are the prototypes is not articulated through fig1.", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"ethics_review_area": ["I don’t know"]}, "extra_text": {}}, {"review_id": "7EZZcuWL4LH", "reviewer": "Reviewer_8Fru", "summary": "This paper introduces ProtoVAE, a novel self-explainable architecture which learns meaningful and diverse prototypical explanations via an architecture that exploits representations learned by a mixture of VAEs. ProtoVAE is able to maintain competitive performance, even against black-box models, while learning to produce prototypical explanations outside the training set in an end-to-end fashion. This allows ProtoVAE to produce clearer local explanations than competing approaches (via attribution maps for each prototype) as well as useful prototypical images from which an explanation can be built. This method is evaluated in a diverse evaluation set of five tasks and compared against existing state-of-the-art methods such as ProtoPNet.", "questions": "After reading the paper a few times, I would appreciate it if the authors could clarify the following:\n\n1. My biggest concern with this work is that it seems that it does not generalize well in complex tasks such as CIFAR-10 (with prototypes being very blurry). Even in some simpler tasks like QuickDraw, we see that prototypes within a class are able to capture the main features of that class but any prototype-specific details remain extremely blurry (e.g., see nearest prototypes in the upper right quadrant of Figure 4). Nevertheless, line 230 indicates that less blurry prototypes can be obtained if one extracts more prototypes to be extracted per class. Is that something that can be shown in the main body of the paper? If so, I believe it would make your evaluation much stronger and qualm any doubts about the applicability of this method to non-synthetic tasks.\n2. Is it possible to include more modern methods than ProtoPNet (e.g., SITE) in the comparison during evaluation? I understand that this may entail some further experimentation to be done but it is unusual to have such empty entries in an evaluation table (as in Table 2) because the papers discussing these works did not include the same tasks as the ones used in this paper. Furthermore, it is important to get an understanding of how this method compares to other prototypical SENs that also learn sample-level prototypes (rather than patch prototypes as ProtoPNet does). This is not a blocking request as the paper does a good job at evaluating ProtoVAE against ProtoPNet but it would again make your evaluation stronger.\n\nIn terms of suggestions for the presentation, the paper is extremely well-written and clear. Nevertheless, the following typos may be worth addressing for a final version:\n\n1. Line 101: “… takes as input features of several hidden layers” should probably be “… takes as input features several hidden layers”.\n2. Line 121: “embedding” should be “embeddings”.\n3. Line 130: $g(z_i)$ should be $g(\\mathbf{z}_i)$ for notational consistency.\n4. In several parts of the paper and the supplementary material, “supplementary” is used instead of “supplementary material”.", "limitations": "The paper does a good job at discussing some of the crucial limitations of their method. For example, it discusses how some tasks result in blurry prototypes in the experimentation section and briefly elaborates on the limitations of fixing the number of prototypes per class in the conclusion. The paper also discusses possible negative societal impacts in its supplementary material.", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 4, "contribution": 3, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Thank you for the very insightful and interesting work presented in this paper. After carefully reading the paper, I believe that its main strengths are:\n\n1. It tackles an important gap in prototypical XAI methods (that of designing a transparent, diverse, and trustworthy end-to-end prototypical model) and does a good job at motivating how this is reached through careful definitions for the model’s desiderata.\n2. The evaluation of the method is broad. It evaluates the proposed method both quantitative to qualitative across tasks with a varying degree of difficulty.\n3. The paper is extremely well-written and the presentation is clear.\n4. The introduction of a VAE-like architecture for learning out-of-training-set prototypes is novel and potentially interesting for future work to further explore.\n\nIn contrast, I believe the following elements are weaknesses of the presented work:\n\n1. Against common practice, some results do not have error bars in them (e.g., SITE results in Table 2 and all results in Table 3).\n2. In SVHN, evaluation results for most baselines are missing. Similarly, when evaluating ProtoVAE against SITE (a potentially important contender for the method proposed), evaluation results are missing for 3 of the selected tasks.\n3. The evaluation of the method in real-world complex tasks shows that its prototypes are hard to interpret: prototypes learned in CIFAR-10 (as seen in Figure 2) seem to be extremely blurry and hard to interpret (as they seem to be some sort of mean of samples in that class). This casts some doubt on the applicability of this method to real-world distributions.", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"ethics_review_area": ""}, "extra_text": {}}]

Author rebuttal:

Decision: Accept