Venue: NeurIPS 2022 Conference

Reviews:

final_score: rating: 6: Weak Accept: Technically solid, moderate-to-high impact paper, with no major concerns with respect to evaluation, resources, reproducibility, ethical considerations.
review_content: summary: The authors presented a nice piece of work in this paper towards improving deep model explainability through concept learning with prototypes. The proposed method is designed to generate explanations that are transparent, diverse and trustworthy. The predictive performance of the proposed method is better compared to related methods with various digit as well as natural image datasets.
strengths_and_weaknesses: Strength: a) The proposed self-explainable method is capable of producing explanations that have some desirable properties i.e. transparency, diversity and trustworthiness. The properties are induced in the generated explanations through different loss terms used for model training.

b) The model achieves better accuracy compared to related methods with 5 different datasets. That improves the usability of the proposed method as the explanations can be generated now without sacrificing the predictive performance much.

Weaknesses: a) I think the technique followed here to generate the prototypes should be explained more clearly. Did you simply use the decoder to reconstruct the prototypes?

b) The structure of the prototypes are not clearly understood from fig1 and the number of prototypes as well. The information that every column vector of the matrix (phi_k) are the prototypes is not articulated through fig1.
questions: Questions: a) VAEs are not well known for reconstruction quality of bigger size images or complex images with more prototypes compared to the 5 datasets used for experimentation in this paper. While this method shows nice results with the said datasets, would this method scale to more complex datasets?

b) In L94, there is a typo at "incorporates generates".
limitations: The authors have addressed the limitations, but did not comment about any potential negative societal impact of their work.
review_title: null

final_score: rating: 6: Weak Accept: Technically solid, moderate-to-high impact paper, with no major concerns with respect to evaluation, resources, reproducibility, ethical considerations.
review_content: summary: This paper introduces ProtoVAE, a novel self-explainable architecture which learns meaningful and diverse prototypical explanations via an architecture that exploits representations learned by a mixture of VAEs. ProtoVAE is able to maintain competitive performance, even against black-box models, while learning to produce prototypical explanations outside the training set in an end-to-end fashion. This allows ProtoVAE to produce clearer local explanations than competing approaches (via attribution maps for each prototype) as well as useful prototypical images from which an explanation can be built. This method is evaluated in a diverse evaluation set of five tasks and compared against existing state-of-the-art methods such as ProtoPNet.
strengths_and_weaknesses: Thank you for the very insightful and interesting work presented in this paper. After carefully reading the paper, I believe that its main strengths are:

1. It tackles an important gap in prototypical XAI methods (that of designing a transparent, diverse, and trustworthy end-to-end prototypical model) and does a good job at motivating how this is reached through careful definitions for the model’s desiderata.
2. The evaluation of the method is broad. It evaluates the proposed method both quantitative to qualitative across tasks with a varying degree of difficulty.
3. The paper is extremely well-written and the presentation is clear.
4. The introduction of a VAE-like architecture for learning out-of-training-set prototypes is novel and potentially interesting for future work to further explore.

In contrast, I believe the following elements are weaknesses of the presented work:

1. Against common practice, some results do not have error bars in them (e.g., SITE results in Table 2 and all results in Table 3).
2. In SVHN, evaluation results for most baselines are missing. Similarly, when evaluating ProtoVAE against SITE (a potentially important contender for the method proposed), evaluation results are missing for 3 of the selected tasks.
3. The evaluation of the method in real-world complex tasks shows that its prototypes are hard to interpret: prototypes learned in CIFAR-10 (as seen in Figure 2) seem to be extremely blurry and hard to interpret (as they seem to be some sort of mean of samples in that class). This casts some doubt on the applicability of this method to real-world distributions.
questions: After reading the paper a few times, I would appreciate it if the authors could clarify the following:

1. My biggest concern with this work is that it seems that it does not generalize well in complex tasks such as CIFAR-10 (with prototypes being very blurry). Even in some simpler tasks like QuickDraw, we see that prototypes within a class are able to capture the main features of that class but any prototype-specific details remain extremely blurry (e.g., see nearest prototypes in the upper right quadrant of Figure 4). Nevertheless, line 230 indicates that less blurry prototypes can be obtained if one extracts more prototypes to be extracted per class. Is that something that can be shown in the main body of the paper? If so, I believe it would make your evaluation much stronger and qualm any doubts about the applicability of this method to non-synthetic tasks.
2. Is it possible to include more modern methods than ProtoPNet (e.g., SITE) in the comparison during evaluation? I understand that this may entail some further experimentation to be done but it is unusual to have such empty entries in an evaluation table (as in Table 2) because the papers discussing these works did not include the same tasks as the ones used in this paper. Furthermore, it is important to get an understanding of how this method compares to other prototypical SENs that also learn sample-level prototypes (rather than patch prototypes as ProtoPNet does). This is not a blocking request as the paper does a good job at evaluating ProtoVAE against ProtoPNet but it would again make your evaluation stronger.

In terms of suggestions for the presentation, the paper is extremely well-written and clear. Nevertheless, the following typos may be worth addressing for a final version:

1. Line 101: “… takes as input features of several hidden layers” should probably be “… takes as input features several hidden layers”.
2. Line 121: “embedding” should be “embeddings”.
3. Line 130: $g(z_i)$ should be $g(\mathbf{z}_i)$ for notational consistency.
4. In several parts of the paper and the supplementary material, “supplementary” is used instead of “supplementary material”.
limitations: The paper does a good job at discussing some of the crucial limitations of their method. For example, it discusses how some tasks result in blurry prototypes in the experimentation section and briefly elaborates on the limitations of fixing the number of prototypes per class in the conclusion. The paper also discusses possible negative societal impacts in its supplementary material.
review_title: null

final_score: rating: 6: Weak Accept: Technically solid, moderate-to-high impact paper, with no major concerns with respect to evaluation, resources, reproducibility, ethical considerations.
review_content: summary: The authors introduce ProtoVAE, a VAE-based framework for interpretable classification. Specifically, the authors propose to learn class-specific prototypes in the VAE's latent space. For a given sample, the distance to those prototypes is then used by a linear layer to classify that sample. Since the authors additionally train a decoder, the learnt prototypes can be visualised in input space, which makes the prototypes and thus the classification pipeline easily interpretable.

To evaluate their approach, the authors compare to various other self-explainable models with a focus on ProtoPNet and discuss advantages w.r.t. classification accuracy and interpretability of their proposed approach.
strengths_and_weaknesses: ## Strengths 
- The proposed approach is conceptually simple, but nevertheless yields better accuracy than other self-explainable models on the evaluated datasets and allows for a direct visualisation of the used prototypes in input space. 
- By using a VAE, the proposed approach yields an easily interpretable latent space and the model can interpolate between different input samples.
- Further, the model allows for both global explanations (the learnt prototypes) as well as local explanations (distances to learnt prototypes of a given sample).
- The manuscript is generally well-written and easy to read.


## Weaknesses

- ProtoPNet was developed for a more complex dataset (larger images, fine-grained classification). In contrast, the proposed approach evaluates only on much simpler datasets — a discussion on potential scalability issues is missing. 
- Continuing the previous point: the interpretability of the proposed model crucially depends on the quality of the decoder. However, on CIFAR10 the prototypes already become much less interpretable (Fig. 6, dog prototype or Fig. 2 CIFAR10 prototypes). Can we even hope for such an approach to be scaled to 'relevant' datasets in terms of interpretability?
- Apart from the interpretability, the accuracy on CIFAR10 also seems very limited. I would appreciate if the authors could comment on the major limiting factors w.r.t. classification performance. A minor point related to this: the comparison to the black-box encoders is confusing (Table 2). While I understand that the authors introduce this as a relevant comparison in Definition 3 (i), this comparison seems to implicitly suggest that ProtoVAEs outperform black-box models. However, 84.6% accuracy on CIFAR10 is, of course, far from what DNNs can achieve.
- A core feature of the ProtoPNet is to learn and compare to *local* prototypes ("This looks like that"). In contrast, the ProtoVAE is designed to learn full-image prototypes, which limits its flexibility. I would appreciate if the authors could comment on learning local prototypes, especially given that the comparison to ProtoPNet is so central in the manuscript. 
- While I understand the motivation for the local explanations according using LRP, I am unsure about their added benefit w.r.t. the interpretability of the model decision. I would appreciate if the authors could discuss this in more detail, specifically because it seems difficult to parse those explanations: what do we learn e.g., from the heatmaps in Fig. 4 (top row)? (minor: and what is the colour coding exactly? red positive, blue negative?) Further, LRP is a general framework that encompasses many potential rules for backpropagating relevance scores. It is unclear from the text which rules were chosen.


Minor:
- There are other generative classification approaches, maybe most notably normalizing flows (e.g., Mackowiak et al. 2021), that might be relevant for a more complete contextualisation. 

questions: Please see the list of weaknesses, I would appreciate if the authors could address my concerns.

Specifically:
- Can we hope to scale this approach to larger datasets? How would this approach compare to ProtoPNet on, e.g., CUB?
- How does the interpretability of ProtoVAE compare to ProtoPNet if the prototypes cannot be visualised anymore (e.g., prototypes are already difficult to interpret on CIFAR10)?
- What do we learn from the LRP-based explanations? The heatmaps do not seem to be easily interpretable and even somewhat conflicting when it comes to highlighting the similarity to the different prototypes (Fig. 4, top row).
- The authors say that the blurriness of the CIFAR prototypes can be reduced by a larger number of prototypes. Is there experimental evidence for this? What is the impact of the L2 reconstruction loss on this? 
limitations: To me, there are no obvious negative societal impacts that the authors would need to address in more detail. However, as expressed in the previous points, I believe the manuscript would benefit significantly from a more detailed discussion of the limitations w.r.t. scalability and current bottlenecks.
review_title: null

Meta-review: metareview: The paper is of good quality and presents an interesting approach to interpretability in a clear manner. Generally, the paper is very well written and due to extensive ablations and experiments offers many insights into the relevant aspects of the proposed approach. At the current stage the method might not be fully practical but might inspire further research down the line that overcomes said shortcomings. On the flip side, one drawback of the paper is relying on variational auto-encoder, which itself doesn't work nicely as a generative model for complex (natural) datasets. There are not many results on real-world complex tasks in the paper, and adding them would definitely add value. While there are some shortcomings, I believe it is still valuable for the research community. The authors have provided clarification to most major concerns in a convincing way. Overall, I suggest for the acceptance of the paper. Please improve the final version with the content from reviewer responses. 
confidence: Certain
Decision: Accept