Paper: Prototype Generation: Robust Feature Visualisation for Data Independent Interpretability (ICLR-2024)

Reviewer comment (Experiments):
While the authors made an attempt to formalize the notion of "path similarity" between a generated prototype and a "natural" image, this notion is not used to generate prototypes.

Author response:
We thank the reviewer for their comments and thoughts. 

While the notion of path similarity is not used to generate prototypes directly, we used this to test our regularisation hyperparameters as shown in Appendix A.

We have experimented with using path similarity as a metric to generate prototypes with limited success pointing to redundancies in oversubscribing to this objective.

Reviewer comment (Evaluation):
My main concern lies with the qualitative nature of the findings presented in the paper and the resulting ambiguity in the explanations. The authors claim that “the academic gown prototype in Figure 6a shows a lighter-skinned face prominently”. Here 6a refers to the image of the prototype for ‘academic gown’. Just by looking at that image, I cannot definitively say that the light colored patch is definitely a human face. Without looking at images in the dataset, one cannot assume that there is no other object that shows up in a prototype. Since one of the main claims of the paper is that their method allows us to explain models without combing through the dataset, I’m unconvinced about the practical gain the authors claim their method provides.

Author response:
Although the prototypes require some subjective reasoning, they do enable development of hypotheses that can be effectively tested. Looking through a dataset of millions of examples is prohibitively impossible and even in the case of a reasonably sized dataset, hypotheses built purely on dataset analysis are boundless. Our prototypes help bound this process and guide us to actionable insights by directly querying the model.

Reviewer comment (Evaluation):
The second experiment in the paper is about the ‘mortarboard’ (graduation cap) class where the authors look at the prototype and conclude that the model would get confused if they saw faces with caps in the image instead of just caps. Again, the prototype images only contain light colored patches so it is impossible to conclude that they are faces with certainty unless you are explicitly looking for faces in the first place. Thus, in this case, the user already had an explanation in mind and they are only looking at the model for a confirmation of their pre-existing biases rather than a novel explanation.

Author response:
Although the prototypes require some subjective reasoning, they do enable development of hypotheses that can be effectively tested. Looking through a dataset of millions of examples is prohibitively impossible and even in the case of a reasonably sized dataset, hypotheses built purely on dataset analysis are boundless. Our prototypes help bound this process and guide us to actionable insights by directly querying the model.

Reviewer comment (Experiments):
Experiments not comprehensive enough: The authors report experiments on two classes from ImageNet (no justification was given for why these two were chosen specifically). To make sure their method works, we would need to know if this method generalizes to more classes.

Author response:
Our comprehensive experiments consist of comparing spearman similarity between prototype activations and natural image activations. Since we show high similarity we go ahead and show a case study of gathering insights from prototypes for two classes that seem interesting in our subjective analysis.

Reviewer comment (Experiments):
Why have a single prototype for each class? I would imagine having multiple prototypes would enhance coverage over the full diversity of class images.

Author response:
Although multiple prototypes would cover all possible features that would contribute towards a particular class prediction, in this paper we were interested in exploring the potential of prototypes to cover the most relevant features of a class. We are very interested in adding robust diversity functionality in the prototype generation process and we have included this in our expanded future work section.

Reviewer comment (Evaluation):
What about outliers? When we take a mean over activations, we would effectively start to ignore outliers in our explanations.

Author response:
While it is true that taking the mean of activations would subdue outliers, this does not stop us from using the mean of activations to meaningfully compare our prototypes that aim to capture the most useful features in the average case with natural images. The notion of path similarity introduced by Geirhos et al 2023 [1] does fall into this trap of not appropriately addressing outliers.

Reviewer comment (Evaluation):
Wouldn’t using the high frequency loss (no large difference in adjacent pixel values) smooth out the prototype image and, thus, cause the prototype to look more diffuse? I’m not sure if this is a good or a bad thing so I’m curious to know what the authors think.

Author response:
The high frequency loss would lead to diffuse prototypes in isolation but it is an important objective to have in conjunction with all the other objectives in play during prototype generation. In our implementation, the high frequency loss helps smooth out adversarial jaggedness and artifacts rather than promote diffusion.