Interpretable Image Classification with Adaptive Prototype-based Vision Transformers

Authors: chiyu  ma
jon  donnelly
wenjun  liu
soroush  vosoughi
cynthia  rudin
chaofan  chen
Venue: NeurIPS_cc_2024_Conference
Published: 2024-05-12 05:31:12

Abstract: We present ProtoViT, a method for interpretable image classification combining deep learning and case-based reasoning. This method classifies an image by comparing it to a set of learned prototypes, providing explanations of the form ``this looks like that.'' In our model, a prototype consists of **parts**, which can deform over irregular geometries to create a better comparison between images. Unlike existing models that rely on Convolutional Neural Network (CNN) backbones and spatially rigid prototypes, our model integrates Vision Transformer (ViT) backbones into prototype based models, while offering spatially deformed prototypes that not only accommodate geometric variations of objects but also provide coherent and clear prototypical feature representations with an adaptive number of prototypical parts. Our experiments show that our model can generally achieve higher performance than the existing prototype based models. Our comprehensive analyses ensure that the prototypes are consistent and the interpretations are faithful.

Main paper: 
 introduction: 
 With the expanding applications of machine learning models in critical and high-stakes domains like healthcare [4, 14, 44, 52, 51] , autonomous vehicles [32] , finance [47] and criminal justice [5] , it has become crucial to develop models that are not only effective but also interpretable by humans. This need for clarity and accountability has led to the emergence of prototype networks. These networks combine the capabilities of deep learning with the clarity of case-based reasoning, providing understandable outcomes in fine-grained image classification tasks. Prototype networks operate by dissecting an image into informative image patches and comparing these with prototypical features established during their training phase. The model then aggregates evidence of similarities to these prototypical features to draw a final decision on classification.Existing prototype-based models [10, 4, 13, 26, 35, 36, 46, 27, 45, 8, 7] , which are inherently interpretable [34] (i.e., the learned prototypes are directly interpretable, and all calculations can be visibly checked), are mainly developed with convolutional neural networks (CNNs). As vision transformers (ViTs) [15, 40, 41] gain popularity and inspire extensive applications, it becomes crucial to investigate how prototype-based architectures can be integrated with vision transformer backbones. Though a few attempts to develop a prototype-based vision transformer have been made [50, 49, 22] , these methods do not provide inherently interpretable explanations of the models' reasoning because they do not project the learned prototypical features to examples that actually exist in the dataset. as ProtoPNet [10] , TesNet [46] and ProtoConcept [26] , use spatially rigid prototypical features (such as a rectangle), which cannot account for geometric variations of an object. Such rigid prototypical features can be ambiguous as they may contain multiple features in one bounding box (see top row of Fig. 1 for an example). Deformable ProtoPNet [13] deforms the prototypes into multiple pieces to adjust for the geometric transformation. However, Deformable ProtoPNet utilizes deformable convolution layers that rely on a continuous latent space to derive fractional offsets for prototypical parts to move around, and thus are not suitable for models like ViTs which output discrete tokens as latent representations. Additionally, the prototypes learned by Deformable ProtoPNet tend to be incoherent (see Fig. 1 , middle row).Addressing the gaps in current prototype-based methods (see Table 1 ), we propose the prototypebased vision transformer (ProtoViT), a novel architecture that incorporates a ViT backbone and can adaptively learn inherently interpretable and geometrically variable prototypes of different sizes, without requiring explicit information about the shape or size of the prototypes. We do so using a novel greedy matching algorithm that incorporates an adjacency mask and an adaptive slots mechanism. We provide global and local analysis, shown in Fig. 4 , that empirically confirm the faithfulness and coherence of the prototype representations from ProtoViT. We show through empirical evaluation that ProtoViT achieves state-of-the-art accuracy as well as excellent clarity and coherence. 
related work: 
 Posthoc explanations for CNNs like activation maximization [16, 28] , image perturbation [17, 20] , and saliency visualizations [3, 38, 39] fall short in explaining the reasoning process of deep neural networks because their explanations are not necessarily faithful [33, 2] . In addition, numerous efforts have been made to enhance the interpretability of Vision Transformers (ViTs) by analyzing attention weights [1, 42] , and other studies focus on understanding the decision-making process through gradients [18, 19] , attributions [9, 53] , and redundancy reduction techniques [30] . However, because of their posthoc nature, the outcomes of these techniques can be uncertain and unreliable [2, 25] .In contrast, prototype-based approaches offer a transparent prediction process through case-based reasoning. These models compare a small set of learned latent feature representations called prototypes (the "cases") with the latent representations of a test image to perform classification. These models are inherently interpretable because they leverage comparisons to only well-defined cases in reasoning. The original Prototypical Part Network (ProtoPNet) [10] employs class-specific prototypes, allocating a fixed number of prototypes to each class. Each prototype is trained to be similar to feature patches from images of its own class and dissimilar to patches from images of other classes. Each prototype is also a latent patch from a training image, which ensures that "cases" are well-defined in the reasoning process. The similarity score from the test image to each prototype is added as positive evidence for each class in a "scoresheet," and the class with the highest score is the predicted class. The Transparent Embedding Space Network (TesNet) [46] refines the original ProtoPNet by utilizing a cosine similarity metric to compute similarities between image patches and prototypes in a latent space. It further introduces new loss terms to encourage the orthogonality among prototype vectors of the same class and diversity between the subspaces associated with each class. Deformable ProtoPNet [13] aims to decompose the prototypical parts into smaller sub-patches and use deformable convolutions [12, 54] to capture pose variations. Other works [35, 27, 36] move away from class-specific prototypes, which reduces the number of prototypes needed. This allows prototypes to represent a similar visual concept shared in different classes. Our work is similar to these works that define "cases" as latent patch representations by projecting the trained class-specific prototypes to the closest latent patches.As an alternative to the closest training patches, ProtoConcept [26] defines the cases as a group of visualizations in the latent spaces bounded by a prototypical ball. Although they are not projected, the visualizations of ProtoConcept are fixed to the set of training patches falling in each prototypical ball, which establishes well defined cases for each prototype. On the other hand, works such as ProtoPFormer [50] do not project learned prototypes to the closest training patches because they observed a performance degradation after projection. The degradation is likely caused by the fact that the prototypes are not sufficiently close to any of the latent patches. The design of a "global branch" that aims to learn prototypical features from class tokens also raises concerns about interpretability, as visualizing an arbitrary trained vector (class token) does not offer any semantic connection to the input image. On the other hand, work such as ViTNet [22] also lack details about how the "cases" are established, yielding concerns about the interpretability of the model. Without a mechanism like prototype projection [10] or prototypical concepts [26] to enable visualizations, prototypes are just arbitrary learned tensors in the latent space of the network, with no clear interpretations. Visualizing nearby examples alone cannot explain the model's reasoning, since the closest patch to a prototype can be arbitrarily far away without the above mechanisms. Simply adding a prototype layer to an architecture without well defined "cases" in the reasoning process does not make the new architecture more interpretable.Our work is also related to INTR [31] , which trains a class specific attention query and inputs it to a decoder to localize the patterns in an image with cross-attention. In contrast, our encoder-only model, through a different reasoning approach, learns patch-wise deformed features that are more explicit, semantically coherent, and provides more detail about the reasoning process than attention heatmaps [21, 48, 6] -it shows how the important pixels are used in reasoning, not just where they are. 
methods: 
 We begin with a general introduction of the architecture followed by an in-depth exploration of its components. We then delve into our novel training methodology that encourages prototypical features to capture semantically coherent attributes from the training images. Detailed implementations on specific datasets are shown in the experiment sections.Fig. 2 shows an overview of the ProtoVit architecture. Our model consists of a feature encoder layer f , which is a pretrained ViT backbone that computes a latent representation of an image; a greedy matching layer g, which compares the latent representation to learned prototypes to compute prototype similarity scores; and an evidence layer h, which aggregates prototype similarity scores into a classification using a fully connected layer. We explain each of these in detail below. Let H × W × C be the shape of an input image x, where H, W, C are the height, width and number of channels of the image respectively. The feature encoder layer f is a ViT backbone, which first splits the input images into N units of L × L × C sized patches (such as the Chestnut Sided Warbler in Fig. 2 ), where L is the height and width of the image patch. The feature encoder layer f then flattens and encodes the image patches into a set of latent feature tokens z f defined later in Eq. 1, where each latent feature token z i f ∈ R d and d is the hidden dimension of the model. The network aims to learn m prototypes P = {p j } m j=1 , where each p j ∈ R K×d is composed of K sub-prototype vectors p k j ∈ R d . The greedy matching layer g then finds the most similar latent feature token z i f , measured by cosine similarity, of the input image to each sub-prototype p k j without replacement (i.e., if p 1 j matches z 1 f , p 2 j cannot match with z 1 f ). We further introduce an adjacency mask A to ensure that, for each prototype p j , its sub-prototypes p k j are geometrically contiguous. We then introduce our adaptive slots mechanism, which decides whether each prototype p j should include each sub-prototype p k j based on its coherence to the other sub-prototypes and its influence on model performance, allowing the model to learn prototypes with a dynamic number of sub-prototypes. Finally, the evidence layer h computes the logit values for each class based on the summed cosine similarity score g greedy pj for each prototype p j . The logit values are then normalized by a softmax function to make predictions. Intuitively, the prototypes, which are later projected to the closest latent feature tokens, can be viewed as the most representative features of each class that the model found among the training examples. The model performs classification based on the input's similarity to these key features.Given an input image x ∈ R H×W ×C , the feature encoder layer first splits the image into N unit patches x patch each of shape L × L × C. It then flattens the unit patches and projects them to the embedding space as x Embed ∈ R N ×d , which is then prepended with a trainable class token x class ∈ R d , and passed into the Multi-Head Attention layers along with a learnable position embedding E pos ∈ R (N +1)×d . The ViT backbone then outputs the encoded tokens asz := [z class ; z 1 patch ; z 2 patch ; • • • ; z N patch ], where z ∈ R (N +1)×d . In this sense, each of the output patch tokens z i patch is the latent representation of the corresponding unit patch x i patch . The class token can be viewed as a way to approximate the weighted sum over patch tokens, enabling an image-level representation. It is, thus, difficult to visualize the class token and therefore unsuitable for comparisons to prototypes. Drawing inspiration from the idea of focal similarity [36] , which involves calculating the maximum prototype activations minus the mean activations to achieve more focused representations, we take the difference between patch-wise features and the image-level representation. In doing so, we aim to similarly produce more salient visualizations. Specifically, we define the feature token z f by taking the difference between each patch token and the image-level representation. Thus, latent feature tokens can be written as:EQUATIONBy this design, we not only encode richer semantic information within the latent feature tokens, but also enable the application of prototype layers to various ViT backbones that contains a class token.An ablation study shows that model performance drops substantially when removing the class token from the feature encoding (see Appendix Sec. E.1) .The greedy matching layer g integrates three key components: a greedy matching algorithm, adjacency masking, and an adaptive slots mechanism, as illustrated in Fig. 2 .The greedy matching layer contains m prototypes P = {p j } m j=1 , where each p j consists of K sub-prototypes p k j that have the same dimension as each latent feature token z i f . Each sub-prototype p k j is trained to be semantically close to at least one latent feature token. To measure the closeness of the sub-prototype p k j and latent feature token z i f , we use cosine similarity cos(z i f , p k j ) =z i f T p k j ∥z i f ∥2∥p k j ∥2, which has range -1 to 1. The overall similarity between prototype p j and the latent feature tokens z f is then computed as the sum across the sub-prototypes of the cosine similarities, which has a range from -K to K.Greedy Matching Algorithm: We have not yet described how the token z i f to which p k j is compared is selected. In contrast to prior work, we do not restrict that sub-prototypes have fixed relative locations. Rather, we perform a greedy matching algorithm to identify and compare each of the K sub-prototypes p k j of p j to any K non-overlapping latent feature tokens. To be exact, for a given prototype p j with K sub-prototypes, we iteratively identify the sub-prototype p k j and latent feature token z i f that are closest in cosine similarity, "match" z i f with p k j , and remove z i f and p k j from the next iteration, until all K pairs are found. This is illustrated in Fig. 2 , and more details can be found in Appendix Sec. C.Adjacency Masking: Without any restrictions, this greedy matching algorithm can lead to many sub-prototypes that are geometrically distant from each other. Assuming that the image patches representing the same feature are within r ∈ R spatial units of one another in horizontal, vertical and diagonal directions, we introduce the Adjacency Mask A to temporarily mask out latent feature tokens that are more than r positions away from a selected sub-prototype/latent feature token pair in all directions. Within each iteration k of the greedy matching algorithm, the next pair can only be selected from the latent feature tokens,z A k j = A({p k-1 j , z i f k-1 }; z f ; r), that are within r positions of the last selected pair. An example of how the adjacency mask with r = 1 works is illustrated in Fig. 2 . Incorporating adjacency masking with the greedy matching algorithm, we thus find K non-overlapping and adjacent sub-prototype-latent patch pairs.Adaptive Slots Mechanism: Because not all concepts require K sub-prototypes to represent, we introduce the the adaptive slots mechanism S. The adaptive slots mechanism consists of learnable vectors v ∈ R m×K and a sigmoid function with a hyperparameter temperature τ . We chose a sufficiently large value for τ to ensure that the sigmoid function has a steep slope. The vectors v are sent to the sigmoid function to approximate the indicator function as 1{Include p k j } = Sigmoid(v k j , τ ). Each 1{Include p k j } is an approximation of the indicator for whether the k-th sub-prototype will be included in the j-th prototype p j . More details can be found in Appendix Sec. D.Computation of Similarity: As described above, we measure the similarity of a selected latentpatch-sub-prototype pair using cosine similarity. We use the summed similarity across sub-prototypes to measure the overall similarity for prototype p j with K selected non-overlapping latent feature tokens. As pruning out some sub-prototypes for a prototype p j reduces the range of the summed cosine similarity for p j , we rescale the summed similarities back to their original range by K/ k 1{include p k j }. We formally define the reweighted summed similarity function obtained from the greedy matching layer g as:EQUATIONTraining of ProtoViT has four stages: (1) optimizing layers before the last layer by stochastic gradient descent (SGD); (2) prototype slots pruning; (3) projecting the trained prototype vectors to the closest latent patches; (4) optimizing the last layer h. Note that a well-trained first stage is crucial to achieve minimal performance degradation after prototype projection. A procedure plot is illustrated in Appendix Fig. 5 .Optimization of layers before last layer: The first training stage aims to learn a latent space that clusters feature patches from the training set that are important for a class near semantically similar prototypes of that class. This involves solving a joint optimization problem over the network parameters via stochastic gradient descent (SGD). We initialize every slot indicator 1{Include p k j } as 1 to allow all sub-prototypes to learn during SGD. We initialize the last layer weight similarly to ProtoPNet [10] . The slot indicators and the final layer are frozen during this training stage. The slot indicator functions are not involved in computing cluster loss or separation loss because each sub-prototype should remain semantically close to certain latent feature tokens, regardless of its inclusion in the final computation. Since we deform the prototypes, we propose modifications to the original cluster and separation loss defined in [10] to:EQUATIONThe minimization of this new cluster loss encourages each training image to have some unit latent feature token z i f that is close to at least one sub-prototypes p k j of its own class. Similarly, by minimizing the new separation loss, we encourage every latent feature token of a training image to stay away from the nearest sub-prototype from any incorrect class. This is similar to the traditional cluster and separation loss from [10] if we define the prototypes to consist of only one sub-prototype. We further introduce a novel loss term, called coherence loss, based on the intuition that, if the sub-prototypes collectively represent the same feature (e.g., the feet of a bird), these sub-prototypes should be similar under cosine similarity. The coherence loss is defined as:EQUATIONIntuitively, the coherence loss penalizes the sub-prototypes that are the most dissimilar to the other sub-prototypes out of the K slots for prototype p j . The slots indicator function is added to the coherence loss term to prune sub-prototypes that are semantically distant from others in a prototype. Moreover, we use the orthogonality loss, introduced in previous work [46, 13] , to encourage each prototype p j to learn distinctive features. The orthogonality loss is defined as:EQUATION)where C is the number of classes. For each class l with ρ assigned prototypes, P (l) ∈ R ρ×Kd is a matrix obtained by flattened the prototypes from class l, and ρ is the number of prototypes p j for each class. I ρ is an identity matrix in the shape of ρ × ρ. Overall, this training stage aims to minimize the total loss as:L total = L CE + λ 1 L Clst + λ 2 L Sep + λ 3 L Coh + λ 4 L Orth (6)where L CE is the cross entropy loss for classification and λ 1 , λ 2 , λ 3 , λ 4 are hyper-parameters.Slots pruning: In this stage, our goal is to prune the sub-prototypes p k j that are dissimilar to the other sub-prototypes for each prototype. Intuitively, we aim to remove these sub-prototypes because sub-prototypes that are not similar will lead to prototypes with an inconsistent, unintuitive semantic meaning. We freeze all of the parameters in the model, except the slot indicator vectors v. During this stage, we jointly optimize the coherence loss defined in Eq. 4 along with the cross entropy loss. We lower the coherence loss weight to avoid removing all the slots. Since the slot indicators are approximations of step functions using sigmoid functions with a sufficiently high temperature parameter τ , the indicator values during this phase are fractional but approach binary values close to 0 or 1. The loss in the stage is defined as:EQUATIONProjection: In this stage, we first round the fractional indicator values to the nearest integer (either 1 or 0), and freeze the slot indicators' values. Then, we project each prototype p j to the closest training image patch measured by the summed cosine similarity defined in Eq. 2, as in [10] . Because the latent feature tokens from the ViT encoder correspond to image patches, we do not need to use up-sampling techniques for visualizations, and the prototypes visualized in the bounding boxes represent the exact corresponding latent feature tokens that the prototypes are projected to.As demonstrated in Theorem 2.1 from ProtoPNet [10] , if the prototype projection results in minimal movement, the model performance is unlikely to change because the decision boundary remains largely unaffected. This is ensured by minimizing cluster and separation loss, as defined in Eq. 3. In practice, when prototypes are sufficiently well-trained and closely clustered to certain latent patches, the change in model performance after the projection step should be minimal. Additionally, this step ensures that the prototypes are inherently interpretable, as they are projected to the closest latent feature tokens, which have corresponding visualizations and provide a pixel space explanation of the "cases" in the model's reasoning process.Optimization of last layers: Similar to other existing works [10, 46] , we performed a convex optimization on the last evidence layer h after performing projection, while freezing all other parameters. This stage aims to introduce sparsity to the last layer weights W by penalizing the 1-norm of layer weights W b,l (initially fixed as -0.5) associated with the class b for the l-th class prototype p l , where l ̸ = b. By minimizing this loss term, the model is encouraged to use only positive evidence for final classifications. The loss term is defined as:EQUATION4 Experiments 
case study 1: bird species identification: 
 To demonstrate the effectiveness of ProtoVit, we applied it to the cropped Caltech-UCSD Birds-200-2011 (CUB 200-2011) dataset [43] . This dataset contains 5,994/5,794 images for training and testing 2). The quantitative results for the ablation can be found in Appendix Table . 6. We assigned the algorithm to choose 10 class-specific prototypes for each of the 200 classes. Each of the prototypes is composed of 4 sub-prototypes. Each set of sub-prototypes was encouraged to learn the 'key' features for its corresponding class through only the last layer weighting, and the 4 sub-prototypes were designed to collectively represent one key feature for the class. More discussion on different choices of K can be found in Appendix F. For more details about hyperparameter settings and training schedules, please refer to Appendix Sec. A and Appendix Sec. B respectively. Details and results of user studies regarding the interpretability of ProtoViT can be found in Appendix. Sec. K.The performance of our ProtoViT with CaiT and DeiT backbones is compared to that of existing work in Table 2 , including results from prototype-based models using DenseNet161 (CNN) backbones.Integrating ViT (Vision Transformer) backbones with a smaller number of parameters into prototypebased algorithms significantly enhances performance compared to those using CNN (Convolutional Neural Network) backbones, particularly on more challenging datasets. Compared with other prototype-based models that utilize ViT backbones, our model produces the highest accuracy, outperforming even the black box ViTs used as backbones, while offering interpretability. We provide accuracy using an ImageNet pretrained Densenet-161 to match the ImageNet pretraining used in all transformer models.Fig. 3 shows the reasoning process of ProtoViT for a test image of a Barn Swallow. Example visualizations for the classes with the top two highest logit scores are provided in the figure. Given this test image x, our model compares its latent feature tokens against the learned sub-prototypes p k j through the greedy matching layer g. In the decision process, our model uses the patches that have the most similar latent feature tokens to the learned prototypes as evidences. In the example, our model correctly classifies a test image of a Barn Swallow and thinks the Tree Swallow is the second most likely class based on prototypical features. In addition to this example reasoning process, we conducted local and global analyses (shown in Fig. 4 ) to confirm the semantic consistency of prototypical features across all training and testing images, where local and global analyses are For example, the first prototype compared to the white breasted kingfisher seems to identify the bird's blue tail, and is compared to the tail of the bird in the test image. Further, our global analysis shows that each prototype consistently activates on a single, meaningful concept. For example, the prototype in the first row of the global analysis consistently highlights the black face of the common yellowthroat at a variety of scales and poses. Taken together, these analyses show that the prototypes of ProtoViT have strong, consistent semantic meanings. More examples of the reasoning process and analysis can be found in Appendix Sec. H and Sec. J respectively.Vision Transformers (ViTs) use an attention mechanism that blends information from all image patches, which may prevent the latent token at a position from corresponding to the input token at that In this case study, we apply our model to car identification. We trained our model on the Stanford Car dataset [24] of 196 car models. More details about the implementation can be found in Appendix. Sec. L. The performance with baseline models can be found in Table . 2. Example visualizations and analysis can be found in Appendix L. We find that ProtoViT again produces superior accuracy and strong, semantically meaningful prototypes on the Cars dataset. 
limitations: 
 While we find our method can offer coherent and consistent visual explanations, it is not yet able to provide explicit textual justification to explain its reasoning process. With recent progress in large vision-language hybrid models [23, 11, 29] , a technique offering explicit justification for visual explanations could be explored in future work. It is important to note that in some visual domains such as mammography or other radiology applications, features may not have natural textual descriptions -thus, explicit semantics are not yet possible, and a larger domain-specific vocabulary would first need to be developed. Moreover, as discussed in Sec. 4.1.3, our model is not completely immune to location misalignment, meaning the learned prototypical features may be difficult to visualize in local patches. However, this issue is not unique to our approach; CNN-based models face the same challenge. As layers deepen, the receptive field often expands to cover the entire image, leading to the same problem encountered with attention mechanisms in vision transformers. 
conclusion: 
 In this work, we presented an interpretable method for image classification that incorprates ViT backbones with deformed prototypes to explain its predictions (this looks like that). Unlike previous works in prototype-based classification, our method offers spatially deformed prototypes that not only account for geometric variations of objects but also provide coherent prototypical feature representations with an adaptive number of prototypical parts. While offering inherent interpretability, our model empirically outperform the previous prototype based methods in accuracy. Question: Does the paper specify all the training and test details (e.g., data splits, hyperparameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?Answer: [Yes]Justification: We have listed all the settings in the appenix for paramtere settings and training schedules. The data splits are decribed under each case study section in the main paper.Guidelines:• The answer NA means that the paper does not include experiments.• The experimental setting should be presented in the core of the paper to a level of detail that is necessary to appreciate the results and make sense of them. • The full details can be provided either with the code, in appendix, or as supplemental material. 
experiment statistical significance: 
 Question: Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?Answer: [Yes] Justification: All of our results are listed with error bars.Guidelines:• The answer NA means that the paper does not include experiments.• The authors should answer "Yes" if the results are accompanied by error bars, confidence intervals, or statistical significance tests, at least for the experiments that support the main claims of the paper. • The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall run with given experimental conditions). • The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.) • The assumptions made should be given (e.g., Normally distributed errors).• It should be clear whether the error bar is the standard deviation or the standard error of the mean. • It is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have a 96% CI, if the hypothesis of Normality of errors is not verified. • For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that are out of range (e.g. negative error rates). • If the authors answer No, they should explain the special circumstances that require a deviation from the Code of Ethics. • The authors should make sure to preserve anonymity (e.g., if there is a special consideration due to laws or regulations in their jurisdiction). 10. Broader Impacts Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed? Answer: [Yes] Justification: The broader impact is discussed in the appendix and shortly discussed in conclusion section in the main paper. Guidelines:• The answer NA means that there is no societal impact of the work performed.• If the authors answer NA or No, they should explain why their work has no societal impact or why the paper does not address societal impact. • Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.• The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is a direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster. • The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology. • If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms to monitor how a system learns from feedback over time, improving the efficiency and accessibility of ML). 
safeguards: 
 Question: Does the paper describe safeguards that have been put in place for responsible release of data or models that have a high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?Answer:[NA] Justification: The paper poses no such risks.Guidelines:• The answer NA means that the paper poses no such risks.• Released models that have a high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters. • Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images. • We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make a best faith effort.12. Licenses for existing assets Question: Are the creators or original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?Answer: [Yes] Justification: They are properly cited and credited.Guidelines:• The answer NA means that the paper does not use existing assets.• The authors should cite the original paper that produced the code package or dataset.• The authors should state which version of the asset is used and, if possible, include a URL. • The name of the license (e.g., CC-BY 4.0) should be included for each asset.• For scraped data from a particular source (e.g., website), the copyright and terms of service of that source should be provided. • If assets are released, the license, copyright information, and terms of use in the package should be provided. For popular datasets, paperswithcode.com/datasets has curated licenses for some datasets. Their licensing guide can help determine the license of a dataset. • For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided. 
 
 Appendix: This section documents the specific hyper-parameters used to train ProtoViT, shown in Table 4 . We used identical parameter settings across all the backbones. We used the values suggested in prior work for terms that already existed, and used a small grid search to select the other values. No dedicated tuning procedure is necessary for these coefficients.This section documents the specific training schedule used to train ProtoViT. As discussed in Sec.3.4, the training stages are generally divided into four steps as shown in Appendix Fig. 5 . Training schedule settings can be found in Appendix Table 5 . Warm-up optimization is performed by training the feature encoder with a extremely small learning rate while training the prototype vectors for 5 epochs. Then, we increase the learning rate for the feature encoder layer for the following 10 epochs.As mentioned in Appendix Sec. A, the number of sub-prototypes being pruned during the slots pruning stage depend on the learning rate of the slots parameters and the weight of the coherence loss in the slots pruning stage. C More discussion on the greedy matching layerℒ total = ℒ CE +𝜆 1 ℒ Clst 𝜆 2 ℒ Sep 𝜆 3 ℒ Coh 𝜆 4 ℒ Orth + + +Intuitively, a non-deformed prototype p j consists of K non-overlapping sub-prototypes p k j with rigid adjacency (i.e., a rectangular shape). These non-deformed prototypes are compared with K non-overlapping latent feature tokens z i f in the same geometric shape (i.e., in a rectangle). To deform such a prototype, we could treat each sub-prototype vector p k j as an independent "prototype" to train. Then, the prototype p j is naturally deformed into K independent sub-prototypes p k j that move freely in the latent space. However, this naïve approach could lead to significant overlap among the subprototypes. Although this could be mitigated by incorporating a unit-wise orthogonality loss during the training phase to encourage dissimilarity among the sub-prototypes, similar to the orthogonality loss outlined in Eq. 5, such a unit-wise orthogonality loss is in conflict with the objectives of the coherence loss defined in Eq. 4, which encourages the sub-prototypes p k j to remain neighboring in cosine distance to preserve the semantic coherence of each prototype p j . Thus, an algorithm that could provide unit-wise and non-overlapped matching is ideal, and the greedy matching algorithm with adjacency masking meets the needs. An summary of the algorithm for greedy matching with adjacency masking is shown in Alg. 1.Require: Input latent feature tokens z, and prototypes p, 1: Compute cosine similarity between latent feature tokens and k sub-prototypes.2: Initialize masks in shape of latent feature tokens, and sub-prototypes to include all possible pairs 3: for k in K slots of sub-prototypes do 4:Mask out non-adjacent and selected patches by replacing their similarity scores with a high negative value 5:Identify the closest latent feature token for each remaining sub-prototype 6:Select the pair that has the highest cosine similarity out of all identified pairs 7:Update all masks based on the selected pair 8:Track the sequence of selected sub-prototypes for reordering 9: end for 10: Sort the pairs and the corresponding cosine similarity based on its original sub-prototype orderAs discussed in the Sec. 3.3, The adaptive slots mechanism consists of learnable vectors v ∈ R m×K and a sigmoid function with a hyper-parameter temperature τ . The vectors v are sent to the sigmoid function to approximate the indicator function as 1{Includep k j } = Sigmoid(v k j , τ ). Each 1{Include p k j }is an approximation of the indicator for whether the k-th sub-prototype will be included in the j-th prototype p j . To be specific, we define the approximated indication function 1{Include p k j } with temperature τ as:EQUATIONBy the range of the sigmoid function, we are able to approximate the indicator function with a high temperature τ . Fig. 6 shows an example of different choices of τ . As the temperature value goes up, the sigmoid function more closely approximates the behavior of the indicator function. Thus, we picked a sufficiently large τ for approximation, and later rounded the values to the closest integer (i.e 0 or 1) to serve as the actual slot indicator for each sub-prototype. This section details ablation studies on the class token, coherence loss, and adjacency mask. We performed the ablation studies with backbone DeiT-Small with r = 1 and K = 4 for the adjacency mask and number of sub-prototypes respectively.Table 6 : Ablation study on ProtoViT. Model is compared with K = 4, and r = 1 for slots and adjacency mask settings. We use Deit-Small as the backbone, and the model is trained on CUB-200-2011 dataset. For the current setting of ProtoViT, we included class token, coherence loss, and adjacency mask along with the greedy matching algorithm. Saliency test: Subtracting out the class token may have interesting implications for the class specificity of our prototypes. We expect subtracting the class token out operates as a kind of focal similarity, where each token (after taking the difference) represents the unique information available in that position which is relevant to the target class. Prototypes will, then, learn to represent unique positional information that is relevant to the predicted class. As such, we might expect prototypes to be more tightly tied to their class than they would be if they were learning with simple arbitrary tokens. We thus expect an improvement in class-specific saliency of the prototypes.To test if class token improved the saliency of prototype to its assigned class, we performed the analysis on correct and incorrect class similarity scores. If prototypes are more specific to their assigned class when including the class token, we would expectthe gap between the mean activation for prototypes of the correct class (summed cosine similarity scores as defined in Eq. 2) and the largest mean activation for prototypes of an incorrect class to be larger. To test this, we randomly pick a model with the class token and one without the class token and evaluate each model on the test set of CUB200-2011 [43] . For each test image, we average over the correct class prototypes activations, and denote this average as g abl cor for the model trained with patch-tokens only, and g cls cor for the model trained with class tokens. Similarly, we denote the largest mean activations from the incorrect class prototypes as g cls incor , and g abl incor for the models with and without class token respectively. We use δ cls and δ abl to denote the difference between the two measures for the two models. As shown in Appendix Fig. 7 , there is a clear shift in distribution between the two models for all three measures. When we include the class token, prototypes tend to activate more highly on the correct class, and the gap between the mean correct class activation and the highest mean incorrect class activation tends to be larger.We further perform one sided t-tests to test whether g cls cor -g abl cor and δ cls -δ abl are significantly greater than 0. As shown in Appendix Table . 7, including the class token statistically significantly increases each measure relative to a model trained with patch token only. That is, including the class token improves the saliency of encoded features.As introduced in Sec. 3.3, the adjacency mask is designed to ensure that sub-prototypes are geometrically adjacent. Without the adjacency mask, sub-prototypes are able to match with latent feature tokens from anywhere in the image.As indicated in Appendix Table 6 , removing the adjacency mask typically results in a slight improvement in performance. This outcome is expected since prototypes are learned for the maximum possible performance with fewer constraints. However, removing the adjacency mask tends to damage the semantics of the model's prototypes. For instance, as demonstrated in Appendix Fig. 8 , without the adjacency mask, the model might correctly identify the beak of a least tern in one image patch but erroneously relate other prototypical sub-parts to the bird's feet in another patch. Although the slim feet of the least tern indeed looks similar to the slim beak on the scale of the sub-prototype, such comparisons are not meaningful. In contrast, implementing the adjacency mask in ProtoViT effectively prevents such issues by enforcing geometric adjacency among the sub-prototypes. Furthermore, without the adjacency mask, the model may learn sub-prototypes that, though visually similar to other sub-prototypes, are fundamentally different, introducing noisy representations and disrupting the coherence of prototypical features. For instance, as demonstrated in Appendix Fig. 8 , both models misclassified a test image of a black tern as a pigeon guillemot which also has red feet. Without the adjacency mask, the prototypical feature capturing the red feet of the pigeon guillemot also mistakenly includes reflections of the red feet in the water. Although the water reflection looks similar to the actual red feet captured by the other sub-prototypes, this misrepresentation undermines the coherence of the feature representation. On the other hand, ProtoViT compares the feet of the black tern to the red feet of the pigeon guillemot. Thus, the use of an adjacency mask is essential for preserving the coherence of prototypical feature representations. As explained in Eq. 4, coherence loss is used to ensure that the sub-prototypes within each prototypical feature are semantically similar to each other. By design, this loss term helps sub-prototypes to collectively represent a coherent feature. By removing the coherence loss, the prototypical features would still contain diverse parts of features in one prototypical feature, a similar problem that the non-deformed prototypes have, even though the sub-prototypes remain geometrically adjacent. For example, as shown in Appendix Fig. 9 , a model trained without coherence loss tends to mix the wing and belly of the Myrtle Warbler into one prototypical feature, while it mixes the head and wing for the prototypical features for Painted Bunting. In comparison, training ProtoViT with coherence loss ensures that all the sub-prototypes collectively represent one prototypical feature, in this case containing the striped belly of the Myrtle Warbler, or the wing and the red lower part of the Painted Bunting.We computed the Prototype Location Change (PLC) for each ablated model using greedy matching algorithms. Following the PLC metric defined in Sacha et al. (2024) [37] , we measured the shift in the 90th percentile of prototype activations across test images. As shown in Table 6 , the combination of coherence loss and adjacent masks effectively reduces changes in prototype locations after adversarial attacks. This indicates that these mechanisms promote greater stability in the learned prototypes. Interestingly, incorporating the class token into the latent representations increases PLC. This is expected, as the class token contains more global information, which, while improving overall model performance, leads to larger shifts in prototype locations.Why K=4 : By the design of the vanilla ProtoPNet [10] and the other non-deformed CNN based prototype models [46, 15, 26] , the input images are encoded into latent features z by CNN backbones, where z ∈ R 7×7×d and d denotes the latent dimension varying by different choices of backbones. These models learn prototypical features p of shape 1 × 1 × d from the latent features. On the other hand, ViT backbones such as CaiT and DeiT encode the input images into 14 × 14 × d latent feature tokens. To be consistent with existing models, we design the model to learn prototypes of shape 2 × 2 × d from the R 14×14×d latent features, so that each prototype corresponds to the same proportion of the input image. Using the greedy matching algorithm, we can move away from learning fixed 2×2 rectangular prototypes and instead have adaptively shaped prototypes consisting of four sub-prototypes. It is worth noting that prototype-based models with CNN backbones heavily rely on up-sampling for prototype visualizations. This process can introduce errors, leading to the use of the top 5% most similar regions for visualization, which results in irregularly sized bounding boxes, as shown in the top row of Fig. 1 . In contrast, with ViT backbones, we are able to visualize the exact image patches that the prototype projected to, and thus produce more precise and accurate prototype visualizations.Other choices of K : Although we selected K= 4 to maintain consistency with other existing prototype-based models, alternative values of K are also viable. The model performance for K = 5 and K = 6 settings can be found in Appendix Table. 8 . We observed that the model performs best with r = 2 when K = 5 is used, and with r = 3 when K = 6. As shown in the table, the models with different choices of K perform similarly to the setting with K = 4. Appendix Fig. 10 and Fig. 11 show examples of reasoning process for ProtoViT(K = 5, r = 2) and ProtoViT(K = 6, r = 3) respectively. As shown in the figures, though having more sub-prototypes is helpful to capture larger featuires such as the tail of the Lincoln Sparrow and Forster Tern in Fig. 10 and Fig. 11 respectively, because of variations in scale, many features such as the beak and the head in 11 do not always need that many sub-prototypes. Local and global analysis for ProtoViT(K = 5, r = 2) and ProtoViT(K = 6, r = 3) are shown in Fig. 13 and Fig. 13 respectively. As shown in the figures, having more sub-prototypes is advantageous in representing features that are large in scale such as the white belly of Sayornis shown in the second row of global analysis in Fig. 12 , and brown pattern of Eastern Towhee in the bottom row of Fig. 13 . Both the local and global analysis again show that the prototypes of ProtoViT have strong, consistent semantic meanings.It is worth noting that prior prototype-based models could not easily support prototypes with 5 or 6 sub-prototypes. Since methods like Deformable ProtoPNet treat prototypes as convolutional features, the only way to handle prototypes with 5 sub-prototypes would be to form each prototype as a 1 by 5 dimensional convolutional filter. This would allow the model to have 5 sub-prototypes, but would enforce a very strange shape on prototypes (a horizontal line).We provide several instances of the perturbation examples, in which we mask out the region selected by each prototype, as shown in Fig. 14 . In each row, we mask out all matched locations for a prototype (middle-left column) using a white mask on the black wings and a black mask on the red parts, and check where that prototype activates after masking (shown in the leftmost column). We then confirm that the activated region for other prototypes remains reasonable when the mask from another prototype is applied (right two columns). We observed that after removing the preferred region by each prototype, it activates on another reasonable alternative (e.g., a red belly prototype might activate on a red back as a second choice).Experiment on masking prototype activation area: In this section, we provide the reasoning process of how our model misclassified a test image of a summer tanager and a slaty backed gull in Fig. 24 . We found that the misclassification of the given summer tanager example may be because of data mislabeling in the original dataset. These examples showcase the ability of our method to help us understand the reasoning process of the model, not just when it is right, but also when it is wrong.This section provides more examples for the local and global analysis. Overview: This section provides details on the user study we conducted. Through our user study, we show that ProtoVit improves both the clarity of reasoning process and coherence of prototypical features relative to ProtoPNet [10] and Deformable ProtoPNet [13] . We randomly selected 10 bird images from test set, and show the comparison with the three most similar prototypes from its top-1 predicted class of the three models. We then presented the test to 10 participants, all of whom attend college in the U.S. and have some background in Machine Learning. We instructed participants to rate how well they understood the models' reasoning process through the presented examples, and asked them to rate their confidence in their evaluation. Intuitively, the user should easily be able to distinguish which feature the model is comparing to, if the model has superb clarity. At the end of the experiment, we asked participants to rank the three models based on overall clarity of reasoning as well as coherence of prototypical features learned. Coherence of feature refers to when the visualization represents one, and only one feature. Some sample questions are shown in Appendix. Fig. 28 . Although this task does not pose any risk to the participants, they were nevertheless informed of their rights, and were asked for consent before data collection. The participants took on average 10 to 20 minutes to complete their assigned tasks, and were compensated at $30 per hour.Result: To quantify the user study result on model clarity, we assign a score from 4 to 1 as the participant rated their understanding of the model's reasoning from best to bad. Similarly, we assign a score from 4 to 1 for the confidence rating from completely confident to not confident. We then multiply the confidence score and the understanding score to have the total score on the clarity of the model reasoning. The maximum total score for a question is 16, which indicates that the participant has a very clear and confident understanding of the model's reasoning. The minimum total score for a question is 1, which indicates that the participant does not understand the model's reasoning at all. As shown in Table. 9, on average, the participants believe that they understand the reasoning process of ProtoViT the best with the highest confidence. The result of the one-sided t-test on understanding score and total score further illustrate that there is a statistically significant improvement in model clarity of ProtoViT to the vanilla ProtoPNet. Fig. 29 shows the result of participants ranking on the three models based on coherence of prototypical features and the clarity of the reasoning process. Again, ProtoViT is mostly ranked as the best in providing coherent prototypical features and clear reasoning process. It is worth noting that the majority of the participants commented that the prototypical features by ProtoPNet are usually too broad to understand what specific part the model is looking at. Moreover, the prototypical features by Deformable ProtoPNet are less accurate in pinpointing the specific parts. On the other hand, the prototypical features by Protovit can provide more accurate and specific visualizations. It is easy to tell if the prototypical features are representing the head of birds or the feet of the birds. Answer: [Yes] Justification: The data are public, and code is provided in the github link.Guidelines:• The answer NA means that paper does not include experiments requiring code. • Providing as much information as possible in supplemental material (appended to the paper) is recommended, but including URLs to data and code is permitted. 


Decision: The paper proposes to integrate deformable prototypes with vision Transformers for interpretable image classification. The initial review ratings were mixed. Yet after the rebuttal and discussion, all reviewers unanimously recommend this paper. They recognize the technical contributions and applaud the presentation of the paper. The authors are encouraged to incorporate review comments and rebuttal materials into the final version.

Reviews: review: paper_summary: paper_summary: The paper presents ProtoViT, a method for interpretable image classification. ProtoViT incorporates ViT backbone with deformed prototypes that explains its predictions. ProtoViT consists of three components: 

(1) a feature encoding layer with a pre-trained ViT backbone, which computes a latent representation of an image; 

(2) a greedy matching layer which compares the latent representation to learned prototypes; and 

(3) an evidence layer which aggregates prototype similarity scores into a classification using a fully connected layer. 

Quantitatively, ProtoViT achieves better performance than previous prototype-based methods. Qualitatively, it identifies meaningful prototypes to explain the prediction.
strength_weakness: strength_weakness: 1. It is understandable that one goal of interpretable image classification is to tell why something works. However, a bigger goal is to understand why something didn't really work. Is it possible to highlight the examples where the method could not predict the correct class? It would be then interesting to understand the reasons for failure.

2. The method relies on a strong assumption of a solid backbone model that can already achieve a good performance on the given task. This restricts the applicability of the method to limited scenarios where we do not need interpretable image classification at the first place. In such a scenarios, the nearest neighbors obtained using latent feature representation and pixel correspondences may be sufficient.

3. Limitations of the method is not clear from the paper. I could guess at certain places in the method section where things could go wrong. It would be better if the authors could illustrate those points with suitable qualitative analysis.

4. User study is not properly designed and is not statistically significant.

Overall: The paper is interesting. In a first reading, everything looks good. But then you start asking yourself questions about the different scenarios where the method will not work (and there are definitely such scenarios as can be judged from quantitative evaluation and user study) and why it will not work. The paper falls short in explaining them with details.
questions: questions: Please see Weaknesses Section.
limitations: limitations: Authors did provide limitations but a far-fetched one that does not really talk about the limitations of the current method.
paper_summary: paper_summary: This paper presents a novel strategy to learn interpretable visual prototypes for visual transformers, with a good property of offering spatially deformed prototypes. The method also introduce an slot mechanism which can learn an adaptive number of prototypical parts. The proposed are wisely designed for visual transformer architectures.
strength_weakness: strength_weakness: 1. The first concerns is about the use of deformable prototypes that containing K sub-prototypes, which means the proposed method will have more learnable prototype vectors (K times) than ProtoPNet, TesNet, ProtoPFormer, and so on. These previous approaches only use 1*1 prototypes. Does the performance improvement of the proposed method come from the largely increased number of sub-prototypes?
2. The idea of greedy matching algorithm is similar to the greedy prototype projection, proposed in [1, 2]. The authors are suggested to state their difference of the related works. Also, some important work using prototypes for interpretable image classification should be reviewed and discussed, such as [3, 4, 5]. 
3. Regarding the adaptive slots mechanism, it is good for the motivation of to learn an additional indicator to measure the importance of sub-prototypes. From my understanding, the learnable vector v is like a gate, which should be saved as model parameters after training. One potential limitation is such mechanism introduces an extra gate parameter v, compared with previous methods.
4. It not much clear about the adjacency masking. Since the prototypes are not initialized to have the position information, how do choose the patch/feature tokens around the prototypes within r? 
5. The authors mention the issue of performance degradation after prototype projection. Does the proposed method still suffer from such issue? What extent will the performance drop?
6. The method has too much loss coefficients, which are selected without detailed tuning procedure. 


References:

[1] Knowledge Distillation to Ensemble Global and Interpretable Prototype-Based Mammogram Classification Models

[2] Pixel-grounded prototypical part networks

[3] PIP-Net: Patch-Based Intuitive Prototypes for Interpretable Image Classification

[4] Learning support and trivial prototypes for interpretable image classification

[5] Concept-level debugging of part-prototype network
questions: questions: N/A
limitations: limitations: N/A
paper_summary: paper_summary: The authors introduce ProtoViT, a model that leverages the Visual Transformer (ViT) architecture and integrates prototypical parts for case-based reasoning. This method is self-explainable, adhering to the rule "this looks like that." A novel aspect of ProtoViT is the use of prototypical parts of varying sizes, utilizing a ViT backbone. The authors claim that these prototypical parts are coherent and inherently interpretable. They evaluate ProtoViT on the CUB and Stanford Cars datasets, using accuracy as the metric for comparison. The training process for the model involves five loss components in addition to cross-entropy.
strength_weakness: strength_weakness: The primary concern lies in ensuring that the ViT backbone can maintain prototypical parts that are both local and inherently interpretable. Since ViT uses attention mechanisms that mix information from all patches, there is a risk of confusion for the end user. To address this, I suggest conducting a spatial misalignment benchmark [1] to analyze its influence.

Additionally, there are no metrics related to explainability demonstrating whether the model improves interpretability, such as with FunnyBirds [2] or through a user study [3].

[1] Sacha, Mikołaj, et al. "Interpretability benchmark for evaluating spatial misalignment of prototypical parts explanations." Proceedings of the AAAI Conference on Artificial Intelligence. Vol. 38. No. 19. 2024.
[2] Hesse, Robin, Simone Schaub-Meyer, and Stefan Roth. "FunnyBirds: A synthetic vision dataset for a part-based analysis of explainable AI methods." Proceedings of the IEEE/CVF International Conference on Computer Vision. 2023.
[3] Kim, Sunnie SY, et al. "HIVE: Evaluating the human interpretability of visual explanations." European Conference on Computer Vision. Cham: Springer Nature Switzerland, 2022.
questions: questions: How are you measuring all the claims, especially coherence and inherently interpretable models?
Could you provide a prototype purity metric for PIP-Net? 
Is it possible to analyze whether ViT can be a backbone for prototypical parts or not, test with spatial misalignment benchmark?
limitations: limitations: There is no quantification of interpretability, no user-study, nor no reference to XAI benchmarks such as FunnyBirds and spatial misalignment benchmarks.
paper_summary: paper_summary: The authors present a novel method for interpretable image classification by incorporating a vision transformer (ViT) into the prototypical neural network framework which provides case-based reasoning to neural network based image classifiers. They claim that most existing prototypical methods are convolutional neural network (CNN)-based, and those methods are limited by spatially rigid prototypes, thus failing to handle geometric variations of objects. Existing methods that try to handle this geometric variation either rely on a continuous latent space, which is not compatible with ViTs, or are other prototype-based ViT that fail to provide inherently interpretable explanations. Due to these existing problems, the author present ProtoViT with the following contributions:
Incorporates a ViT backbone that can adaptively learn interpretable prototypes that can handle geometric variation of different sizes.
They achieve the above with a greedy matching algorithm utilizing an adjacency mask and an adaptive slots mechanism.
They give empirical evaluation showing SOTA accuracy and a qualitative analysis showing the faithfulness and coherence of the learned prototype representations.
strength_weakness: strength_weakness: Soundness:

The lack of qualitative comparison with other ViT methods. I know the authors state that these other vision transformer methods do not project the learned prototypical features to the closest latent patches, but they still provide explanations. Could more be expanded on this and/or a figure showing this lack of reasoning/inherent interpretability?
- This is my biggest concern

Presentation:

In the section 3.4 for “Optimization of last layers” did you mean “... l-th class prototypes…” with a plural on the prototypes? This was unclear to me

Contribution:

However due to already preexisting ViT prototype methods (with a lack of comparison to them), the contribution this ViT makes compared to others is unclear.
questions: questions: Examples in the paper only show good outcomes. What do incorrect predictions look like, and does the explanation (prototypes) show a reason for why the model was incorrect?

Do the authors have additional comments about potential information leakage between image patches in the attention step occurring in the activation of the prototypes? I ask this in context of a latent patch containing more information regarding other patches around it contributing to the learned prototype. When we project on this prototype, we look at its spatial position and project to the real image, but it may not tell the whole story.

When determining the next sub-prototype, why do you only consider the last sub-prototype when creating the adaptive mask given radius r? Why not all the currently selected sub-prototypes? Wouldn’t you get more coverage and cohesive information if considering them all?

Why can’t prototypes be shared across classes? While I still think the prototypes learned are good, I think a limitation that wasn’t stated is the lack of across class prototype sharing.

I feel like the coherence loss could limit your overall prototype representation. If all sub-prototypes have to be similar, then wouldn’t that hurt a prototype representation that has distinct parts. I’m thinking of a potential example of three differently colored stripes being next to each other. If the sub-prototypes are each of different colors, wouldn’t they be very different; thus discarded due to this loss? This is even shown in E.3. I know that is where the adaptive mask could combat this problem, but I would like more discussion on this. 

I’m curious how sensitive your method is to masking / perturbation. For example, if you mask out some or all the sub-prototypes for the top activated area in the test image, can the model still find the prototype elsewhere if it still exists in the image? A particular case I’m interested in is if a prototype is about the blue on a bird’s feathers, so you make that part that the model initially detected. Does the model still pick up the blue elsewhere?

Could you expand more on why you “believe that simply adding prototype layers to an architecture without well defined “cases” does not make the new architecture more interpretable.” In particular, what do you mean by ‘well defined cases’?
limitations: limitations: The method lacks across class prototype sharing.

The coherence loss lack discussion on its limitation of making sub-prototypes being similar thus hindering diverse representation which may be important in a prototype.

The paper lacks comparison to other interpretable ViT methods

Comments: title: Thanks!
Rebuttal Response
citations
comment: Thank you for the rebuttal! I have raised my scores.
We hope that our response has helped explain our work's contributions and address your concerns. Please feel free to let us know if you have any further questions. We are very happy to discuss!
Thank you for your response! 

> I understand that other methods may not project to a latent of a training example, but this doesn't seem like a big step / modification to add to the other methods for comparison. If this was done then having this comparison would have added strength to this paper.

We agree that adding visualizations from other VIT based models would be very interesting and could directly show the strength of our method. However, since those methods also learn prototypes on the class tokens which have no correspondence to the image as we discussed in the related work, it is unclear how to project and visualize those prototypes even with projection. The main reason that those methods do not perform projection is because they observe a dramatic performance drop. This is explicitly stated on page 11 of ProtoPFormer, where they say: 

"Our proposed ProtoPFormer does not employ the “push” process for two main reasons. One is that this process causes the performance degradation with ViT backbones, and the other is that our global and local prototypes are the high-level abstraction of associated visual explanations, representing their features based on the whole training set."

This drop is because that their method is not able to learn a well clustered latent space, which reviewer **GK9W** agreed on. Thus, it is extremely unclear how those methods produce the visualizations used in their manuscripts. 


>Thank you for the perturbation and misclassification examples! Please do add this to the final paper if accepted!

We will make sure to do so!

>If several parts are semantically different, but their combination is essential for the downstream task, then I still think this loss would restrict that. 

When they are semantically different, it would be captured by other prototypes associated with that class as encouraged by orthogonality loss, which is why we adopt the orthogonality loss in our method. Coherence loss is encouraging the similarity inside a prototype (which consists of some sub-prototypes), and **orthogonality loss is encouraging prototypes to be different from each other (to have diverse representation)**. So, if there are several semantically different parts that are each necessary for classification, a different prototype should learn to identify each one of them. We will make sure to discuss this more in the final manuscript. 

We hope this would further address your concerns. And thank you again for updating your score and engaging with us! We greatly appreciate your feedback.
Thank you for a detailed rebuttal!

I understand that other methods may not project to a latent of a training example, but this doesn't seem like a big step / modification to add to the other methods for comparison. If this was done then having this comparison would have added strength to this paper. I understand that they would just be similar to ProtoPNet if this was added, but they wouldn't be the same. Thank you for emphasizing that other ViT-based methods are unable to handle geometric variations. Thank you for adding parts of this discussion in the paper if accepted. On this point overall, having a unifying comparison between these approaches would have contributed well to this area.

Thank you for the perturbation and misclassification examples! Please do add this to the final paper if accepted!

I appreciate the response on the coherence loss. I think further analysis of this loss and it's limitations would be desirable. I understand that this is in the latent space, but the concern is still that same. If several parts are semantically different, but their combination is essential for the downstream task, then I still think this loss would restrict that. I understand this loss adds more control which is a reason I added it as a strength, but I still think it comes with limitations (unless there are more analysis that says otherwise).

Thank you for addressing some of my other concerns.

I've updated my score to reflect the added strength from the rebuttal.
Thank you very much for your response! We will make sure to include the discussion of the performance drop caused by projection in the body of the final manuscript. It will be _discussed in the training algorithm section after the discussion of projection starting on page 7_. It is encouraging to know that our detailed revisions have addressed your concerns. Your recognition of our work in terms of contribution, novelty, soundness, and presentation is greatly appreciated. 

We also appreciate it _if you could consider updating the rating_, as this would kindly reflect your most recent evaluation and kind recognition of the paper and acknowledge our efforts.

Thank you again for engaging with us!
Thank you for the responses. I am happy with your work. 
It is great to have an exhaustive discussion about the performance drop caused by prototype projection. I hope such discussion can be included in the main paper, if your work is accepted, since a few prior work noticed this practical issue. Good job!
Thank you for your response! 

>  If your work is accepted, please include values for the purity metric, as it is an established metric for evaluation. Including this will enhance the consistency of the research.

We will make sure to include the purity metric to the final manuscript if the paper is accepted. 

> Regarding the user study, I really appreciate the presentation of results from Ma et al. [1], who also tested if users performed better than random. It would be worthwhile to check this for your method as well.

We agree that this could be an interesting addition to our paper, but it will take a long time to design and complete such a user study. We don't believe that we can finish it before the end of the discussion period. We will add it to the final manuscript if the paper is accepted. 

Thanks again for engaging with us!
That is indeed a great response to my review. Thank you for the clarifications and discussion. If your work is accepted, please include values for the purity metric, as it is an established metric for evaluation. Including this will enhance the consistency of the research.

The results of the spatial misalignment benchmark are surprising, as I did not expect the ViT to be more robust in terms of explanation quality and faithfulness. Good job!

Regarding the user study, I really appreciate the presentation of results from Ma et al. [1], who also tested if users performed better than random. It would be worthwhile to check this for your method as well.

After consideration, I am leaning toward increasing my grade, but I still need to digest the information from other reviewers and their discussions.

[1] Ma, Chiyu, et al. "This looks like those: Illuminating prototypical concepts using multiple visualizations." Advances in Neural Information Processing Systems 36 (2024).
[1] Xue, Mengqi, et al. "Protopformer: Concentrating on prototypical parts in vision transformers for interpretable image recognition." arXiv preprint arXiv:2208.10431 (2022)

[2] Kim, Sangwon, Jaeyeal Nam, and Byoung Chul Ko. "Vit-net: Interpretable vision transformers with neural tree decoder." International conference on machine learning. PMLR, 2022.

[3] Rymarczyk, Dawid, et al. "Interpretable image classification with differentiable prototypes assignment." European Conference on Computer Vision. Cham: Springer Nature Switzerland, 2022.

[4] Rymarczyk, Dawid, et al. "Protopshare: Prototype sharing for interpretable image classification and similarity discovery." arXiv preprint arXiv:2011.14340 (2020).