Venue: ICLR 2022 Conference

Reviews:

final_score: rating: 3: reject, not good enough
review_content: summary_of_the_paper: The premise of the paper is that example train instances with the corresponding label can be an effective explanation of the model's prediction rather than post-hoc explanation which is not deterministic. The framework jointly trains label prediction and prototype clustering. Then, the framework computes the similarity between input and prototypes to retrieve the most similar prototype as an explanation while predicting the label as well. Moreover, the framework can reflect human-in-the-loop feedback on the prototype.
The primary result of the paper is that their framework can show the proper explanation by a similar example while preserving the performance.
main_review: Strengths
- S1 - The idea of choosing a similar example from the train data as an explanation is an interesting idea.

Weaknesses and Questions:
- W1 - I'm quite unsure about the set of prototypes. It seems not described in the main contents. I checked examples of prototypes in the appendix but the 1:1 label distribution (positive vs. negative) seems to me they are human-picked.
- W1-Q1 - If the set of prototypes pre-defined, do we really need to train the prototype? what if we just compute sentence similarity between input and the prototype?
- W2 - The main table is not clear, especially on faithfulness. What if we just consider the random prototype as a prototype? what is the faithfulness of that random prototype then?
- Q2 - Just curious, in the introduction, why the post-hoc explanation is potentially reporting-bias? Isn't this method also potentially reporting-bias since it only uses samples from train data as prototypes?
summary_of_the_review: My overall disposition towards the paper is indifferent although the paper proposes an interesting idea. It is a bit difficult to follow the method due to a lack of information. More details are needed especially on experimental settings, and methods.
review_title: null

final_score: rating: 5: marginally below the acceptance threshold
review_content: summary_of_the_paper: This paper aims to model explanation and task prediction such that task performances are not (or less) traded off for interpretability. It proposes a novel framework for transformer models where classification and explanation generation are based on shared prototype embeddings which are learnt from training data by a combination of losses. The framework is also compatible with settings that requires human in the loop for extra supervision on prototype learning. Experiment results show that adding the proposed ProtoTrex benefit task performances on 3 sentiment classification tasks.
main_review: This paper seems motivated by a prior NeurIPS 19 work "This Looks Like That..." in the sense that the architecture and loss designs are sourced from there. The nice thing about this paper is that it focuses on NLP tasks, so the framework could potentially benefit the explanation community. The early part of this paper is very straightforward and intuitive. Related works have limited coverage. The explanation generation part is vague. The experiment section is weak. Analysis on generated explanation is also weak.

The architecture relies on using prototypes which are nicely discussed in this paper. The problem if improving interpretability without trading off downstream task F1 is interesting since the trade-off was common among prior works. However a couple confusing points still. 

1. There are many lines of explanation works, such as those use prompt engineering, information bottleneck, and purely generative approaches. This paper has limited coverage on these topics. Touching different approaches is important here since the way ProtoTrex handle explanation might not easily extend to all other cases.

2. Even though this design was from the NeurIPS 19 paper, but in the task of NLP, how does the prototype embeddings compare against the label-wise weights in the final classification layer? This is to imagine that, without the use of the complicated loss in Eq 1, how does simply treating the label-wise embeddings as prototypes perform?

3. The explanation generation (sec 3.5) needs elaboration. It seems this paper uses prototype embedding to find a training example as it nearest neighbor, and then use this data point as explanation to its prediction. This design has certain limitations: a) not context/example dependent; b) this is hardly generation, instead, it is more in line with salience-based explanation works. Tab2 indeed shows some examples with explanation that partially depends on the input example. But no idea how they were generated.

4. Tab 1b is confusing. I don't understand what each number means. This paper very briefly went over some statements without getting into details.

5. I don't see a solid explanation evaluation in this paper. Tab 1c shows rationale performances however these numbers are quite low compared with prior works (e.g. Paranjape's work at EMNLP 20). And not sure if rationale performances are based on token or sentence selection. Either way, this evaluation has nothing to do with generation. And when it comes to generative explanations, ideally, there should be some human-based evaluation over a subset of testing data. But no such thing in this paper.
summary_of_the_review: I think the architecture has novelty when it comes to NLP tasks. And this work could benefit the explanation community. However, I found the experiment results are confusing. To the best degree, it offers marginal improvement over the best baselines in terms of task F1. When it comes performance of explanation, I only see confusing numbers, thus no conclusion can be made. Analysis on generated explanation is another weak point since it's absent.
review_title: null

final_score: rating: 5: marginally below the acceptance threshold
review_content: summary_of_the_paper: This paper proposes Proto-Trex model to increase the interpretability of the text classification systems. The proposed model mainly adds a bunch of prototype layers to learn the similarity between the query and prototypes. They also propose an extension caleld iProto-Trex to interactively learn from users' feedbacks. Experimental results and example cases show that the Proto-Trex could give comparable classification accuracy compared to plain classification model (w/o explanation) and could provide reasonable explanations. 
main_review: Strengths:

1. The proposed Proto-Trex/iProto-Trex are technically sound.
2. Through several example cases, the proposed model could generate reasonable explanations.


Weaknesses:

1. The training loss of the model seems to be too complicated. In Eq.1, there are 6 parts for the overall training loss, and each of them associates with a lambda term (I checked the Appendix, but failed to find how you set lambda terms). Each of them sounds reasonable, but unfortunately, there is no specific ablations on how these parts attribute to the final prediction/explanation. 
2. According to the illustrations at the end of Section 3.4, there are many hyper-parameters should be set (along with lambda terms in training loss). How to determine these hyperparameters? If we move to a new dataset, we might lost in tuning these hyperparams.
3. As the model incorporates additional modules (compared to pure classification model), it is questionable how efficient is Proto-Trex (both for parameter size and inference time).


Comments:

1. It is unclear where are the results from in Table 1b and 1c. Yelp? Movie? Toxicity?

summary_of_the_review: The paper addresses an important issue in explainable natural language processing models. The proposed model is reasonable, and the results seem to be OK. However, many issues are left unclear, such as its efficiency, hyper-parameter tuning, etc. I tend to lean towards weak rejection at the moment.
review_title: null

final_score: rating: 5: marginally below the acceptance threshold
review_content: summary_of_the_paper: This paper introduces a method for improving interpretability of black box transformer based text classifiers. The approach is based on “case based reasoning” where the network classifies an input by comparing it against a library of learnt prototypes (each with a corresponding label) and classifying the input based on a weighted similarity with the prototypes. Thus, in a sense it is more interpretable than an end-to-end classifier since a user can directly look at the similarity scores with the prototypes to understand the labeling decision. The approach also promises to be more “faithful” than post-hoc interpretability methods since the predictions are based directly on similarity scores. The paper also shows that end-users of the system can interact with it by either editing prototypes (if they are experts) or providing weak feedback about which prototypes are good, and this feedback can be integrated to re-learn better prototypes. From experiments we see that their approach is competitive with standard end-to-end finetuning while being more interpretable. Based on sufficiency and comprehensiveness scores, we see some evidence that the explanations from prototypes are faithful, though it is unclear if these scores are competitive.

main_review: Strengths: The paper touches upon several interesting threads: interpretability, case based reasoning and using user interactions for improving models. It is well written, with a comprehensive set of experiments. 

Weaknesses:
- Hyperparameters: My initial impression of the method is that it involves far too many hyperparameters (various $\lambda$ for combining losses, choosing similarity metrics etc). How hard was tuning hyperparameters?
- Results:  
    - different models respond differently to the proposed approach. For instance, we see large performance gains on BERT but performance drops for SBERT. It would be good to know why this happens.
    - User interaction results are not convincing. From Table-3, it seems like the interactions barely help, even though the prototype changes sometimes. Could the authors comment more about this?
    - Can the authors provide with confidence intervals for all results so we can compare across different settings better?
Prototype quality: prototypes do not look very convincing. I glanced at the prototypes in the Appendix and they seem to be quite redundant / of low diversity. For instance, from Table-6 a lot of the prototypes either model “bad service and bad food” or “good service and good food” (P2/P4/P6/P8/P10). However yelp also has examples with “bad service, good food”, “good food, bad service”, examples not related to restaurants at all but the prototypes do not uncover these. 
Faithfulness: How competitive are these numbers with post-hoc methods? I think for presentation purposes, it would be good to also compare this with SHAP/LIME/IG. 

Overall, I think this paper is currently borderline. However, I’m happy to increase my score if the weaknesses can be quickly addressed —- the main ones are providing evidence that the learnt prototypes can be diverse and cover a lot more of the “modes” in the training distribution, providing a more comprehensive comparison with posthoc interpretability approaches, and confidence intervals.  

summary_of_the_review: While i think this work combines many interesting threads in interpretability, using user feedback and case based reasoning, the paper in its current state is not ready. I’m happy to increase my score / recommendation if the weaknesses are addressed, however.
review_title: null

Meta-review: The authors propose a method—"Proto-Trex"—that incorporates prototype networks into text classification architectures to facilitate model explanations via presentation of similar training examples. There was agreement that the direction here is promising and the work contains some nice ideas and a good initial set of evaluations. However, the presentation can be improved substantially to better situate the contribution with respect to related work (and clarify the specific contributions on offer here), and to clarify the key technical details of the proposed approach.
Decision: Reject