Venue: ICLR 2020 Conference

Reviews:

final_score: rating: 1: Reject
review_content: This paper considers the problem of text classification, especially the settings in which the number of labeled sentences is very small. However, authors assume, annotations of rationales behind the label, i.e. highlighting tokens in a sentence which are important in deciding its label. As per my understanding, this is a big limitation. Second, the proposed model makes inference of class labels just based upon occurrence of words in a sentence, rather than making more sophisticated inferences relying upon sub-sequence patterns at least. 

The idea proposed in the paper is to learn prototype vectors which have high similarity w.r.t. tokens in sentences, especially the highlighted one. I didn't understand the justification for learning such prototypes in the first place.

This works build upon a workshop paper. 

The idea proposed in the paper, even in the specific problem context considered, are incremental. I don't think that this kind of work aligns with the theme of learning representations. This paper may be suitable for publication in an NLP workshop as the baseline model.

review_title: Official Blind Review #1

final_score: rating: 3: Weak Reject
review_content: The authors propose PARCUS ("Pattern Representations on Continuous Spaces"), a model which computes a soft-matching probability for all words in an input sequence with so-called prototypes in order to predict a label for the input. Furthermore, for training, PARCUS makes use of rationales. Those are indicators of input importance, and help to boost the loss for relevant tokens.

The main motivation to use PARCUS is that it works better in a low-resource setting than recent state-of-the-art models for the high-resource case. This is due to it having relatively few parameters and to it having a strong inductive bias. However, the fact that models with less parameters perform better than BERT-based models in the low-resource case is not very surprising. Looking at the experiments, the results on HATESPEECH show less differences between models than for SPOUSE or MOVIEREVIEW.

Another selling point of PARCUS is that it's interpretable. While neural networks can also be analyzed in different ways, I agree with the authors that this is nice to have.

Overall, the paper seems solid.

==========

Update: After reading the other reviews and the responses by the authors, I lowered my score from 6 to 3.
review_title: Official Blind Review #2

final_score: rating: 3: Weak Reject
review_content: After responses:

I read the authors response and decided to stick to my original score mostly because:

1 - I understand that interpretability is hard to define. I also agree with the authors response. However, this is still not reflected in the paper in any way. I expect a discussion on what is the relevant definition used in the paper and how does it fit to that definition. Currently, it is very confusing to the reader.

2 - I understand the authors' response that few-shot learning is a different empirical setting. However, authors also agree that settings are some-what relevant. I really do not see any gain by NOT discussing the few-shot learning literature. At the end, a reader is interested in this work if they have limited data. Moreover, other ways to address limited data issue should be discussed.

-----
The manuscript is proposing a few-shot classification setting in which training set includes only few examples. The main contribution is using prototype embeddings and representing each word as cosine distances to these prototype embeddings. Moreover, the final classification is weighted summation of the per-token decisions followed by a soft-max. Per-token classifiers are obtained with an MLP using the cosine distances as features. When the relevance labels are available, they are used in training to boost gradients.

PRO(s)
The proposed method is interesting and addressing an important problem. There are many few-shot scenarios and finding good models for them is impactful.

The results are promising and the proposed method is more interpretable than the existing NLP classifiers. I disagree with the claim that the model is interpretable. However, I appreciate the effort to interpret the model.

CON(s)
The model is not interpretable because 1) it starts with embeddings and they are not interpretable, 2) model is full of non-linearities and decision boundaries are not possible to find. In other words, it is not possible to answer "what would make this model predict some other classifier".

The authors should discuss the existing few-shot learning mechanisms. Especially, "Prototypical Networks for Few-shot Learning" is very relevant. I also think it can be included as a baseline with very minimal modifications.

The writing is not complete. The authors do not even discuss how the prototypes are learned. I am assuming it is done using full gradient-descent over all parameters. However, this is not clearly discussed. Implementation details should be discussed more clearly.

SUMMARY
I believe the manuscript is definitely interesting and has a potential. In the mean time, It is not ready for publication. It needs a through review of few-shot learning. Authors should also discuss can any of the few-shot learning methods be included in the experimental study. If the answer is yes, it should be. If the answer is no, it should be explained clearly. 

Although my recommendation is weak-reject, I am happy to bump it up if these issues are addressed.
review_title: Official Blind Review #3

Meta-review: The authors focus on low-resource text classifications tasks augmented with "rationales". They propose a new technique that improves performance over existing approaches and that allows human inspection of the learned weights.

Although the reviewers did not find any major faults with the paper, they were in consensus that the paper should be rejected at this time. Generally, the reviewers' reservations were in terms of novelty and extent of technical contribution.

Given the large number of submissions this year, I am recommending rejection for this paper.

Decision: Reject