Venue: ICLR 2020 Conference

Reviews:

final_score: rating: 6: Weak Accept
review_content: As an extension of recent developments on adversarial attacks and defenses, this paper proposes a simple but effective technique called undersensitivity on machine comprehension task, where the input question is changed but the prediction does not change when it should be. They use two linguistically informed tricks; PoS and NER, to produce the perturbations. In addition to that, several techniques are developed for reducing the adversarial search spaces (Eq 1 and 3) and controlling the level of undersensitivity (Eq 2). 

In general, the paper is very well written and clear to read. The formulation of the problem is very straightforward, too. I enjoyed reading the overall paper, especially the experimental results, which provides lots of insights about the techniques. The proposed techniques are simple but they are well-executed in the experiment with reasonable justification. Please find my detailed comments below. 


Method. 
I appreciate the simplicity of the proposed models with clear motivations. Also, validation of the approaches is well-executed in the experiment.

I like the idea of linguistically-controlled perturbations using PoS and NER. However, there might be many other ways to control it: for example, parsing a sentence using a constituency parser and replacing each phrase with corresponding synonyms/antonyms using WordNet might be interesting. Or, based on the parse, negating the verb might be another way to try. I would expect more linguistically-informed perturbations like these, and I could find some of them from (Kang et al 2018, Ebrahimi et al., 2018). Also, adding a couple of them in the experiment might be interesting to understand the underlying logic of the perturbations. 

One major concern of the proposed approach is the sub-optimality by the pre-trained RC model. The undersensitivity (Eq 2) and adversarial search (Eq 3) are calculated by the probability scores predicted by the pre-trained models. This means that producing the new sample x’ is only based on the correctness of the pre-trained model on new samples generated, which sounds to be unreliable. Moreover, using the samples produced by this sub-optimal model may be very limited to produce samples under the sub-optimal space of questions. I wonder how the authors tackle this issue in the experiment. 

Experiment
Adversarial attacks should show how an existing system is fragile to be attacked, but at the same time augmenting or adversarially training with them needs to improve its generalization power of the system against the attacks. However, many of the adversarial attack papers mostly focus on the former but not the latter part. In this work, authors showed a result of adversarial training/augmentation but its generalization power on original task (i.e., HasAns case) was not that powerful. The unbiased data setup is interesting but still did not provide any insights about generalization from the adversaries. It would be more convincing to see how this generalization from adversarial attacks can take benefits from bit different tasks such as open-end reading comprehension as a perspective of data augmentation. 

I see no comparison with other attacking/defending methods in Tables 3 and 4. Adding the recent models (Ebrahimi et al., 2018, Wallance et al., 2019) may help understand how the proposed models are more effective than other techniques. 


review_title: Official Blind Review #1

final_score: rating: 3: Weak Reject
review_content: The paper proposes a framework for evaluating the sensitivity of a QA model to perturbations in the input. The core of the idea is that one can replace content words (i.e. named entities and nouns) in questions in such a way that makes QA models more confident of their original answer (despite, presumably, the question now being unanswerable). Replacements are constructed by mining equivalence classes in Squad data (i.e. all words w/ pos = noun are one set).    Depending on how many such substitutions are searched over (and whether multiple are applied),  one can find at least one such failure in about 50% of cases, on a BERT model trained on Squad2.  The paper also proposes a simple mitigation technique: an objective that modifies a given QA example with all possible substitutions and trains for "no answer" (or alternatively substitutions which break the system).  Results demonstrate that performance on Squad2 is roughly unchanged while the success rate of the attack is significantly decreased.

While the idea of forming such equivalence sets is very interesting, my concern with the paper is both in terms of impact and experimental methodology. 

Impact: the method is essentially a data augmentation approach over a fixed list of words. This isn't very different than what was proposed in https://arxiv.org/pdf/1804.06876.pdf and https://arxiv.org/pdf/1807.11714.pdf . While there are some nice nuggets in the analysis, in particular that model confidence is a factor for the attack, I'm not sure anything very novel is being proposed. 

Experimental Methodology: Other works in this vein explicitly create a split between counterfactual examples evaluated at train vs at test. The methodology proposed here requires a search where there isn't a clear split between what aspects of the search are allowed at train vs test. In doing counterfactual data augmentation, it is possible the model observes most elements of the search that will be evaluated at test time, making it almost inevitable that the search will be less successful after the model is trained. A simple solution would be splitting the equivalence sets into train/test. I was not able to confirm whether or not this happened from the paper.

That being said, the paper did evaluate on Lewis&Fan( https://openreview.net/pdf?id=Bkx0RjA9tX ) 's bias training simulation, which I appreciate, but I was disappointed that (a) the results from Lewis&Fan were not included for comparison, and when compared the augmentation method proposed here works much worse, in some settings, than generative based training. 

review_title: Official Blind Review #3

final_score: rating: 6: Weak Accept
review_content: This paper studies undersensitivity of the neural models for reading comprehension. First, the neural model is trained on reading comprehension tasks with unanswerable questions. Then, they add perturbations to the input to turn an answerable question into an unanswerable question, using two methods, POS tag based and named entity based. Then, they search for adversarial attacks to find perturbations that the model still predicts the same prediction with even a higher probability. Experiments show that the error rate (attack success rate) is high, over 0.9 with POS tag based method and over 0.5 with named entity based method. Finally, this paper shows data augmentation and adversarial training for this perturbation help the model to be more robust, especially in a biased data scenario.

The contribution of this paper is clear to me: it is one of the first studies which investigates undersensitivity of the model when the input text after the perturbation is complete (e.g. in contrast to Feng et al 2018 and other related work where the perturbation causes the input text to be incomplete).

The weakness of this paper is:
1) the observations are somewhat obvious: it is hard to expect the model to always assign lower probabilities to the original answer when, for example, the named entity in the question is replaced to entities with the same type. Also, I think the observation could be more interesting if the adversarial attack works across different models.
2) Table 2 shows that the perturbation does not always work; especially with POS based method, only half of cases work. How many samples were used for this analysis? Is there a breakdown of the error rate (attack success rate) showing that the rate is still significant for valid perturbations? I think it is significant since perturbations seem to cause invalid attack with a pretty high probability.

Despite the weakness, I think this paper demonstrates comprehensive studies on this focused area and is worth to be published in ICLR overall.

review_title: Official Blind Review #2

Meta-review: The paper investigates the sensitivity of a QA model to perturbations in the input, by replacing content words, such as named entities and nouns, in questions to make the question not answerable by the document. Experimental analysis demonstrates while the original QA performance is not hurt, the models become significantly less vulnerable to such attacks. Reviewers all agree that the paper includes a thorough analysis, at the same time they all suggested extensions to the paper, such as comparison to earlier work, experimental results, which the authors made in the revision. However, reviewers also question the novelty of the approach, given data augmentation methods. Hence, I suggest rejecting the paper.
Decision: Reject