Venue: ICLR 2018 conference

Reviews:

final_score: rating: 4: Ok but not good enough - rejection
review_content: The paper shows that neural networks are sensitive to adversarial perturbation for a set of NLP text classifications. They propose constructing (model-dependent) adversarial examples by optimizing a function J (that doesn't seem defined in the paper) subject to a constraint c(x, x') < \gamma (i.e. that the original input and adversarial input should be similar)

c is composed of two constraints:
1. || v - v' ||_2 < \gamma_1, where v and v' are bag of embeddings for each input 
2. |log P(x') - log P(x)| < \gamma_2 where P is a language model

The authors then show that for 3 classification problems "Trec07p", "Yelp", and "News" and 4 models (Naive Bayes, LSTM, word CNNs, deep-char-CNNs) that the models that perform considerably worse on adversarial examples than on the test set.  Furthermore to test the validity of their adversarial examples, the authors show the following:
1. Humans achieve somewhat similar accuracy on the original adverarial examples (8 points higher on one dataset and 8 points lower on the other two)
2. Humans rate the writing quality of both the original and adversarial examples to be similar
3. The adversarial examples only somewhat transfer across models

My main questions/complaints/suggestions for the paper are:

-Novelty/Methodology. The paper has mediocre novelty given other similar papers recently. 

On question I have is about whether the generated examples are actually close to the original examples. The authors do show some examples that do look good, but do not provide any systematic study (e.g. via human annotation)

 This is a key challenge in NLP (as opposed to vision where the inputs are continuous so it is easy to perturb them and be reasonably sure that the image hasn't changed much). In NLP however, the words are discrete, and the authors measure the difference between an original example and the adversary only in continuous space which may not actually be a good measure of how different they are.

They do have some constraint that the fraction of changed words cannot differ by more than delta, but delta = 0.5 in the experiments, which is really large! (i.e. 50% of the words could be different according to Algorithm 1)

-Writing: the function J is never mathematically defined, neither is the function c (except that it is known to be composed of the semantic/syntactic similarity constraints).

The authors talk about "syntactic" similarity but then propose a language model constraint. I think is a better word is "fluency" constraint. 

The results in Table 3 and Table 6 seem different, shouldn't the diagonal of Table 6 line up with the results in Table 3?

-Experimental methodology (more of a question since authors are unclear): The authors write that "all adversarial examples are generated and evaluated on the test set".

There are many hyperparameters in the proposed authors' approach, are these also tuned on the test set? That is unfair to the base classifier. The adversarial model should be tuned on the validation set, and then the same model should be used to generate test set examples. (The authors can even show the validation adversarial accuracy to show how/if it deviates from the test accuracy)

-Lack of related work in NLP (see the anonymous comment for some examples). Even the related work in NLP that is cited e.g. Jia and Liang 2017 is obfuscated in the last page. The authors' introduction only refers to related works in vision/speech and ignores related NLP work.

Furthermore, adversarial perturbation is related to domain transfer  (since both involve shifts between the training and test distribution) and it is well known for instance that models that are trained on Wall Street Journal perform poorly on other domains.  See SJ Pan and Q Yang, A Survey on transfer learning, 2010, for some example references.
review_title: Ok paper, but needs some revision

final_score: rating: 4: Ok but not good enough - rejection
review_content: This paper proposes a method to generate adversarial examples for text classification problems. They do this by iteratively replacing words in a sentence with words that are close in its embedding space and which cause a change in the predicted class of the text. To preserve correct grammar, they only change words that don't significantly change the probability of the sentence under a language model.

The approach seems incremental and very similar to existing work such as Papernot et. al. The paper also states in the discussion in section 5.1 that they generate adversarial examples in state-of-the-art models, however, they ignore some state of the art models entirely such as Miyato et. al.

The experiments are solely missing comparisons to existing text adversarial generation approaches such as Papernot et. al and a comparison to adversarial training for text classification in Miyato et. al which might already mitigate this attack. The experimental section also fails to describe what kind of language model is used, (what kind of trigram LM is used? A traditional (non-neural) LM? Does it use backoff?).

Finally, algorithm 1 does not seem to enforce the semantic constraints in Eq. 4 despite it being mentioned in the text. This can be seen in section 4.5 where the algorithm is described as choosing words that were far in word vector space. The last sentence in section 6 is also unfounded.


Nicolas Papernot, Patrick McDaniel, Ian Goodfellow, Somesh Jha, Z.Berkay Celik, and Ananthram Swami
Practical Black-Box Attacks against Machine Learning.
Proceedings of the 2017 ACM Asia Conference on Computer and Communications Security

Takeru Miyato, Andrew M. Dai and Ian Goodfellow
Adversarial Training Methods for Semi-Supervised Text Classification.
International Conference on Learning Representation (ICLR), 2017

* I increased the score in response to the additional experiments done with Miyato et. al. However, the lack of a more extensive comparison with Papernot et. al. is still needed. The venue for that paper might not be well known but it was submitted to arXiv computer science too and the paper seems very related to this work. It's hard to say if Papernot et. al produces more perceptible samples without doing a proper comparison. I find the lack of a quantitative comparison to any existing adversarial example technique problematic.
review_title: No comparison to existing work

final_score: rating: 6: Marginally above acceptance threshold
review_content: Nice overview of adversarial techniques in natural language classification. The paper introduces the problem of adversarial perturbations, how they are constructed and demonstrate what effect they can have on a machine learning models. 

The authors study several real-world adversarial examples, such as spam filtering, sentiment analysis and fake news and use these examples to test several popular classification models in context of adversarial perturbations. 

Their results demonstrate the existence of adversarial perturbations in NLP and show that several different types of errors occur (syntactic, semantic, and factual). Studying each of these errors type can help defend and improve the classification algorithms via adversarial training.

Pros: Good analysis on real-world examples
Cons: I was expecting more actual solutions in addition to analysis
review_title: Nice overview of adversarial techniques in natural language classification

Meta-review: This paper presents a way to generate adversarial examples for text classification.   The method is simple -- finding semantically similar words and replacing them in sentences with high language model score.  The committee identifies weaknesses in this paper that resonate with the reviews below -- reviewer 1 suggests that the authors should closely compare with the work of Papernot et al, and the response to that suggestion is not satisfactory.  Addressing such concerns would make the paper stronger for a future venue.
Decision: Reject