Venue: ICLR 2018 conference

Reviews:

final_score: rating: 6: Marginally above acceptance threshold
review_content: The paper proposes a modification to adversarial training. Instead of alternating between clean and examples generated on-the-fly by the fast gradient sign during training, the model training is performed by alternating clean examples and adversarial examples generated from pre-trained models. The motivation behind this change is that one-step method to generate adversarial examples fail at generating good adversarial examples when applied to models trained in the adversarial setting. In contrast, one-step methods applied to models trained only on natural data generate adversarial examples that transfer reasonably well, even on models trained with usual adversarial training. The authors also propose a slight modification to the fast gradient sign method, in which an adversarial example is created using a random perturbation and the current model's gradient, which seems to work better than the fast gradient sign method. Experiments with inception models on ImageNet show increased robustness both against "black-box" attacks using held-out models not used in ensemble adversarial training.

One advantage of the method is that it is extremely simple. It uses pre-trained models that are readily available, and gains robustness against several well-known adversaries widely considered in the state of the art. The experiments are carried out on ImageNet and are seriously conducted.

On the negative side, there is a significant loss in accuracy, and the models are more vulnerable to white-box attacks than using standard adversarial training. As the authors discuss in the conclusion, this leaves open the question as to whether the models are indeed more robust, or whether it is an artifact of the static black-box attack schemes that are considered in the paper, which measures how much a single model is robust to adversarial examples for other models that were trained independently. For instance, there are no experiments against what is called adaptive black-box adversaries; one could also imagine finding adversarial examples that are trained to fool all models in a predefined collection of models. In the end, while the work presented in the paper found its use in the recent NIPS competition on defending against adversarial examples, it is still unclear whether this kind of defence would make a difference in critical applications.


review_title: a simple an effective method against static black-box attacks, but it remains unclear whether the models are more robust in general

final_score: rating: 6: Marginally above acceptance threshold
review_content: This paper proposes ensemble adversarial training, in which adversarial examples crafted on other static pre-trained models are used in the training phase. Their method makes deep networks robust to black-box attacks, which was empirically demonstrated.

This is an empirical paper. The ideas are simple and not surprising but seem reasonable and practically useful.
Empirical results look natural.

[Strong points]
* Proposed randomized white-box attacks are empirically shown to be stronger than original ones.
* Proposed ensemble adversarial training empirically achieves smaller error rate for black-box attacks.

[Weak points]
* no theoretical guarantee for proposed methods.
* Robustness of their ensemble adversarial training depends on what pre-trained models and attacks are used in the training phase.

review_title: The ideas are not surprising but seem reasonable and practically useful. 

final_score: rating: 6: Marginally above acceptance threshold
review_content: This paper describes computationally efficient methods for training adversarially robust deep neural networks for image classification. (These methods may extend to other machine learning models and domains as well, but that's beyond the scope of this paper.) 

The former standard method for generating adversarially images quickly and using them in training was to do a single gradient step to increase the loss of the true label or decrease the loss of an alternate label. This paper shows that such training methods only lead to robustness against these "weak" adversarial examples, leaving the adversarially-trained models vulnerable to multi-step white-box attacks and black-box attacks (adversarial examples generated to attack alternate models).

There are two proposed solutions. The first is to generate additional adversarial examples from other models and use them in training. This seems to yield robustness against black-box attacks from held-out models as well.  Of course, it requires that you have a somewhat diverse group of models to choose from. If that's the case, why not directly build an ensemble of all the models? An ensemble of neural networks can still be represented as a neural network, although a more computationally costly one. Thus, while this heuristic appears to be useful with current models against current attacks, I don't know how well it will hold up in the future.

The second solution is to add random noise before taking the gradient step.  This yields more effective adversarial examples, both for attacking models and for training, because it relies less on the local gradient. This is another simple idea that appears to be effective. However, I would be interested to see a comparison to a 2-step gradient-based attack.  R+Step-LL can be viewed as a 2-step attack: a random step followed by a gradient step. What if both steps were gradient steps instead? This interpolates between Step-LL and I-Step-LL, with an intermediate computational cost. It would be very interesting to know if R+Step-LL is more or less effective than 2+Step-LL, and how large the difference is.

I like that this paper demonstrates the weakness of previous methods, including extensive experiments and a very nice visualization of the loss landscape in two adversarial dimensions. The proposed heuristics seem effective in practice, but they're somewhat ad hoc and there is no analysis of how these heuristics might or might not be vulnerable to future attacks.
review_title: Interesting empirical analysis and heuristics

Meta-review: The paper studies a defense against adversarial examples that re-trains convolutional networks on adversarial examples constructed to attack pre-trained networks. Whilst the proposed approach is not very original, the paper does present a solid empirical baseline for these kinds of defenses. In particular, it goes beyond the "toy" experiments that most other studies in this space perform by experimenting on ImageNet. This is important as there is evidence suggesting that defenses against adversarial examples that work well on MNIST/CIFAR do not necessarily transfer well to ImageNet. The importance of the baseline method studied in this paper is underlined by its frequent application in the recent NIPS competition on adversarial examples.
Decision: Accept (Poster)