Venue: ICLR 2020 Conference

Reviews:

final_score: rating: 3: Weak Reject
review_content: The paper proposes to construct adversarial attacks by training a neural network to produce a distortion, rather than by constructing the distortion directly via PGD. It then uses this during adversarial training to produce networks that it purports to be more robust.

Pros:
-Faster speed of adversarial training could substantially democratize ability to work on adversarial ML
-Neural net attack could overcome gradient masking in some situations

Cons:
-Some of the main arguments seem incorrect, although the paper may still be strong if these incorrect arguments are removed and the experiments are further substantiated.

Details:
I think there are some interesting ideas here, which if further substantiated could be the base of a strong paper. However, I found the overall argument in the current paper to be misleading or at least mistaken. A core claim is that the neural-net generated attack is better than PGD because PGD only generates a "single attack" while the neural network generates many different attakcs. But PGD is an adaptive attack method that depends on the current model parameters, so it changes over the course of training just like the neural net attacker. Furthermore, both models approximate the same maximization over the L2 ball (in the neural net's case, rather than maximizing over the L2 ball we penalize the L2 norm, but a similar modification can easily be made to PGD). So it is not clear why we should expect the neural net to perform better, except perhaps that it is a different type of way of approximating the same objective that PGD approximates, and perhaps this approximation has some favorable properties? But this is never justified.

Here is one attempt to justify why the neural net attack should be better: perhaps while PGD often works well, it gets stuck on certain examples, e.g. due to gradient masking issues. While the neural net might initially get stuck on those examples (it also has to work by taking gradients), since it can share knowledge across examples it initially learns a good direction for the "stuck" examples by generalizing from the "unstuck" examples. This overall allows it to generate a more consistently good attack even in the presence of (partial) gradient masking. If we believe this story, then actually the neural net attacker should already work well just as an attack (i.e. just train it to generate good attacks against a fixed model for some set of examples). I would find this quite interesting if true but unfortunately it wasn't explored in the paper. One way to do it would be to take models that are believed to be robust based on attacking them with PGD, and then show that the neural net attacker is substantially more accurate.

I do find the point about faster speed interesting. This is given as a minor comment but it may be the biggest benefit of the current method. Adversarial training on large-scale datasets like ImageNet is computationally infeasible, and your method could potentially address this and allow people who don't have $100k+ compute budgets to actually work on adversarial ML.

I also have some comments on the evaluation:

-Reporting average accuracy across attacks in the evaluation is an inappropriate summary statistics. Min accuracy would be better, which in this case is zero for all methods. This suggests that you either did not adversarially train appropriately, or used too large of a norm (what norm did you use anyways? Or is this allowing the norm to be unbounded until you fool the model? In that case all the numbers should be zero for any reasonable attack.)

-For evaluation I would rather see the full curve of accuracy vs. allowed attack norm, rather than just a single summary metric. It's too hard to tell what is going on from a single number. It is also hard to compare against adversarial training methods that penalize the L2 norm versus constrain the L2 norm, as the former might do better than the latter simply due to more closely conforming to your evaluation of average distortion. Having the full curve helps better assess this. Average distortion is also only meaningful if the method is finding the minimum-norm attack point that changes the label, which most of the methods you consider do not do. You do include the full curve in Figure 4 (please make the font bigger though). Under the full curve it seems the improvement over other methods is minimal, and could possibly be due to hyperparameter tuning.

Finally, more minor but the Gaussian derivation does not make sense. It is unlikely that your norms actually follow a Gaussian distribution. A more appropriate claim would be that you are constraining or penalizing the expected norm of the perturbations. I would remove the math part as it does not add anything to the paper (and is also wrong as per preceding comment) and focus more on how you actually construct the generator network (this is only briefly discussed in the appendix and not at all in the main text, even though it is a key point to getting the method to work). You could also use the space for more detailed experiments, following best practices as in https://arxiv.org/abs/1902.06705 to ensure that your evaluation is sensible.
review_title: Official Blind Review #3

final_score: rating: 3: Weak Reject
review_content: This paper proposes to use the GAN framework for adversarial training. The proposed algorithm is mini-max loss plus L2-regularization on the perturbations generated by a generator network. Additionally, the paper shows a mathematical interpretation of the L2-term. Experiments on CIFAR-10 and CIFAR-100 shows that the algorithm achieved higher clean/adversarial accuracy in the settings.

I vote for rejection. A major difference between the proposed method and adversarial training variants is whether we use neural networks or gradient-based algorithms to calculate the internal maximum in the mini-max formulation of the robust training. The focus of discussions in this paper should be why the former performs better. However, this paper does not provide enough justifications. Experiments are not convincing to support its advantage.

Major comments:
(1) On page 2, this paper claims, "The gradient descent based adversarial examples for robust optimization is not adaptive. Therefore, those neural networks are vulnerable to other types of adversarial attacks (Athalye et al., 2018)." Please expand these two sentences.
(2) It is weird to observe that adversarial training with PGD performs worse than FGM. It is possible that the baselines are weaker than they should be. For example, if evaluations use L2-based adversarial training as baselines, it can be improper. Please refer to this comment on OpenReview: https://openreview.net/forum?id=Hk6kPgZA-&noteId=BJVnpJPXM .

======= Update =======

Thank you for the replies and updates. After reading the comments and the updated draft, I am still not convinced that the proposed method should be superior to existing attacks. Even though I agree that the proposed method might outperform PGDs or other attacks in some settings, the paper needs to confirm it either theoretically or empirically. However, it was doubtful whether the baseline methods are strong enough. The concern remained after authors' responses. Hence I keep my score.
review_title: Official Blind Review #2

final_score: rating: 3: Weak Reject
review_content: This paper introduces a new adversarial training approach, where a generator is used to generate the most challenging adversarial examples and the classifier is trained to correctly classify the generated adversarial examples. In this way, the robustness of the classifier is expected to be improved. For the generator, the input includes the original data sample, the gradient of the classifier, and the true label of the data sample. The paper is clearly presented and easy to follow. The experimental results relatively support the main claims of the paper.

The reasons that I go towards mild rejection for this paper are as follows:

1. The general idea of improving the robustness of a classifier to by feeding adversarial examples to it is not a new idea. As shown in the paper, the idea has been adopted in Goodfellow et al. (2015) and Madry et al. (2018). In addition, [1] comprehensively studied how to ensemble different adversarial attacks as "training data" of a classifier to improve its robustness. The paper generates adversarial examples with a generator that takes various things as inputs, which can be viewed as an extension to the method in [2]. [2] introduced a generator that only takes the data sample as inputs and the difference to the paper is that the one in this paper additionally takes gradients and label as inputs. With Goodfellow et al. (2015), Madry et al. (2018), and [1], it may not be too hard to apply the method in [2] to improve classifier robustness. Therefore, it is questionable whether the additional information used in the generator of this paper helps and how it helps. I did not see any comparison in the experiments.

2. Although the experiments look to be well conducted, it is not comprehensive enough. For the defence methods in comparison, only two approaches that fall into the exact same line of the proposed method are included. In this line, I think it is necessary to compare with [1], which combines various attacks. Moreover, I would also expect a comparison with other closely related methods such as [3] and [4], to further demonstrate the effectiveness of the proposed one.

3. It is also suggested to do ablation tests on the inputs of the generator to see how each part of the inputs helps. If gradients and true label are removed from the generator, it reduces to [2]. Therefore, those ablation tests also help the comparison with [2].

Minor comments:

1. It could be interesting to visualise the perturbations that are generated from the generator.

2. Some of the figures and captions are too small to see in a printout.


[1] Tramèr, Florian, Alexey Kurakin, Nicolas Papernot, Ian Goodfellow, Dan Boneh, and Patrick McDaniel. "Ensemble Adversarial Training: Attacks and Defenses." (2018).

[2] Xiao, Chaowei, Bo Li, Jun-Yan Zhu, Warren He, Mingyan Liu, and Dawn Song. "Generating adversarial examples with adversarial networks." arXiv preprint arXiv:1801.02610 (2018).

[3] Samangouei, Pouya, Maya Kabkab, and Rama Chellappa. "Defense-GAN: Protecting Classifiers Against Adversarial Attacks Using Generative Models." (2018).

[4] Matyasko, Alexander, and Lap-Pui Chau. "Improved network robustness with adversary critic." In Advances in Neural Information Processing Systems, pp. 10578-10587. 2018.
review_title: Official Blind Review #1

Meta-review: This paper proposes to use the GAN (i.e., minimax) framework for adversarial training, where another neural network was introduced to generate the most effective adversarial perturbation by finding the weakness of the classifier. The rebuttal was not fully convincing on why the proposed method should be superior to existing attacks.
Decision: Reject