Venue: ICLR 2020 Conference

Reviews:

final_score: rating: 6: Weak Accept
review_content: This paper introduces a parameterized approach to generate adversarial samples by balancing the speed-distortion trade-off. The method first tries to reach the boundary of classes in the classifier space, then walks on the classifier manifold to find adversarial samples that make the classifier to fail in prediction while minimizing the level of distortion in the sample. Having a limited number of iterations, the method reduces the fluctuations around the boundary and paves the classification manifold.

The idea is novel, interesting and well-formulated, while the intuition could be better explained. The paper is a good read, has an adequate amount of literature review, and the results are supporting the claims of the paper: lower distortion while having comparable accuracy, the use of generated samples in fortifying the classifier, and keeping distortion to a reasonable level (qualitative results in appendix). However, one of the claims is to trade the distortion level to speed that needs verifying in the main manuscript, therefore, it is suggested that the section B.1 moves to the main manuscript and discussed more thoroughly. Also the effect of other parameters on this trade-off (such as the number of iterations K).

It is also interesting to discuss how the algorithm performs in classes that are linearly separable on a toy dataset.
review_title: Official Blind Review #3

final_score: rating: 3: Weak Reject
review_content: This paper proposed an adversarial attack method based on optimization on the manifold. The authors claim it is a fast and effective attack even with quantization. 

It would better to also evaluate the method on the state of the art robust models (such as Madry et al ICLR'18) instead of only testing it on natural models. Generating adversarial examples on natural models is rather a well-solved problem and I do not think a 0.1 decrease in L2 norm is a big contribution since it is already so small that humans cannot distinguish. A better way to prove the strength would be to test it on a robust model to achieve higher success rates given a maximum distortion.

I do not think the results in Table 3 are convincing or necessary. It is well-known that the FGSM is so weak that the adversarial examples produced by it are not strong enough for adversarial training. The state of the art adversarial training defense uses the adversarial examples obtained from PGD. Also, a popular way to evaluate model robustness would be to evaluate the attack success rate under a given upper bound of distortion (e.g. 0.3 for MNIST). If there is no constraint on the distortion, we can always achieve a 100% attack success rate by simply use an image from another class. So in Table 3, the authors may either make sure all attacks have a 100% success rate and compare the distortion, or set an upper bound of distortion and compare the success rate (just as in the operating characteristics plot). With the current results, I do not believe the robust training with BP can be any better than FGSM. Similar issues also exist in Table 2. 
review_title: Official Blind Review #2

final_score: rating: 3: Weak Reject
review_content: This paper considers efficiently producing adversarial examples for deep neural networks and proposes boundary projection (BP), which quickly searches an adversarial example around the classification boundary. The BP approach is tested on three benchmark datasets and compared with existing adversarial attacking methods.

The key idea of BP, searching on the class boundary manifold, is interesting and promising. However, despite the excess of the recommended 8 pages, the main parts of the proposed method are not so clearly explained.

- It is not so clear which parts of the proposed method (Section 3) are mathematically justified. For example, \gamma_i in Eq. (14) looks heuristically introduced.
- Although the abstract and introduction emphasize that the main focus of BP is speed-distortion tradeoff, the experiments section does not discuss it so much and so clearly. While the operating characteristic of probability of success and distortion is mainly discussed, it is unclear which argument most demonstrate the improvement in speed-distortion tradeoff.

p.5, l.7: 1(a) -> Figure 1(a)
p.8, l.10: measure measure
p.8, right after Eq. (21): `"`is conditioned is" -> ``"is conditioned


review_title: Official Blind Review #1

Meta-review: In this paper the authors highlight the role of time in adversarial training and study various speed-distortion trade-offs. They introduce an attack called boundary projection BP which relies on utilizing the classification boundary. The reviewers agree that searching on the class boundary manifold, is interesting and promising but raise important concerns about evaluations on state of the art data sets. Some of the reviewers also express concern about the quality of presentation and lack of detail. While the authors have addressed some of these issues in the response, the reviewers continue to have some concerns. Overall I agree with the assessment of the reviewers and do not recommend acceptance at this time.
Decision: Reject