Venue: ICLR 2020 Conference

Reviews:

final_score: rating: 3: Weak Reject
review_content: In this paper, the authors proposed two methods of Nesterov Iterative Fast Gradient Sign Method (NI-FGSM) and Scale-Invariant attack Method (SIM) to improve the transferability of adversarial examples. Empirical results on ImageNet dataset demonstrate its effectiveness. In general, the paper is clearly written and easy to follow but I still have several concerns:
1.	Although the method is easy to understand, the authors are expected to clarify why the methods can improve the transferability. The authors are expected to make more theoretical analysis.
2.	The authors are expected to make more comprehensive comparisons with the recent methods in adversarial attacks, e.g, PGD, and C&W even if some methods are designed for white-box attack. 
3.	The authors are expected to make more evaluations on the models with defense mechanism, and numerous important methods are missing. Without this, the authors cannot claim its effectiveness since only experiments on NIPS2017 is not enough.    

review_title: Official Blind Review #1

final_score: rating: 6: Weak Accept
review_content: In this paper, the authors apply the Nesterov Accelerated Gradient method to the adversarial attack task and achieve better transferability of the adversarial examples. Furthermore, the authors introduce a scale transformation method to provide the augmentation on the model, which also boosts the transferability of the attack method. Experiments are carried out to verify the scale-invariant property and the Nesterov Accelerated Gradient method on both single and ensemble of models. All experiments turn out to be a positive support to the authors' claim.

However, one small drawback of this paper is that the author does not claim any comparison between the Nesterov Accelerated Gradient Method and other momentum methods (e.g. Adam, momentum-SGD, etc). This experiment is somehow important since it shows the better transformability is obtained from 1) Nesterov Accelerated Gradient Method only, or 2) all momentum method, which is significant for further research.

Also, in the setting of the Scale-Invariant Transformation, the authors forget to address that what if the attacked network has an input normalization. Does it mean to downsample the value of each pixel in the input image? If so, is the equation $S_i(x) = x / 2^i$ better to be $S_i(x) = [x / 2^i]$ where $[]$ means casting to the nearest integer? 

One more question of this work is:  The Nesterov Accelerated Gradient method is known for its proveable fast descent property comparing to the traditional Gradient method. Do you observe any speed-up during your training? 
review_title: Official Blind Review #3

final_score: rating: 6: Weak Accept
review_content: This paper studies how to generate transferable adversarial examples for black-box attacks. Two methods have been proposed,  namely Nesterov Iterative Fast Gradient Sign Method (NI-FGSM) and Scale-Invariant attack Method (SIM). The first method adopts Nesterov optimizer instead of momentum optimizer to generate adversarial examples. And the second is a model-augmentation method to avoid "overfitting" of the adversarial examples. Experiments on ImageNet can prove the effectiveness of the proposed methods.

Overall, this paper is well-written. The motivation of the proposed methods are generally clear although I have some questions. The experiments can generally prove the effectiveness.

My detailed questions about this paper are:
1. The motivation in Section 3.1, which regards generating adversarial examples as training models, and transferability as generalizability, is first introduced in Dong et al. (2018). The authors should acknowledge and refer to the previous work to present the motivation.
2. It's not clear why deep neural networks have the scale-invariant property. Is it due to that a batch normalization layer is usually applied after the first conv layer to mitigate the effect of scale change?
3. It's not fair to directly compare DIM with SI-NI-DIM (also TIM vs. SI-NI-TIM; TI-DIM vs. SI-NI-TI-DIM), since SI-NI needs to calculate the gradient over 5 ensembles. It's better to compare the performance of two methods with the same number of gradient calculations.
4. Is there an efficient way of calculating the gradient for scale-invariant attacks like translation-invariant attacks in Dong et al. (2019)? 
review_title: Official Blind Review #2

Meta-review: Under the optimization formulation of adversarial attack, this paper proposes two methods to improve the transferability of adversarial examples, namely Nesterov Iterative Fast Gradient Sign Method (NI-FGSM) and Scale-Invariant attack Method (SIM). NI-FGSM adapts Nesterov accelerated gradient into the iterative attacks to effectively look ahead and avoid the “missing” of the global maximum, and SIM optimizes the adversarial perturbations over the scale copies of the input images so as to avoid “overfitting” on the white-box model being attacked and generate more transferable adversarial examples. Empirical results demonstrate the effectiveness of the proposed methods. The ideas are sensible, and the empirical studies were strengthened during rebuttal.
Decision: Accept (Poster)