Venue: ICLR 2020 Conference

Reviews:

final_score: rating: 3: Weak Reject
review_content: The authors propose to use a generative adversarial network to train a substitute that replicates (imitates) a learned model under attack. They then show that the adversarial examples for the substitute can be effectively used to attack the learned model. The idea is straightforward. The proposed approach leads to better success rates of attacking than other substitute-training approaches that require more training examples. Promising experimental results against decision and score-based attach schemes also demonstrate the effectiveness of the proposed approach.

My main concern is that the comparison to other approaches seem unfair, because

(1) Substitute models typically use all training data and query the learned model once per training example; the proposed approach uses fewer training data but needs 1800 queries per example. So it is not surprising that the proposed approach can be better than substitute models. As the authors focus on "practical" scenarios, the number of queries should be fixed as a constraint for all approaches to be fair.

(2) Having said (1), there is not enough detail in this paper to understand the similarity and difference between the proposed approach and decision-based ones (which is claimed to have similar query complexity during training). The authors mix the results with score-based (requiring more information), making the results more confusing to the readers. The authors are encouraged to present more detailed discussions on the comparison with decision-based competitors.

Also,

(3) One thing that is missing from the experiments is how well the replica clones the original model. All the information in the experiments are somewhat "indirect" (success rate, test accuracy, etc.) to answer this question, but there is no direct evidence. Is a well-cloned replica really a better substitute to construct adversarial examples? For instance, for replica with different "cloning accuracy" (rather than test accuracy), is a better replica also a better substitute? The paper fails to answer this question that best matches its design motivation.

The above are my main concerns. A minor one is

(4) The abstract that directly uses notations like T, G and D is horribly hard to read.

review_title: Official Blind Review #2

final_score: rating: 3: Weak Reject
review_content: Authors propose a GAN-based adv imitation attack that can use less training data to produce a replica of the model.

The idea of using a GAN in producing adv examples in quite interesting. But the proposed approach is closely related to the following paper:

https://arxiv.org/pdf/1801.02610.pdf

Therefore, I am not sure about the novelty of the proposed approach.

Also can authors comment on the stability of GAN's training? Are any stabilizing methods integrated in GANs being used? 

How does the proposed approach relate to the adversarial distillation literature? 
review_title: Official Blind Review #1

final_score: rating: 6: Weak Accept
review_content: This paper proposes a new approach to conduct adversarial attacks, where an imitation classifier is trained to mimic the behaviours of the targeted/attacked classifier and adversarial attacks can be generated with existing attack methods on the imitation classifier. The training of the imitation classifier only requires the predictions of the targeted classifier, which may fit better in practice with limited access to the targeted classifier.

The reasons that I am going towards accept are as follows:

1. I feel that the idea of learning an imitation classifier is somehow novel and intuitive, which would improve the applicability of existing adversarial attacks in more realistic cases. In addition, using GAN framework in the training of the imitation classifier is also interesting.

2. The experiments are quite comprehensive and promising, including the comparisons with gradient-based attacks as well as the decision-based attacks. In addition, the different model configurations of the imitation classifier are also reported.

Suggestions:

1. It is a bit unclear of the model configuration of the generator, which seems to be not introduced in details.

2. It would be interesting to visualise what the generator generates.

Minor:

"sometimes the ability of G is much stronger than G"
review_title: Official Blind Review #3

Meta-review: This paper proposes to use a generative adversarial network to train a substitute that replicates (imitates) a learned model under attack. It then shows that the adversarial examples for the substitute can be effectively used to attack the learned model. The proposed approach leads to better success rates of attacking than other substitute-training approaches that require more training examples. The condition to get a well-trained imitation model is that a sufficient number of queries are obtained from the target model. This paper has valuable contributions by developing an imitation attacker. However, some key issues remain. In particular, I agree with R1 that the average number of queries per image is relatively high, even during training. In the rebuttal, the authors made the assumption that “suppose their method could make an infinite number of queries for target models”, which is unfortunately not realistic. Another point that I found confusing: at testing, I don’t see how you can use the imitation model D to generate adversarial samples (D is a discriminative model, not a generator); it should be G, right?

Decision: Reject