Venue: ICLR 2020 Conference

Reviews:

final_score: rating: 8: Accept
review_content: This paper presents an adversarial attack based on the feature representations at different layers given the classes. Instead of only looking at the final layers, class samples in intermediate feature space information is used to attack and increase transferability at the same time. Then the noise is optimized to perturb the input so that the a specific wrong output will be more likely to chosen. The paper is clear and well-written and different interesting experiments support the claims.

1-	Intermediate feature space is not fully independent of the architecture of  the model. However, the results in Figure 2 shows schematically the same behavior for tSuc for different models. I am wondering how different the results would be if adversarial attack was trained on the black box model. In other words, as a simple testing for example, how much higher the success rate would be in the case of DN121->RN50 if you trained the noise on RN50 itself?
2-	For different scenarios explained in figure 2, the optimal layer is usually one of the intermediate ones, and it might change from case to case. To find the optimal layer, you needed to have access to the black box model. How possible do you think it is to make it black-box model agnostic? Maybe at the cost of less success?
3-	In the Figure captions you have “Figure” and in the text it is Fig. please make it consistent.

review_title: Official Blind Review #1

final_score: rating: 3: Weak Reject
review_content: 
The paper proposes a new transfer attack method (on undefended models), called FDA. The method uses auxiliary class-wise binary classifiers attached on intermediate layers, and optimizes the targeted probability of the binary model. 

The proposed method is simple and shown effective according to the results in Table 1. But I have some serious concerns about the method and results.

Concerns
1. One weakness is the paper didn't seem to justify why using binary classification models on intermediate features is better than just using the final classification result in the original model. The authors should have an apple-to-apple comparison with optimizing the probability using the original model, including the "ms" variant, and also explain why using the auxiliary model is better than that. Even if you use the original model it is still utilizing the distribution of intermediate features, just through the main branch, so this reason alone does not seem convincing to me.

2. The baseline results seem too low (tpgd and tmim in Table 1, <10% target matching success rate). The FDA's result is ~20%. I'm not too familiar with the targeted adversarial attack literature, but one of the very early paper on targeted attack [1] seems to report much higher target matching rate (e.g., see their Table 3, ~70,80%). Why is this the case? What am I missing here? Is this just because they used an ensemble approach? If so you should also use ensemble to compare. Also the authors are welcome to want to point me to more recent papers' results and show their paper's result is better, given [1] is back in ICLR 2017.

3. One drawback of the method is it needs to train a custom model by oneself first, with one binary model for each class we want to attack or target. This can be very cumbersome in practice. For 1000 classes in ImageNet it is not possible to cover all of them. But other conventional methods don't need to worry about this, they just operate on a pre-trained model and supports any source and target class. I doubt whether the method is practically useful. This point should be acknowledged in the paper.

Overall, due to the concerns raised above (especially 2), I tend to vote a reject to the paper. I'm happy to reconsider my rating after the discussion period.

[1] Delving into transferable adversarial examples and black-box attacks.

review_title: Official Blind Review #3

final_score: rating: 8: Accept
review_content: The paper proposes a new adversarial attack for the targeted blackbox model Unlike previous approaches which use the output layer possibly with some additional terms and regularization, the proposed approaches only rely on intermediate features. In fact, the adversarial example is based on a single intermediate layer. The adversarial examples are built by training, for each target class, a binary classifier for the class based only on the features of that layer.

- There are multiple similar prior works that use intermediate layers in some way, and the paper does a good job to explain the differences between the proposed approach and these.

The paper proposes three variants. The simplest one appears rather weak in numerical experiments. The two other methods seem to achieve a different tradeoff, between focusing on the general error rate or on the targeted success rate.

In particular, FDA+fd is similar in some ways to Zhou et al. (2018) which uses an additional loss term to maximize the distance of the features at various layers. A comparison with this method (which uses the output layer) would be interesting, and may constitute a relevant additional baseline, at least for non-targeted metrics like error (which is where FDA+ms shines).

Similarly, a comparison to AA is provided in Figure 2 in the case of 10 classes, but not in Table 1 for the 1000 classes experiment. This casts some doubt on the significance of the result in the 1000 classes setting. A major issue with the proposed method is that training binary classifiers in large multi-class settings is quite costly, which is bound to hinder one's ability to identify 'high performing attack settings' In particular, the paper mentions that the 10 classes are used for that, but no evidence is provided that these settings are actually good for all classes.

The conclusions are rather consistent accross different pairs of source/target network.

The paper does not mention adversarial training. As this has become a common defence practice in the literature, it would be interesting to understand to what extent the proposed methods are thwarted by adversarial training. This could increase the significance of the paper if working with intermediate feature representation makes the attack more robust to this type of defense.

In summary, I find the experiments satisfying, although they can definitely be made more convincing by adding additional strong baselines and demonstrating how much performance can really be attained in the 1000 classes scenario.

- Generally, not using the output layer at all is an interesting approach that deserves in my opinion to be discussed and investigated further.

- The paper is clearly written, and notation is rigorous.
review_title: Official Blind Review #2

Meta-review: This paper considers black box adversarial attacks based on perturbations of the intermediate layers of a neural network classifier, obtained by training a binary classifier for each target class.

Reviewers were happy with the novelty of the approach as well as the presentation, described the presentation as rigorous and were pleased with the situation of this method relative to the literature. R3 had concerns about evaluation, success rate, and that the procedure was "cumbersome".  Some of their concerns were addressed in rebuttal, but remained steadfast that the method was too cumbersome to be practical.

I agree with R1 & R2 that this approach is novel and interesting and disagree with R3 that it is too impractical. The paper could be stronger with the addition of adversarial training experiments (and I disagree with the authors that "there are currently no whitebox attacks that do well at attacking AT models", this is very much not the case), but I concur with R1 & R2 that this is interesting work that may stimulate further exploration, enough so to warrant acceptance.
Decision: Accept (Poster)