Venue: NeurIPS 2021 Conference

Reviews:

final_score: rating: 5: Marginally below the acceptance threshold
review_content: summary: One major challenge in the research on adversarial attacks is the absence of methods that can reliably evaluate an adversarial defense. One the one hand, many defense approaches end up targetting particular attack methods. On the other hand, some promising defense approaches are viewed skeptically due to possible new attacks that could break them even if none currently exists. This paper addresses this critical challenge of developing adaptive adversarial attacks by combining some basic building blocks based on a particular defense. The presented approach is shown to outperform AutoAttack - a popularly used baseline attack by community developing adversarial defenses. 

The submission presents an automated approach to the discovery of adversarial attacks (along the lines of AutoAttack) by exploring the search space of adversarial attacks formed by varying three components: (i) parameterized attack algorithms, (ii) network transformations, and (iii) loss functions. 

Overall, the paper is an interesting improvement over the state-of-the-art and clarification to some questions below will help the reviewer better appreciate its overall contribution. 
main_review: Strengths

+ The use of parameter search algorithms to tune adversarial attacks generated by a formal grammar is likely to be useful. More discussion on the parameter search and its efficacy/challenges may be helpful.

+ The paper includes a meticulous comparison with state-of-the-art. In particular, it highlights the key distinctions from AutoAttack and another recently proposed adaptive attack (Tramer et. al. 2020). 

+ The paper has identified dimensions of different adaptive attack methods (attack algorithm params, network transformations,  loss functions), which could be possibly reused to develop new adaptive attack techniques. 

+ The implementation is available open-source. 


limitations_and_societal_impact: Weaknesses

(-) The idea of replacing a human expert attacking a model with an algorithm is interesting. However, it is not clear if an automated approach can be as creative as a human expert. For example, layer removal when the inputs and outputs have the same shape (lines 128-129) can be easily circumvented by a human expert by adding redundant inputs or outputs that this automated tool may not discover. Can the network transformation take care of such cases? 

(-) The experiment results on the top models (A1 and B10-B14) in Table 1 do not show very significant improvement over AutoAttack. Improvements against Madry et al (A1) is 0.9 and against robust models (B10-B14) is below 0.1. Were any ablation studies performed on models (B10-B14)?

(-)  The grammar presented here to represent the space of possible attacks  is quite simple. Would the defense algorithms not be able to exploit the knowledge of this grammar?
review_title: null

final_score: rating: 4: Ok but not good enough - rejection
review_content: summary: This paper claim that adaptive attacks are composed from a set of reusable building blocks that can be formalized in a search space and used to automatically discover attacks for unknown defenses. Based on this observation, they propose an automated framework of discovering adaptive attacks, which does not rely on the expert's knowledge priori.
main_review: **Originality:**
The work is meaningful in engineering, but its contribution and novelty are limited.

Q1: There is a problem with the motivation of this paper, that is, this work believes that adaptive attacks in Tramer et al. (2020) require expert knowledge, so A^3 is proposed. But A^3 does not have experiments to prove that it is superior to an adaptive attack based on expert knowledge. Pang et al. (2019) and Xiao et al. (2020) in Tramer et al. (2020) are both close to 0% after adaptive attacks, and A^3 cannot do this. So, what is the goal of this paper? Perhaps the strongest adaptive attack still requires experts to design.

Q2: Composite Adversarial Attack (CAA) [1] is proposed for automatically searching the best combination of attack algorithms and their hyper-parameters from a candidate pool of 32 base attackers. The experimental result shows CAA beats 10 top attackers on 11 diverse defenses with a less elapsed time (6 × faster than AutoAttack) and achieves the new state-of-the-art on l∞, l2, and unrestricted adversarial attacks. Obviously, CAA and this work have similar ideas, and both are to evaluate the robustness of the model, but this work is not compared with CAA, nor does it cite CAA, even if CAA’s code is open.

**Quality:**
The experiment in this paper proves that it is better than AutoAttack, but there are several problems:

Q3: AutoAttack is a parameter-free, computationally affordable, and user-independent ensemble of attacks to test adversarial robustness and not specifically designed for adaptive attacks. From this perspective, if this work wants to highlight the advantages of adaptive attacks, the comparison with AutoAttack is unfair, even though AutoAttack has been improved with adaptive attacks. A better way is to explicitly compare with existing or adaptive attack methods in the original paper.

Q4: A^3 is closely related to the recent advances in AutoML, such as in the domain of neural architecture search (NAS). How to reflect the advantages of your search strategy? What’s more, the combination of NAS and adversarial attacks already exists (CAA[1]) and should be compared with it.

Q5: In the body, the experiment only has the setting of L∞, can A^3 be extended to other L-norm settings? There are other L-norm experiments in AutoAttack and CAA[1].


**Clarity:**
The paper is clear on the whole. 

**Significance:**
This work is meaningful in engineering applications. If it is superior to adaptive attacks based on expert knowledge, it is vital for reliable evaluation of adversarial defenses.


[1] Xiaofeng Mao, Yuefeng Chen, Shuhui Wang, Hang Su, Yuan He, Hui Xue: Composite Adversarial Attacks. AAAI 2021: 8884-8892
limitations_and_societal_impact: The authors have addressed the limitations and potential negative societal impact of their work.
review_title: null

final_score: rating: 5: Marginally below the acceptance threshold
review_content: summary: This paper's main premise is that existing defense evaluations (whether adaptive or based on an ensemble of fixed attacks) only rely on a small number of common techniques and that a search over the space of possible attacks can efficiently uncover strong attacks.
main_review: The automatization of defense evaluations is ultimately a good thing, I think, if it leads to fewer clear mistakes and omissions in these evaluations. The proposal in this paper is quite natural and straightforward, in that it combines existing techniques and evaluates them against previously-evaluated defenses. To make this contribution more compelling, the paper could benefit from expanding along the following two axes:

1) Generality

Existing attacks indeed seem to make use of only a small number of different components. This is especially true for defenses that don't do anything "weird" at evaluation time. And many of the defenses (re-)evaluated in this paper are of this form (e.g., all forms of adversarial training).

But now let's imagine that a new defense comes out, that hasn't been independently evaluated before.
Should we trust the procedure outlined in this paper to find a strong attack?
I encourage the authors to explicitly think about this next step: what would it take to design a defense that is (a) non-robust, yet (b) resits your automated attack procedure?
My guess is that it wouldn't be too hard to build such a defense, and that this should be acknowledged.

The paper's claim that it automates the manual search for attacks (e.g., of Athalye et al. or Tramer et al.) is also hard to assess, as the proposed attack search is only evaluated against a small subset of the defenses evaluated in those works. Unless I am mistaken, I count 3/9 defenses in common with the work of Athalye et al. and 3/13 in common with the work of Tramer et al.

It thus seems challenging to reach a conclusion on whether automated search techniques could have found good attacks for some of these challenging defenses. And as noted above, we also don't know how well the proposed search procedure works against defenses that haven't been re-evaluated before.

2) Ease-of-use

Related to the above discussion on the attack's generality is the question of its ease-of-use. My worry here is that the more general an automated attack aims to become, the harder it will become to use as it will inevitably require the defense to be represented in a tool-specific way.
E.g., certain defenses may not be designed in a way that makes it easy to replace individual components with a BPDA approximation. Or some defenses (e.g., detectors) may produce outputs that cannot immediately be converted into the type of score your attack expects.
Is there reason to believe that this (manual) process of adapting a defense to an automated attack tool is necessarily easier and less error-prone than the implementation of an appropriate attack?


Additional comments:

- how do you automate the process of applying EOT? Often, different defenses will rely on very different forms of random transformations.
- For the comparisons in Table 1, it would be worth including some confidence intervals when comparing A3 and AA. E.g., for all the B* models, the difference between the two attacks is most certainly not statistically significant.
- Why do you include attacks such as FGSM or DeepFool in your search space, when these are known to super-seeded by newer stronger attacks?
limitations_and_societal_impact: As stated in the main review, the generality of the approach should be further discussed, as well possible limitations against future defenses.
review_title: null

final_score: rating: 6: Marginally above the acceptance threshold
review_content: summary: The paper advances state-of-the-art automated pen-testing tools for breaking defenses designed to protect deep neural networks against norm-based adversarial attacks as a search problem. The search process first considers techniques to simplify the network being attacked (called Network Transformations), followed by a search over the space of existing attacks (and common ways to modify these attacks). Given the search space is large, the authors try to improve the efficiency by leveraging sub-sampling methods.
main_review: ### Strengths

- The paper is easy to follow.
- The paper structures the idea of adaptive attacks into a search algorithm, making existing tricks leveraged in attacks simply a search node in the space of attacks.
- The results show that, in certain cases, their improvement to AutoAttack can result is finding stronger or faster attacks. I appreciate that they to do a fair evaluation, they also empower AutoAttack with some additions (such as BPDA) that their framework leverages.
- The analysis of how the attacks found relate to existing observations made by existing research was an intersting read.
- The ablation studies does do help to ensure that various parts of the search algorithm do indeed help in finding the best attack.

### Weaknesses / Correctness / Clarification

- The paper adds a bunch of bells-and-wistles to AutoAttack and is more of an automated pen-testing software paper; I could not find much novelty or insights for improving the attack of deep neural networks. While auto attack proposes two extensions of PGD and then considers an ensemble approach, the contribution of this paper seems to be a search process formulated over the space of existing attacks (modifications thereof), and network transformations etc.
- $A^3$ hardly provides improvement over adversarial training based defenses. This indicates that without novel attacks (lines 233-235), the search procedure may not be add any value in finding the best attack against SOTA defenses, tying back to the lack of novelty pointed above.
- While reading, I kept thinking if a meta-learning approach to the search would be an interesting consideration. I found the authors mentioned this as future work at the very end. I surely think using something similar to meta-learning would add more novelty that simply bucketing existing tips and tricks at different layers of a computationally expensive search process.
- Why did the authors evaluate on a sub-set of the defenses on which AutoAttack was evaluated? It sort of gives the impression that $A^3$ did not provide any gains compared to AA (w.r.t accuracy or time) on the defenses omitted.
limitations_and_societal_impact: The idea of developing automated pen-testing tools is often discussed in the context of correctly calibrating the security offered by existing defenses. At the same time, it benefits an attacker, who now has to spend less resources if figuring out the best attack against a defense.

While this paper is not solely responsible in this regard, we should consider a better way to release such methods (say, learn from the best practices in cybersecurity). In cybersecurity, vulnerabilities often found by automated pen-testing software have know fixes/patches; this is not presently the case for deep neural networks (esp. if we consider real-world attacks and defenses). Thus, we (as a community) need to devote more thought before arming attackers with such automated software, which must consider re-evaluating research progress in this direction starting from the impact of releasing code to accepting such papers etc.
review_title: null

Meta-review: This paper introduces a technique to better perform adaptive attacks on adversarial example defenses. Current evaluations are large done by hand, and the paper here introduces a technique to automate some of that process. While the reviewers all like the overall tone of the paper, the reviewers are generally concerned about three aspects of the paper.

First, the paper does not demonstrate that it can automate all possible hand-specified attacks (for example, it has a limited design space that might not identify redundant neurons).

Second, the paper considers only a subset of the attacks used in prior papers. And so saying that this attack can match prior work may not actually be valid.

Third, the attack may not yet be practical to use for defenders.

Taken as a while, while I agree that these are definite limitations to this technique, this is the first paper to try these kinds of automated adaptive attacks and is worth publishing. While the implementation is imperfect at the moment, and is more work to be done on evaluation and improved attack, this can be done with followup work. If it turns out that this idea is not actually better than the human-designed attacks, future work could help identify the cases where this is true or improve the attack to make it more effective. Even though this attack is not likely practically useful for defenders yet, the new direction it opens might help in the future.
Decision: Accept (Poster)