Venue: NeurIPS 2021 Track Datasets_and_Benchmarks Round2

Reviews:

final_score: rating: 8: Top 50% of accepted papers, clear accept
review_content: summary and contributions: This paper contributes RobustBench, a benchmark for adversarial robustness. The benchmark uses a standardized parameter-free attack called AutoAttack to give an upper bound for robustness, which obviates the need for third parties to perform time-consuming adaptive evaluations per-defense. For defenses that follow a set of restrictions, AutoAttack generally gives reasonable bounds. RobustBench does not currently support defenses that violate these restrictions. In case AutoAttack doesn't give a tight bound on a particular defense, RobustBench allows third parties to submit results from an adaptive attack.

In addition to RobustBench, the paper contributes a model zoo for adversarially robust models, giving researchers a unified interface to over 80 robust models proposed in the literature. Finally, the paper gives some analyses of robust models that are enabled by the unified API of the model zoo.

strengths: Given that there are over 3000 papers written on the topic of adversarial examples, with a large subset of these papers proposing new techniques to train robust models and defend against adversarial examples, the community is in need of a systematic way to track and compare these defenses. RobustBench is perhaps the best attempt to date to do this, overcoming the limitations of prior work like robust-ml.org that attempted to solve this problem.

RobustBench makes several important insights. One is that many reasonable defenses satisfy a reasonable set of restrictions: (1) differentiable, (2) deterministic, and (3) free of optimization loops. The other is that AutoAttack, a parameter-free attack that performs variations of several well-known white-box and black-box attacks, often gives reasonably good upper bounds for defenses that satisfy that set of restrictions.

Basing a benchmark/leaderboard on AutoAttack allows for reasonable comparison between defenses even when the defenses haven't been carefully analyzed by third-party researchers seeking to perform an adaptive attack, which makes the benchmark much more useful overall. Adding a new defense to the benchmark is easy and useful. RobustBench recognizes that AutoAttack isn't perfect, and so it supports submissions of adaptive attacks on specific defenses as well.

RobustBench already appears to have a demonstrated usefulness and traction among the community.

weaknesses: The single biggest concern I have with RobustBench's approach is that the popularity/acceptance of RobustBench as _the_ way to evaluate adversarial robustness may lead to future work on adversarial examples de-emphasizing adaptive attacks and "overfitting" to AutoAttack. I know that RobustBench does accept adaptive attacks, but do the authors have ideas on whether greater emphasis should be placed on those, and if so, how? Could the leaderboard website better emphasize adaptive attacks? A related question: what is the default sort order for the leaderboard --- is it by AutoAttack accuracy, or best known robust accuracy?

Another conern I have is with the restrictions: how can the restrictions be enforced? Some should be possible to check in a straightforward way, e.g. ensuring that the model does not use randomness, but how will the property of non-zero gradients be enforced? It should be easy to catch models that are "obviously" non-differentiable, e.g. models using quantization, but what about models that are technically differentiable but often end up having "non-useful" gradients, e.g. due to numerical instability? Such models could "break" AutoAttack without actually being robust.

Minor concerns:

- The abstract mentions "reasonable computational budget". What does this mean? Is it even necessary to include as part of the threat model? The threat model already bounds perturbation distance under an Lp norm.
- How reliable is the "flag" added to AutoAttack? For defenses that satisfy the three restrictions, how often is it the case that an adaptive attack performs significantly better than AutoAttack, and in how many of those cases did AutoAttack raise a flag? Is there a danger of the flag being misleading with false negatives?
- Why does the benchmark allow submitting a defense without publicly sharing the checkpoint? It seems like that would make it hard/impossible for other researchers to perform adaptive attacks on the defense.

correctness: Yes. The benchmark design seems like the best possible while having a fully automated attack. The paper has a careful treatment of threat models, empirical evaluations being upper bounds, and adaptive attacks.

clarity: Yes; the paper as well as the website and GitHub issue templates are well-done.

relation to prior work: The paper has a good comparison to prior work on libraries for attacks, as well as prior works on leaderboards/benchmarking of defenses.

additional feedback: - I found the orange text in Figure 1 and the website a bit difficult to read. (The contrast ratio is below the value recommended by WCAG: https://web.dev/color-and-contrast-accessibility/)
- The model zoo seems very useful. Is there any harm in making it a requirement to integrate a model into the model zoo as part of submitting a new defense for inclusion in the leaderboard?
review_title: Well-designed benchmark that fills a need

final_score: rating: 7: Good paper, accept
review_content: summary and contributions: This submission presents a standardized benchmark for empirical adversarial robustness under a set of specific restrictions in order to keep the benchmark and associated leaderbords and model zoo more meaningful, useful and standardized at the (openly discussed) expense of a broader coverage of attack types, defenses, adversarial robustness measures and application settings.  The benchmark consists of a currently continously updated set of leaderbords for adversarial robustness and a model zoo of the tested robust models, all hosted on a website and github. The benchmark exlusively focuses on the problem of adversarial robustness in the context of image classification models under  threat models of $l_{inf}$ and $l_{2}$ perturbations represented by the attacks implemented in the autoattack package, allowing for certain set of defenses in the benchmarked robust models.
strengths: - This work represents a valuable contribution to the ongoing and very active field of adversarial machine learning, and especially to the research into adversarial robustness.
- The associated  - currently continuosly updated and curated -  leaderbords of many published models and the robust model zoo together with the very accessible open source implementation and easily usable wrapping into a python package represents extremely valuable, commendable and probably also sometimes tedious work. All the more so, when one additionally considers the efforts to reach out into the community for contributions and corrections. 
- While the presented limitations and restrictions can seem quite extensive and thus render the direct usability of the benchmark less general and more specifically geared towards image classification problems under  threat models of $l_{inf}$ and $l_{2}$ (with additional restrictions on the listed defenses and no standard exploration of adaptively estimated adversarial robustness, and the almost complete exlusion of verified robustness methods etc.), it is absolutely necessary that this type of work is conducted to advance the field. Thus, I agree with the basic assessment that a standardized benchmark needs to start with a set of specific (standardizable) problems, and the choices seem mostly reasonable to me, i.e. using the most studied case of image classification together with the most widely studied threat models. Of course, this is only true as long as all limitations are clearly stated, which too a large degree is done by the authors (see below for minor criticism).
- With section 4, the study is already providing a range of analyses based on the benchmarks regarding the impact of adversarial robustness in models on a range of subjects such as distributional shifts, out-of-distribution detection, privacy leakage, transferability, especially when considering the rich supplementary material. This work already provides an example of the wide range of analysis that is possible with this kind of benchmark and even draws many smaller conclusions with regard to existing literature.
weaknesses: Apart from the  stated limitations of the benchmark themselves (which i don't necessarily consider as weaknesses) there are two main points of criticism:
- The limitations are not always fully laid out to the reader in a clear way. This is certainly in part due to the complexity of the discussed problem and the limited space, where some limitations just are being stated shortly at a later stage when the necessary background is clear (such as which data sets are used, or which exact model architectures are benchmarked). However,  the title of the publication seems too broad and also the abstract does not mention some of the limitations such as  the considered threat models, how adaptive attacks are considered, the restriction to mostly empirical adversarial robustness and upper bounds, or the restriction to image classification problems. I would encourage the authors to stand more clearly by their own words in the  supplemental materials, section A, where in a rebuttal to a previous reviewer they made the clear point that even creating such benchmark just for the most studied problem of image classification and $l_{p}$ robustness is already very challenging. Indeed!
- The rich supplementary material is both a good and a problematic element. Clearly, the authors have a lot more to present and say than what they could reasonably provide within 9 pages and much of it is interesting and valuable. The specific choice is sometimes a bit unfortunate, because some of the material would enrich the main text, which sometimes borders to a point where the supplementary material actually becomes necessary to fully understand (or at least appreciate) the main text. A clear example is line 64, in which the contributions from further analysis are listed and among them are several topics only addressed in detail in the supplementary material (fairness, transferability).

A few additional smaller points
- it is unclear to me, why the tested models seem to be exclusively based on the Resnet and wideResnet architecture (line 255). Is there a specifc reason for this? And shouldn't that be more openly considered a bias?
- In 257-262 it is elaborated that some defenses of additional goals alongside pure adversarial robustness and that these factors need to be taken into account. However, a discussion is missing as to whether RobustBench is doing this (or not, and then, why).
- clearly, the AutoAttack package is a cornerstone of RobustBench. I would have liked more details on the package and how its ensemble of attacks was designed. In particular, it seems important how the standardized robust accuracy cited in the leaderboards of RobustBench has been derived from the ensemble of attacks. As it stands, i have the feeling that it is almost necessary to also read the publication of the AutoAttack package to get the full view on RobustBench.


correctness: To the best of my knowledge all the presented work seems to be correct. The study is deeply grounded in existing literature and practices with many relevant references throughout the pages. The benchmark is constructed in a sound way, with much attention to detail, limitations, safeguards (e.g. the flag in AutoAttack and subsequent individual testing) and cross-referencing other methods and contributions from the community, if available.
clarity: The paper is well written, clear and understandable. It seems to me that there are a few smaller issues with image captions:
- line 329 captions in figure 6 seem wrong (three different data sets)
- line 313 captions in figure 4 are maybe wrong (all l_inf?)
relation to prior work:  The paper contains an extensive list of references and is deeply embedded in the ongoing research work on adversarial robustness. There is a specific discussion of the differences and advantages of RobustBench in relation to other adversarial robustness benchmark plattforms (RobustML, Deepsec).
additional feedback: The authors have already published an article of the same title in the ICLR 2021 workshop on Security and Safety in Machine Learning Systems, see here https://aisecure-workshop.github.io/aml-iclr2021/papers/47.pdf

There is significant overlap between the two versions, albeit the current version is substantially fruther developed. I did not consider this in my review (also because i am not fully aware of the details of Neurips policy on this kind of situation) so far and i'd like to give the authors the room to justify  their new submission to this conference, if necessary at all (e.g. it may already have been cleared).
review_title: "RobustBench: a standardized adversarial robustness benchmark" - a strong and carefully designed contribution for advancing research in adversarial robustness

final_score: rating: 7: Good paper, accept
review_content: summary and contributions: This paper provides a standard adversarial benchmark for evaluating adversarial training methods. By collecting 120+ models on different datasets, it builds a leaderboard based on AutoAttack. In addition, it also analyzes the impact of robustness on the performance on distribution shifts, calibration, out-of-distribution detection, etc. 
strengths: 1. It is very important to have a standard adversarial benchmark and leaderboard for the community. 

2. The paper has included most of the important datasets and models for a comprehensive evaluation. 

3. It also provided a deep and interesting analysis of the benchmark. 
weaknesses: 1. The paper missed a few recent works on the adversarial attack and library, for example:
a. RayS: A Ray Searching Method for Hard-label Adversarial Attack
b. DeepRobust: A PyTorch Library for Adversarial Attacks and Defenses
correctness: The paper provided enough details of how the dataset and the model zoo are collected.
clarity: Yes, the paper is well written. 
relation to prior work: The paper included most of the prior works related to adversarial benchmarks. But still, it missed a few important ones. 
additional feedback: My overall evaluation of the paper is good. The author can address the minor comments I made about the related works.
review_title: Official Review of Paper 108 of reviewer YCwj

final_score: rating: 7: Good paper, accept
review_content: summary and contributions: Robustness overestimation is a major problem in the research community of adversarial robustness. The paper proposes RobustBench, a standardized benchmark that addresses this problem. With AutoAttack, an ensemble of white- and black-box attacks, the benchmark improves the evaluation in most of the previous works. A leaderboard is provided with more than 120 models that help to track the progress of the domain. Extensive experiments with RobustBench show various aspects of robust classifiers, including calibration, OOD detection, privacy leakage, etc. Overall, the proposed benchmark can help the community to better understand the adversarial robustness of image classification. 
strengths: * The benchmark provides a leaderboard with more than 120 evaluations that help to track the progress of the domain. 
* A model zoo is built to facilitate the usage of SOTA models. 
* Thorough analyses are provided to see the impact of different aspects on the adversarial robustness. 
* The authors address the concerns of reviewers in Round 1 and add clarification in the paper. 
weaknesses: * The paper is titled as a standardized adversarial robustness benchmark, however, it actually only focuses on the very specific task of image classification. 
* For all figures in Section 4, it is impossible to identify the specific methods, which makes it hard to find the most effective method. 
* Currently, three datasets are provided (CIFAR-10, CIFAR-100, ImageNet). If the ranking differs in different datasets, how to evaluate the adversarial robustness of the method? Furthermore, are there any other contradictory results found using the benchmark (e.g. ranking changes when varying the attack constraints)? In this case, how can one determine a model is more robust than another?
correctness: The benchmark is established in a reasonable way. The evaluation results in experiments seem correct.
clarity: The paper is well written and easy to follow.
relation to prior work: The work is compared with prior works. Works like FoolBox, Cleverhans are toolkits rather than benchmarks. Benchmark like RobustML was designed differently and is not updated for a long time.
additional feedback: My concerns are listed in weakness. I think the benchmark is an interesting work to the community and I hope the authors keep the promise to regularly update it.
review_title: A standardized benchmark on evaluating adversarial robustness in image classification

Meta-review: In this paper the authors provide a benchmark for adversarial robustness, enabling testing with a wide variety of attacks, nuanced discussion of incorporating adaptive attacks, and already evaluations and open-sourcing of numerous models.

The largest concern raised by multiple reviewers was on the limitations of this benchmark. As with most benchmarks, the discussion of the limitations of this leaderboard is crucial so as to not discourage new research outside the scope of focus of this framework. I'd strongly encourage the authors to continue to clarify and qualify the limited scope of this evaluation, e.g. in how the work is titled, on the website, etc.

That said, all reviewers agreed this was a very valuable resource to the community and as such I believe it should be accepted.
Decision: Accept