Venue: NeurIPS 2021 Track Datasets_and_Benchmarks Round1

Reviews:

final_score: rating: 7: Good paper, accept
review_content: summary and contributions: This work introduces a benchmark for adversarial robustness of machine learning methods for image classification tasks. The benchmark consists of a leaderboard which compare user-contributed defenses against $\ell_p$ perturbations with a standard attack method. The main contributions are (1) providing a common view of the state-of-the-art on adversarial robustness against particular attack types through the leaderboard, and (2) providing a library of easily-accessible robust models.

---

After the response

Thank you adding the certified defense and responding on the scope. I would encourage authors to actively consider newer threat models, and secondary analyses metrics to move the field towards newer and more impactful problems.
strengths: Adversarial attacks has emerged as a major challenge to safe deployment of image classification systems. Research efforts on effective defenses against such attacks will benefit from benchmarking of promising approaches with a commonly-accepted evaluation methodology. 
The proposed benchmark is a significant effort at standardizing the evaluation and providing a set of analyses automatically run for all submitted defenses. The proposed set of analyses including the effect of robust learning on calibration, privacy, and robustness to distribution shifts are a great addition to the benchmark which expand the standard evaluation methods and enables studying side-effects of robust learning against adversarial perturbations.
weaknesses: The major weakness is the narrow scope of the benchmark. It is, by construction, meant for image models, classification, $\ell_p$ perturbations, and non-certifiable attacks. At this intersection, there is already a large body of work. Partly, this is the reason to synthesize knowledge on defenses and attacks that work the best, and hence the proposed benchmark. However, a benchmark should be forward-looking and should aim to encourage progress on important yet overlooked parts of the problem. Adversarial robustness (or broadly distributional robustness) is a phenomenon common across input modalities and machine learning tasks. Limiting the scope to image classifications and $\ell_p$ perturbations may not encourage development of methods applicable to this broader scope of the problem.
correctness: The evaluation methodology is sound and documented well. Restrictions on defense methods are clearly described and motivated. The current evaluation is based on a successful attack method, AutoAttack [26]. Further, authors declare that any improvements to the evaluation method will be added. In addition to reporting worst-case accuracy, the benchmarks reports effect of robust learning on important metrics such as calibration and out-of-distribution detection.
clarity: The paper is written well including the description of the components of the benchmark, related work, and experiments.
relation to prior work: Authors discuss improvements on past work. They improve on the design of previous benchmarks by standardizing the type of defenses to be considered, fixing an attack method which will be run automatically on submitted defenses, and proposing a wide set of performance metrics beyond the usual average and worst-case accuracy metrics.
additional feedback: I suggest that the scope of the datasets can be expanded beyond image modality (or provisions can be made to do so) to include text, speech, and tabular modalities; and beyond classification problems to include regression problems. The benchmark should pro-actively encourage threat models beyond $\ell_p$ perturbations such as perceptual distance [73] and others cited in the paper. 

Authors should more concretely outline the plan for continually updating the benchmark to include state-of-the-art defenses and attacks. How do authors plan to support evaluations of models built with libraries other than PyTorch like TensorFlow and Jax? How will the new submissions be solicited from works in-progress which want to report performance on the benchmark and be anonymous for the purpose of blind review?

Comments

I did not understand the concern about including certified defenses in the benchmark. Even though they underestimate robust accuracy, the gap in performance between certified and non-certified defenses in the benchmark can be informative. Certified methods can be labelled differently.

Consider naming the fairness metric differently or clarify the fairness concern when there are performance difference by classes in the applications corresponding to the datasets in the benchmarks.

Analysis can also include impact of robust learning on model explainability.

How does the benchmark disincentivize over-fitting to the current datasets and attack methods used? Can the benchmark include an additional dataset, like CIFAR-10-C or other auto-generated but real datasets, a part of which is shared to the community for training but a part is hidden for evaluation.

Are there alternatives in place to GPU servers provided by the authors’ research group for evaluation? For example, Robust Vision benchmark collaborated with Paperspace.
review_title: A well-designed benchmark

final_score: rating: 8: Top 50% of accepted papers, clear accept
review_content: summary and contributions: The paper proposes a standardized benchmark suite in order to evaluate adversarial robustness of models using AutoAttack. They setup a leaderboard with over 90 models and provide access to models that are more robust in a Model Store for easy access/evaluation. Finally, based on their robustness evaluation they provide insights on how robustness affects different aspects of the model such as the model performance and privacy leakage.

strengths: - A standardized robustness benchmark and leaderboard is addressing an important need in the community. The leaderboard is well evaluated, presented and made available in a model store for easy access.
- RobustBench includes a set of strong and standardized white and black box attacks, and standard&state-of-the-art defenses.
- Analysis of a collection of 60+ models gives interesting insights about how robustness of the models effects performance on distribution shifts, calibration, out-of-distribution detection, fairness and finally privacy leakage. 
weaknesses: .
correctness: Yes.
clarity: It is a well-written, clear paper. The leaderboard website is well-presented.
relation to prior work: Yes, this work differs from the prior work in four different aspects:
(1) A baseline worst-case evaluation with a suite of standardized  white- and black-box attacks that can be optionally extended by adaptive evaluations
(2) Clearly defined threat models that correspond to the ones used during training for submitted defenses
(3) Evaluation of standard and recent state-of-the art defenses
(4) A Model Zoo that provides access to the 60+ most robust models from the literature which can be used for downstream tasks and facilitate the development of new standardized attacks
additional feedback: .
review_title: RobustBench: a standardized adversarial robustness benchmark

final_score: rating: 4: Ok but not good enough - rejection
review_content: summary and contributions: The paper represents a standardized benchmark for evaluating adversarial robustness in the domain of image classification. To evaluate the robustness, it uses AutoAttack, an ensemble of white- and black-box attacks, to attack various models under a fully white-box setting and then report the robust accuracy. It has a leaderboard containing evaluations of 90+ models under the $l_{\infty}$- and $l_2$- threat models and also against common corruptions. It establishes a standard evaluation process with three proposed restrictions and a unified scenario with predefined constraints. Besides, the authors did extensive analysis on the impact of robustness on the performance on various aspects including distribution shifts, calibration, out-of-distribution detection, etc.

**After rebuttal**

Thanks a lot for the authors' rebuttal. I appreciate that you have added a draft of the results on ImageNet, and also update the paper for missing broader impact. However, my major concern is still the proposed standard. As far as I understand, the benchmark is mainly designed based on AutoAttack, the three restrictions of defenses are designed in the way that AutoAttack becomes applicable. As mentioned by the authors, the leaderboards also accept adaptive attacks. However, there are very few adaptive attacks reported. 

The intense usage of AutoAttack and the related restrictions on defenses makes me wonder if this design will lead to unexpected biases.  Imagine that as a standard benchmark, researchers will design robust models that resist AutoAttack, but here comes the question when evaluating their results: is this enough? How can we be sure that there is no adaptive attack that outperforms AutoAttack? As mentioned in the paper, there are already several works that do the evaluation **only** based on AutoAttack. Many works have been done in previous years that try to encourage researchers to evaluate adversarial robustness by using adaptive attacks, I think this is still important. **AutoAttack can serve as a strong baseline, but it should never take the place of adaptive attacks.** I wish the authors can clarify somewhere the usage of AutoAttack and adaptive attacks. 

Overall, I will keep my initial rating due to the above concerns. For future improvement, I suggest the authors include a guideline that specifies that, for example, "in which condition we should also consider adaptive attack rather than AutoAttack?". I also recommend redefining the robust accuracy, which is currently mainly based on AutoAttack. If the adaptive attack is not applied, it should be clarified.  
strengths: The benchmark has a leaderboard containing evaluations of 90+ models and also a model zoo with implementations of 60+ models, which facilitates researchers to contribute in the area. The website for the benchmark is well represented and codes have detailed documentation. Besides, the paper provides some interesting analyses based on these models in various aspects. 
weaknesses: My concerns mainly come from the proposed standard on evaluating adversarial robustness:


1. The proposed restrictions might lead to unexpected biases as a benchmark. It is unreasonable to restrict the form of defenses only because they can’t be attacked by AutoAttack or other gradient-based attacks. Indeed, works in [1] demonstrated some defenses based on gradient obfuscation can be broken by BPDA, but it is questionable if all models with zero gradients can’t increase robustness. Besides, it is always possible to apply gradient-free attacks or gradient-based attacks with BPDA for the evaluation. Same for the other two constraints, they might lead to undesirable bias in the direction of future research. It seems more like a simplification rather than standardization.


2. The usage of AutoAttack under a fully white-box setting is controversial. In this setting, attackers can use any information to apply an adaptive attack, while AutoAttack is not necessarily the most effective one. Again, this might encourage defenses that resist AutoAttack without considering potential attacks adapted to the defense mechanism, which is essential for evaluating the actual robustness [2].


3. Evaluations only with small-scale datasets like CIFAR-10 or CIFAR-100 are insufficient. It is recommended to also consider larger datasets to demonstrate the scalability of both attacks and defenses. One can find dozens of works evaluated on ImageNet [3] in the past years.


4. For the leaderboard, various model architectures are compared under the same setting. It is helpful to find defenses with the highest robust accuracy for a certain dataset, however, it is hard to figure out whether the robustness gain is from the backbone architecture or the defense mechanism. Furthermore, there is no error bar for all results on the leaderboards, knowing that there is a stochastic optimization process for the attack.

[1] Athalye A, Carlini N, Wagner D. Obfuscated gradients give a false sense of security: Circumventing defenses to adversarial examples[C]//International conference on machine learning. PMLR, 2018: 274-283.

[2] Tramer F, Carlini N, Brendel W, et al. On adaptive attacks to adversarial example defenses[J]. arXiv preprint arXiv:2002.08347, 2020.

[3] Dong Y, Fu Q A, Yang X, et al. Benchmarking adversarial robustness on image classification[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2020: 321-331.
correctness: Most of the claims made in the submission are correct. However, as mentioned in the weakness, there are several concerns on the proposed standard on evaluating adversarial robustness. The proposed restrictions and the usage of AutoAttack might lead to unexpected biases as a benchmark.

For experiments, I believe they are performed correctly under the proposed evaluation process. However, the error bar for randomness is missed for all results on the leaderboard. Besides, the experiments are only performed in two small-scale datasets.
clarity: The paper is well written and easy to understand. Just one problem, the broader impact is not discussed.
relation to prior work: The difference between this work from previous works is clearly discussed. The main difference is to use AutoAttack as a standardized attack and to include a number of state-of-the-art methods.
additional feedback: I think it is an important and challenging task to establish a comprehensive benchmark for adversarial robustness on image classification, especially when there are already 3000+ papers in this domain. I appreciate the efforts that the authors made, however, it still needs some refinements to have a convincing solution. Considering that a standardized benchmark might have potential influences on the community, I suggest the authors carefully rethink and improve the proposed standard.
review_title: Need to revisit the proposed standard on evaluating adversarial robustness.

Meta-review: This paper proposes a new benchmark to test the adversarial robustness of machine learning methods on image classification. The proposed benchmark comes with a leaderboard that compares user-contributed defenses against different perturbations. 

The authors have done a good job on addressing the concerns raised by the reviewers during the rebuttal.

However, the reviewer qH7D had some serious concerns about the paper that the authors tried to address in the rebuttal and the reviewer maintained their low score with very high confidence despite the rebuttal. I think the changes required to address the issues raised by the reviewers may require substantial changes in the paper. Those substantial changes may require re-review of the paper. I would recommend the authors to go through all the concerns raised by the reviewers, and revise the paper accordingly to resubmit it to the second round of thee Datasets and Benchmarks track in August.
Decision: Reject