Venue: AAAI 2022 Workshop AdvML

Reviews:

final_score: rating: 7: Good paper, accept
review_content: This paper discusses several limitations of AutoAttack and RobustBench for robustness evaluation. It points out two limitations: 1) The adversarial examples generated by AutoAttack can be easily detected. 2) AutoAttack does not generalize well to datasets with higher resolutions. This paper provides excessive experiments to validate these two findings.

Overall, it is interesting to see such discussions on AutoAttack, which is the most popular benchmark to evaluate adversarial robustness. The findings are insightful and could be useful for future research. The authors are encouraged to further complete their paper to a long version. 
review_title: Review of paper "Is AutoAttack/AutoBench a suitable Benchmark for Adversarial Robustness?"

Meta-review: The reviewer agrees to accept this paper.
Decision: Accept (Short Paper)