Venue: NeurIPS 2022 Conference

Reviews:

final_score: rating: 5: Borderline accept: Technically solid paper where reasons to accept outweigh reasons to reject, e.g., limited evaluation. Please use sparingly.
review_content: summary: In this work the authors investigate differences in calibration-related properties between standardly trained models and those with robustness interventions. Starting from the well known result that standardly trained models are over confident (in that they show high confidence even on incorrectly classified examples), the authors find that models with adversarial robustness interventions generally are less overconfident and better calibrated in terms of empirical calibration score, a heuristic for measuring calibration. They also find that while one can predict whether or not a model will correctly classify a given example about the same for both categories of models on natural test set examples, models trained with adv. interventions can much better predict whether or not an example that has been adversarially perturbed will be predicted correctly (both in the original threat model, and in an $\ell_0$ threat model). The authors claim these results as the basis for a newly discovered relationship between adversarial robustness and calibration.
strengths_and_weaknesses: The paper presents results that uncover interesting properties of robustly trained models. However, there are some concerns with treatment of prior work / novelty.

Prior work/novelty: there are two prior related papers connecting adversarial robustness and calibration:
- [0]: https://arxiv.org/abs/2006.16375 (NeurIPS 2021): finds that adversarially robust models have much better calibration properties according to the ECE heuristic (Section 3).
- [1]: https://arxiv.org/abs/1910.06259 (ICML 2021): Use calibration to perform adversarial training by reducing confidence on adv examples at train time.

[0] in particular looks like it contain results that could be highly related to the results presented here (i.e. seems to show a result very close to one of the core contributions in this paper); it would be good to clarify this situation.
questions: Questions are in strengths and weaknesses.
limitations: Yes
review_title: null

final_score: rating: 3: Reject: For instance, a paper with technical flaws, weak evaluation, inadequate reproducibility and incompletely addressed ethical considerations.
review_content: summary: The paper studies the calibration abilities of robust models. The paper investigates 71 adversarially trained models and compares these with naturally trained counterparts. The paper observes that most non-robust models are over-confident but robust models are less confident so they are better calibrated. Additionally, the paper observes that specific layers, downsampling, and activation functions can lead to better calibration.
strengths_and_weaknesses: The paper analyzes the calibration abilities thoroughly with many robust models and arrives at the conclusion. However, the observation is not very interesting as it is expected behavior. Additionally, technical contributions are limited. I do not believe the contribution of this paper is enough for top-tiered conferences including ICLR. 
 

questions: The paper can be improved better if the authors can actually improve applications or existing methods based on the observation in this paper.

limitations: The limitation is discussed.


-- Post rebuttal

Thank the authors for their response. I am sorry for the reference, but this definitely does not affect my final score. Even though your claim is correct, I still think that the contribution of this paper is very marginal and not enough for acceptance. And after reading other reviews, it seems like other reviewers also agree on this point. Therefore, I kept my original score.
review_title: null

final_score: rating: 5: Borderline accept: Technically solid paper where reasons to accept outweigh reasons to reject, e.g., limited evaluation. Please use sparingly.
review_content: summary: This paper collected 71 robust models and their counterpart non-robust models, do inference on CIFAR-10, CIFAR-100, and ImageNet. They find that generally robust models have less confidence in both clean data and attacked data. They also find that downsampling strategies and activation functions influence much on prediction confidence.
strengths_and_weaknesses: Strength: 
1. Writing is easy to follow.
2. Enough results to support their argument.

Weakness:
1. Just making statistics on off-the-shelf checkpoints and reporting the results. No novel designs of architectures or training strategies are proposed. 
2. Models are simply classified as 'robust' and 'non-robust'. For different 'non-robust' models, there is less analysis of the adversarial training strategies.
3. The conclusion, that robust models are less overconfident, is not surprising. Robust models are trained on examples that maximize the loss function during adversarial training, so empirically they are less confident about their decisions. 
4. Lack of detailed ablations of architecture influences.

======== Post-rebuttal Update =========
After a more thorough investigation, I do find that this idea is not covered by prior works. This work shows the lower-confidence phenomenon of robust models through large-scale contrast experiments, and also gives applications including predicting erroneous decisions and detecting adversarial samples. Based on these reconsiderations, I would like to raise my score to acceptance.
questions: 1. Obviously, high confidence in correct predictions and low confidence in incorrect predictions is good. But why low confidence in both correct and incorrect predictions is better than high confidence in both correct and incorrect predictions?

2. Are there any straight connections between downsampling strategies and prediction confidence?
limitations: An extensive report, but lacks novelty.
review_title: null

final_score: rating: 6: Weak Accept: Technically solid, moderate-to-high impact paper, with no major concerns with respect to evaluation, resources, reproducibility, ethical considerations.
review_content: summary: This paper focuses on the problem of CNNs being overconfident with their predictions and the effect of adversarial training on this matter. It contains extensive empirical analyses of model confidence scores: 
- Adversarially trained (AT) robust models.
- Model architectures with parametrized activation functions and downsampling layers (as explored in [13]). 

The authors show:
- AT results in more calibrated models. They do so by:
1. Taking existing robust model checkpoints from [15].
2. Train an identical model that's not trained with AT.
3. Creating a validation set of clean and adversarial samples in white-box and black-box settings.
4. They support their claims on CIFAR-10 and -100 and ImageNet datasets. They visualize correct and incorrect class confidence scores, predicted score distribution, and the expected calibration error metric.

- The robust models are less confident on attacked samples. This has been shown by comparing the ROC of robust vs. non-robust models on clean and attacked samples. They also show that the robust model confidences can be used to detect adversarial examples directly.

- They show that improved building blocks result in lower confidence scores on adversarially attacked samples. They visualize the distribution of confidence scores when these modifications are made.


strengths_and_weaknesses: Strengths:
- Highlighting an important property of adversarial training.
- Extensive empirical analysis covering different aspects of their hypothesis.
- Paper writing and organization.

Weaknesses:
- In Figure 4, I would expect to see the ROC of clean models used for the same purpose although potentially it's not great.
- More quantitative metrics could be reported: (a) The ROC AUC for attack detection of different approaches for experiments in Figures 4, 5, and 6, 8 for robust and non-robust models and the different building blocks. ECE for the experiments is interesting to see. I couldn't find these values in the appendix either.
- There could be more experiments focused on the generalization of claims to unseen attacks.
questions: -  In Figures 5 and 6, it seems that the ROC AUC of the second row can be good for attack detection. Can you please report this value? It can be the case that AT and learnable activation functions or FLC pooling can result in similar AUC. This could mean that in case of calibration is not important and we just care about attack detection, one can rely on swapping the building blocks which have lower complexity compared to AT.

- Is this a fair conclusion: learnable activation blocks and FLC generalize better to unseen attacks compared to AT for attack detection How do Figures 4, 5, and 6 look like on unseen attacks?

- typo in Figure 7: rigth -> right.
limitations: The authors have adequately addressed the limitations of their work.
review_title: null

Meta-review: metareview: This paper empirically demonstrates that adversarially trained models are better calibrated than naturally trained counterparts. The reviewer found this paper interesting, and initial concerns are mainly about 1) missing discussions of prior works, and 2) requiring more ablations.

The rebuttal well addresses most concerns (especially regarding the novelty w.r.t. prior works). As a result, three (out of four) reviewers unanimously agree to accept this submission. The reviewer itb7 is the only one against accepting this paper; nonetheless, the original review from itb7 is sort of vague and does not provide useful information for instructing authors for preparing a high-quality rebuttal accordingly. Also as the AC, I cannot see any significant concerns/drawbacks raised in the reviewer itb7's comments, therefore decide to ignore it.

In the final version, the authors should include all the clarifications and the additional empirical results provided in the rebuttal.

confidence: Certain
Decision: Accept