Improving Calibration through the Relationship with Adversarial Robustness
Authors: Yao Qin
Xuezhi Wang
Alex Beutel
Ed Chi
Venue: NeurIPS 2021 Poster (2021)
Keywords: Calibration
Adversarial Robustness
Distributional Shift

Abstract: Neural networks lack adversarial robustness, i.e., they are vulnerable to adversarial examples that through small perturbations to inputs cause incorrect predictions. Further, trust is undermined when models give miscalibrated predictions, i.e.,  the predicted probability is not a good indicator of how much we should trust our model. In this paper, we study the connection between adversarial robustness and calibration and find that the inputs for which the model is sensitive to small perturbations (are easily attacked) are more likely to have poorly calibrated predictions. Based on this insight, we examine if calibration can be improved by addressing those adversarially unrobust inputs. To this end, we propose Adversarial Robustness based Adaptive Label Smoothing (AR-AdaLS) that integrates the correlations of adversarial robustness and calibration into training by adaptively softening labels for an example based on how easily it can be attacked by an adversary. We find that our method, taking the adversarial robustness of the in-distribution data into consideration, leads to better calibration over the model even under distributional shifts. In addition, AR-AdaLS can also be applied to an ensemble model to further improve model calibration. 

Improving Calibration through the Relationship with Adversarial Robustness
Yao Qin
Xuezhi Wang
Alex Beutel
Ed H. Chi
Google Research
(yaoqin, xuezhiw, alexbeutel, edchi) com
Abstract
Neural networks lack adversarial robustness, i.e., they are vulnerable to adversarial examples that through small perturbations to inputs cause incorrect predictions. Further, trust is undermined when models give miscalibrated predictions, i.e., the predicted probability is not a good indicator how much we should trust our model. In this paper, study the connection between adversarial robustness and calibration and find that the inputs for which the model is sensitive to small pertur- bations (are easily attacked) are more likely have poorly calibrated predictions. Based on this insight, we examine if calibration can be improved by addressing those adversarially unrobust inputs. To this end, propose Adversarial Robust- ness based Adaptive Label Smoothing (AR-AdaLS) that integrates the correlations of adversarial robustness and calibration into training by adaptively softening labels for an example based on how easily it can be attacked by an adversary. We find that our method, taking the adversarial robustness of the in-distribution data into consideration, leads to better calibration over the model even under distributional shifts. In addition, AR-AdaLS can also be applied an ensemble model to further improve model calibration.
Introduction
The robustness of machine learning algorithms is becoming increasingly important as ML systems being used in higher-stakes applications. In one line of research, neural networks are shown to lack adversarial robustness - small perturbations to the input can successfully fool classifiers into making incorrect predictions (Szegedy et al.) 2014 Goodfellow et al.) 2014 Carlini & Wagner 2017b] Madry et al., 2017; Qin et al. 2020b). In largely separate lines of work, researchers have studied uncertainty in model's predictions. For example, models often miscalibrated where the predicted confidence not indicative of the true likelihood of the model being correct (Guo et al.) 2017 Thulasidasan al. 2019 Lakshminaray ananan et al. [017 Wenn al. 2020 Kull et al. (2019). The calibration issue is exacerbated when models are asked to make predictions on data different from the training distribution (Snoek al. 2019), which becomes an issue practical settings where it is important that we can trust model predictions under distributional shift.
Despite robustness, in all its forms, being popular area of research, the relationship between these perspectives has not been extensively explored previously. In this paper, we study the correlation between adversarial robustness and calibration. We discover that input data points that sensitive to small adversarial perturbations (are easily attacked) are more likely to have poorly calibrated predictions. This holds true number of network architectures for classification and all the datasets that we consider: CIFAR-10 (Krizhevsky) 2009), CIFAR-100 (Krizhevsky) (2009) and ImageNet (Russakovsky et al. 2015). This suggests that the miscalibrated predictions on those adversarially unrobust data points greatly degrades the performance of model calibration. Based on this insight, we hypothesize and study if calibration can be improved by giving different supervision to the model depending on adversarial robustness of each training data.
35th Conference on Neural Information Processing Systems (NcurIPS 2021).

---

CIFAR-10
CIFAR-100
ImaçeNet
1.0
1.0
0.8
0.8
0.8
Score
Scare
Score
0.6
0.6
0.6
Accuracy
Accuracy
Accuracy
0.4
Confidence
0.4
Cor fidercee
0.4
Confidence
2
Adversarial al Robustness Level
Adversarial Robustress Level
Adversarial Robustness Level
CIFAR-10
CIFAR-100
ImageNet
ECE
ECE Varlance
ECE
Vartance
0.2
Variance
Score
Score
0.2
0.2
0.1-
Adversarial Robustness Level
Adversar al Robustness Level
Adversarial Robustness Level
Figure 1: Inputs that are adversarially unrobust more likely have poorly calibrated and unstable predictions on CIFAR-10, 10, CIFAR-100 and ImageNet. Top: Accuracy and confidence of the predicted class. Bottom: ECE (lower is better) and variance (lower is better) in each adversarial robustness subset. Higher adversarial robustness level means the input are more adversarially robust (require larger adversarial perturbations fool the classifier into wrong predictions).
To this end, propose Adversarial Robustness based Adaptive Label Smoothing (AR-AdaLS) to integrate the correlations between adversarial robustness and calibration into training. Specifically, AR-AdaLS adaptively smooths the training labels conditioned how vulnerable adversarial attacks. Our method improves label smoothing (Szegedy et al. 2014) by explicitly teaching the model to differentiate the training data according their adversarial robustness and then adaptively smooth their labels. By giving different supervision to the training data, our method the model without an increase of latency during inference. In
input is to
leads to better calibration addition, since adversarially unrobust data points can be considered as outliers of the underlying data distribution (Carlini et al. 2019), our method can even greatly improve model calibration on held-out shifted data. Further, we propose "AR-Ade ARR-A of Ensemble" to combine our AR-AdaLS and deep ensembles (Laakshminarayanan et al. 2017 Snoek et al., 2019). to further improve the calibration performance under distributional shift. Last, we find an additional benefit of AR-AdaLS improving model stability (i.e., decreasing variance over multiple independent runs), which is valuable in practical applications where changes in predictions across runs (churn) is problematic.
In summary, our main contributions are follows:
We find significant correlation between adversar- robustness and calibration: inputs that are unrobust to adversarial attacks are more likely to have poorly calibrated predictions.
Relationship among Robustness Metrics:
Algorithm: We hypothesize that training model with different supervision based adversarial robustness of each input will make the model better calibrated. To this end, we propose AR-AdaLS automatically learn how much to soften the labels of training data based on their adversarial robustness. Further, we introduce "AR-AdaLS of Ensemble" to show how to apply AR-AdaLS an ensemble model.
On CIFAR- CIFAR-100 and ImageNet, we find that AR-ASaLS is more effective than previous label smoothing methods in improving calibration, particularly for shifted data. Further, we find that while ensembling can be beneficial, applying AR- AdaLs to adaptively calibrate ensembles offers further improvements over calibration.
Experimental Analysis:
2 Related Work
Uncertainty estimates How to better estimate a model's predictive uncertainty an important research topic, since many models with focus accuracy may fall short in predictive uncertainty.

---

A popular way to improve a model's predictive uncertainty is to make the model well-calibrated, e.g.. post-hoc calibration by temperature scaling (Guo et al. 2017), and multi-class Dirichlet calibration (Kull et al., 2019). In addition, Bayesian neural networks, through learning posterior distribution over network parameters, can also be used to quantify a model's predictive uncertainty, e-g., Graves (201T); Blundell et al. (2015): : Welling & Teh (2011). Dropout-based variational inference (Gal & Ghahramani| 2016; Kingma et al. 2015) can help DNN models make less over-confident predictions and be better calibrated. Recently, mixup training (Zhang et al. [201 8) has been shown improve both models' generalization and calibration (Thulasidasan et al. 2019), by preventing the model from being over-confident in its predictions. Despite the success of improving uncertainty estimates over dn-distribbtionn data, Snoek et al. 2019) argue that it does not usually translate to a better performance on data that shift from the training distribution. Among all the methods evaluated by [Snoek et al. (2019) under distributional shift, ensemble of deep neural networks (Lakshminarayanan et al. 2017) shown to be most robust to dataset shift, producing the best uncertainty estimates.
Adversarial robustness On the other hand, machine learning models are known to be brittle (Xin) et al.] (0177) and vulnerable to adversarial examples (Athalye et al., 2018 Carlini & Wagner 2017a b
He et al. 2018; Qin et al. (220a). Many defenses have been proposed to improve model's 's adversarial
robustness (Song et al. [017 Yang al.) Goodfellw al. 20188. however are further attacked
by more advanced defense-awar aware attacks (Carlini & Wagner 2017b; [tt et al.) 2018). Recently,
Carlini et al.] ( Stock & Cissé (2018) define adversarial robustness the minimum distance in
the input domain required to change the model's output prediction by constructing an adversarial attack. The most recent work that is close to ours, Carlini et al. (2019), makes the interesting observation that easily attackable data are often outliers in the underlying data distribution and then use adversarial robustness to determine an improved ordering for curriculum learning. Our work, instead, explores the relationship between adversarial robustness and calibration. In addition, we use adversarial robustness as an indicator adaptively smooth the training labels to improve model calibration.
Label smoothing Label smoothing is originally proposed in Szegedy al. (2016) and is shown to
be effective in improving the quality of uncertainty estimates in Müller et al. (2019); | Thu Thulasidasan
ct al. (201)). Instead of minimizing the cross-entropy loss between the predicted probability P and the one-hot label p, label smoothing minimizes the cross-entropy between the predicted probability and a softened label
p(1 -e)
e) + S. where Z is the number of classes in the dataset and is
hyperparameter which controls the degree of the smoothing effect. Our work makes label smoothing adaptive and incorporates the correlation with adversarial robustness to further improve calibration.
3 Correlations between Adversarial Robustness and Calibration
To explore the relationship between adversarial robustness and calibration, we first introduce the metrics to evaluate each of them (arrows indicate which direction is better).
Adversarial robustness 1 Adversarial robustness measures the minimum distance in the input domain required to change the model's output prediction by constructing adversarial attack (Carlini et al. 2019 Stock& & Cissé, 2018). Specifically, given an input x and a classifier f()) that predicts the class for the input, the adversarial robustness defined as the minimum adversarial perturbation 5 that enables f(x + 5) o f(x).. Following the work (Carlini et al. 2019). we construct the & based CW attack (Carlini & Wagner 2017b) and then use the &2 norm the adversarial perturbation ||ō||2 to measure the distance to the decision boundary. Therefore, a more adversarially robust input requires a larger adversarial perturbation to change the model's prediction.
Expected calibration error + Model calibration measures the alignment between the predicted probability and the accuracy. Well calibrated predictions convey the information about how much we should trust a model's 's prediction. We follow the widely used expected calibration error (ECE) to measure the calibration performance of network (Guo et al. 2017 Snoek al., 2019). To compute the ECE, we need to first divide all the data into K buckets sorted by their predicted probability (confidence) of the predicted class. Let B₁ represent the set of data in the k-th confidence bucket. Then the accuracy and the confidence of By are defined as acc(Bp) = [Bal Σie 1(h. =
y:) and conf = [ ΣieB, 1/01 where ÿ and y represent the predicted class and the true
3

---

Table I: Network architecture and accuracy used for each dataset.
Dataset
CIFAR-10
CIFAR-100
ImageNet
Network Accuracy
ResNet-29
WRN-28-10
ResNet- 101
91.4%
79.2%
77.7%
class respectively, and μ is the predicted probability of g. The ECE then defined as ECE
K
Pul Jaco - conf (Ba)], where N is the number of data points.
1=11
N
3.1
Correlations
Based on the evaluation metrics, we can see that adversarial robustness and calibration are measuring quite different properties: the adversarial robustness measures the property of the data by computing the adversarial perturbation ō from the impul domain, while the calibration metric measures the prop- erties of the model's 's predicted probability the output space. Although adversarial robustness and calibration are conceptually different, they are both connected to the decision boundary. Specifically, adversarial robustness can be used to measure the distance to the decision boundary: if a data point is adversarially unrobust, i.e., easy to find small input perturbation to fool the classifier into wrong classifications, then this data point is close to the decision boundary. Meanwhile, models should have relatively less confident predictions on data points close to the decision boundary. However, as pointed out by (Guo et al., (217)) Snock et al. 2019), existing deep neural networks are frequently confidnni,,n,, L.e., having predictions with high confidence even whey they should be uncertain. Taking these two together, we hypothesize if examples that can be easily attacked by adversarial examples are also poorly calibrated.
To test this, we perform experiments on the clean test set across three datasets: CIFAR-10 (Krizhevsky 2009), CIFAR-100 (Krizhevsky) (209) and ImageNet (Russakovsky et al. 2015) with different networks, whose architecture and accuracy are shown Table[]] We refer to these models as "Vanilla" for each dataset in the following discussion. The details for training each vanilla network are included in Appendix A.
To explore the relationship between adversarial robustness and calibration, we start with the relation- ship between adversarial robustness and confidence together with accuracy. Specifically, we rank the input data according to their adversarial robustness and then divide the dataset into R equally-sized subsets (R = 10 used in this paper). For each adversarial robustness subset, we compute the accuracy and the average confidence score of the predicted class. As shown in the first row in Figure I we can clearly see that both accuracy and confidence increase with the adversarial robustness of the input data, and confidence is consistently higher than accuracy in each adversarial robustness subset across three datasets. This indicates that although vanilla classification models achieve the state-of-the- accuracy, they tend to give over-confident predictions, especially for those adversarially unrobust data points.
Taking one step further, we particularly compute the expected calibration error (ECE) in each adversarial robustness subset, shown in the bottom row of Figure In general, we find that data points falling into lower adversarial robustness levels are more likely to be over-confident and less well calibrated (larger ECE). For those adversarially robust examples, there a better alignment between the model's predicted confidence and accuracy, and the ECE over those examples is close to This nicely validates our hypothesis: inputs that are adversarially unrobust are more likely to have poorly calibrated predictions. On larger-scale ImageNet, while we still see the general trend holds, the least adversarially robust examples are relatively well calibrated. We hypothesize this may due to larger training data and less overfitting.
Furthermore, we also find an interesting correlation between adversarial robustness and model stability, which is measured by the variance of the predicted probability across M independent runs (c.g. M 5). The variance is computed as 0² ~ ΣM m= ΣN (Pm.) p.) where
Pm. is the m-th model's predicted probability of the i-th data and p. 1 ΣM average predicted probability over M runs. As shown in the bottom row of Figure I we see that those adversarially unrobust examples tend to have much higher variance across all three datasets. This indicates that inputs that are unrobust to adversarial attacks are more likely to have unstable predictions.
M m Pm, is the

---

Algorithm Training procedure for AR-AdaLS
number of classes Z, number of training epochs T, number of adversarial robustness subset R. learning rate of adaptive label smoothing a. For each adversarial robustness training subset, we initialize the soft label as the one-hot label Pr.. = Pr. where the initial soft label for the correct class Pr, =1 for 1 1 to 7 do
Input:
Minimize cross-entropy loss between soft label and predicted probability #2 Σr ((rre.Pr.t))
for 1 to do -y Update pr,++1 + Pr.t
{conf(Srra) aco(Sp
>> Eqn. 3
Clip Pr,++1 to be within ( (
Update 6r,+1 + (Pr,0+1 - 1) T-7
> Eqn.
Update Pr,141 + pr(1 - Er,+1) + 51111
> Eqn.
end for end for
Taking all together, these empirical results nicely build connection between very different concepts. In particular, adversarial robustness is measured over the input domain while both calibration and stability are measured over the output space. Given the strong empirical connection, we now ask: can improve model calibration and stability by targeting adversarially unrobust examples?
4 Method
Based on the correlation between adversarial robustness and calibration, we hypothesize and study calibration can be improved by giving different supervision to the model depending the adversarial robustness of training data. To this end, we propose a method named Adversarial Robustness based Adaptive Label Smoothing (AR-AdaLS) which performs label smoothing at different degrees to the training data based on their adversarial robustness. Specifically, we sort and divide the training data into R small subsets with equal size according their adversarial robustness and then use 6, to soften the labels each training subset Strain The soft labels can be formulated as:
pr (1) Z
(I)
where pr stands for the one-hot vector, e.g., PF- for the correct class and pマ for the others, and Z is the number of classes in the dataset. The parameter e, controls the degree of smoothing effect and allows for different levels of smoothing in each adversarial robustness subset. Generally, a relatively larger e,- is desirable for lower adversarial robustness levels such that the model learns to make lower confident prediction. Instead of empirically setting the parameter e, in each adversarial robustness subset, we allow it to be adaptively updated according to the calibration performance the validation set (discussed in Section 4.1) In this way, explicitly train network with different supervision based on the adversarial robustness of training data.
There are two options to obtain the adversarial robustness. One is "on the fly": to keep creating the adversarial attacks during training, which provides precise adversarial robustness ranking but the cost of great computing time. The other is to "pre-compute" the adversarial robustness by attacking a vanilla model with the same network architecture but trained with one-hot labels. This more efficient but at the sacrifice of the precision of adversarial robustness ranking. In practice, we find that is sufficient to make the network differentiate the adversarially robust and unrobust data with the pre-computed adversarial robustness (see more discussion in Section 5.6). Therefore, all experiments related to "AR- AdaLS"" without further specification are based on pre-computed adversarial robustness for efficiency.
4.1 Adaptive learning mechanism
To find the best hyperparameter e for label smoothing, previous methods (Szegedy et al. 2016 Thulasidasan et al. 2019) sweep e in range and choose the one that has the best validation
Note, predicted confidence is not good indicator for splitting the training dataset as the model can easily overfit to the training data and their predicted confidence all close 100%.
5

---

performance. However, in our setting, the number of combinations of c, increases exponentially with the number of adversarial robustmess subsets R. To this end, we propose an adaptive learning mechanism to automatically learn the parameter in each adversarial robustness subset. The overall training procedure is summarized in Algorithm[] [
First, we denote the soft label for the correct class in the r-th adversarial robustness subset as p.- According to Eqn. (.. we can derive:
- G 7
Since well-calibrated predicted probability should be aligned with the empirical accuracy, we use the calibration performance in the validation set to help update p;= for the training data. Specifically, we first rank the adversarial robustness of the validation data and split the validation set into R equally-sized subsets. Then, adversarial robustness validation subset co - acc(S to update the soft label for the correct class of training data in the r-th adversarial robustness training subset Strain
use the difference between confidence and accuracy in the r-th
= --yy Pr,--1
{conf(S)" acc(St"))))
P,,
where pr,t" is the soft label of the correct class in the r-th adversarial robustness training subset at time step t. The accuracy and the confidence of Sqaai are defined as acc(Spal) S Eiesyel 1 y)) and (S:"l) = | Eiespal P. p:= where ÿ and y is the predicted class and the true class
respectively, fa=9 denotes the the predicted probability of the predicted class. The hyperparameter > 0 plays role learning rate to update the soft label z-= based on the difference between the predicted confidence and accuracy in the validation set. Intuitively, if we assign a large p; to training data, then the network tends make a high confident prediction and vice versa. Therefore, if the confidence is greater than the accuracy (S > acc(S?"/) in the validation set, we should reduce p7- -y y to teach the network be less confident. Otherwise, we should increase V In addition, we also need to constrain 77 to be within ( Z.. 1] after each update as it stands for the true probability of the correct class, where Z is the number of classes in the dataset.
For a given pp-M, we can easily obtain 6, by reversing Eqn. 2 Z
Z
and the soft labels for all the classes pr can be computed according Eqn. We update the soft labels after each training epoch in our experiments.
Note that this adaptive learning mechanism can be easily applied to standard label smoothing without adversarial robustness slicing (R = 1). In this case, we can replace sweeping the hyperparameter e with this adaptive learning method, named as "Adaptive Label Smoothing" (AdaLS). Our proposed AdaLS and AR AR-AdaLS do not increase the inference time: we test AdaLS and AR- AdaLS exactly the same vanilla model.
5 Experiments
Datasets We test our method on three datasets CIFAR-1O, CIFAR-100 and ImageNet. In addition, we also report performance on the shifted datasets: CI-AR-I0-C, CIFAR-100-C and ImageNet- C (Hendrycks Dietterich 2019) where there are different types of corruptions (19 types for CIFAR-IO, 17 types for CIFAR-IO0 and 15 types for ImageNet), e.g., noise, blur, weather and digital categories that are frequently encountered in natural images. Each type of corruption has five levels of shift intensity, with higher levels having more corruption.
5.1 How does AR-AdaLS work?
To have deeper understanding of how AR-AdaLS works, in Figure 2 we visualize the effect of label smoothing (LS) and our AR-AdaLS. Comparing Figure and (b), AR-AdaLS is better at calibrating the data than label smoothing, especially the adversarially unrobust examples (lower adversarial robustness level). Further, we show plots of ECE and variance in Figure (c) and (d). Both label smoothing and AR-AdaLs improve model calibration and stability over vanilla model and AR- AR-AS has the best performance among three methods. This suggests that AR-AdLLS is better at improving calibration and stability in adversarially unrobust regions, not just average.

---

(a) L5
(b) Af-AdaLS
[c] ECE
(d) Varience
0.1
0.3
Vanilia
Vanillo
Store
Score
EGE
0.2
AdaLS
Serfnn 01
02
AR.AdaLS
0.6
Accuracy
Accurecy
Confidecc
0.6
Confidence
2
15
10
0.0
10
0.0
Adversarial Robustness Leve
Adversar al Robustness Level
Adverserial Robustness Level
Adversaria iali Robustress Level
Figure 2: Comparison between LS and our AR-AdaLA on the clean test set of CIFAR-10 10. (a) and (b): Accuracy and confidence score of the predicted class in each adversarial robustness subset. (c) and (d): ECE and variance score of Vanilla, LS and AR- Ad-LS..
Table 2: ECE (x 10 3) on CIFAR- 10 and CIFAR-100. AR-A improves calibration and is only rivaled by domain-knowledge based data augmentation or larger ensemble models on CIFAR- 100. Adversarial robustness is generated on-the-fly ffy for AR AR-AdaLS, All the single-model based results are generated over four independent runs with random initialization.
Method
CIFAR-10
CIFAR-100
Method
CIFAR-10 10
CIFAR-I00
Single-model based
Data-augmennttionn based
Vanilla
2.49:10.10
6.11
mixup CCAT
0.78
1.69:0,088
Temperature Scaling
0.80++0.05
4.26+0007
2.37++0.07
0.95++0.35
Label Smoothing
1.07++0.09
2.76+00.26
Ensemble based
AdaLS
1.23:+00.02
2.65++0.31
Mix-n-Match Ensemble of Vanilla
0.97
2.80
AR-AdaLS
0.64.10.02
2.27110.16
0.90
2.21
5.2 AR-AdaLS improves calibration
Baselines We compare our proposed AR-AdaLS with the following 8 different methods: (1) Vanilla model trained with one-hot labels. (2) Temperature Scaling (Guo et al., 2017). post-hoc calibration method where the predicted logits are divided by temperature which is tuned on the hold-out validation set, (3) label smoothing (LS) (Szegedy al.) 2016) that softs labels by sweeping the hyperparameter (the smoothing degree) in range to find the best hyperparameter e, (4) Adaptive Label Smoothing (AdaLS): we use our proposed adaptive learning mechanism introduced in Sec- tion 4. to automatically learn the hyperparameter e rather than sweeping to find the best (5) mixup, which data augmentation technique originally proposed in (Zhang et al., 2018) and recently found to be able to improve calibration in (Thulasidasan et al.) 2019), (6) Contidence-caliibaate adversarial training (CCAT) (Stutz et al., 2020). a method builds on adversarial training by reducing the confidence in the labels of adversarial examples. Note that there is significant difference between our AR-AdaLS and CCAT: CCAT trains a model on the generated adversarial examples to improve model's adversarial robustness. In contrast, our AR-AdaLS, trained on the clean training data, is proposed to use the correlation between adversarial robustness and calibration to improve a model calibration performance. (7) "Ensemble of Vanilla" (Lakshminarayanal et al.) 2017), ensemble of M vanilla models independently trained with random initialization. (8) Mix-n-Match (Zhang eet al. 2020), an ensemble and compositional method proposed for calibration. All the methods are trained with the same network architecture, i.c., WRN-28- (Zagoruyko & Komodakis 2016) on both CIFAR-10 and CIFAR- 100, and the same training hyperparameters: e.g., learning rate, batch size, number of training epochs, for fair comparisor Please refer to Appendix A for all the training details and hyperparameters.
The expected calibration error of all the methods CIFAR-10 and CIFAR-100 displayed in [a We can clearly see that by differentiating the training data based on their adversarial robustness, AR-AdaLS effectively reduces the calibration error compared to other single- model based methods without significant change in accuracy (see Figure Appendix) and it is only rivaled by mixup on CIFAR- 100, which uses extra domain knowledge through data augmentation. Note that AR- Ad-AA is only trained on the clean training data without any data augmentation compared to mixup Thulasidasan et al. 2019) and CCAT (Stutz et al. (020).
Results
"The result of Mix-n Match in Table is from Table I reported in the original work Zhhng et al. 2020 which is trained with the same network architecture, WRN-28-10.
7

---

Table 3: Mean of ECE (x10-3) across 19 types of shift for CIFAR-10-C and 15 types of shift for ImageNet-C. Smaller better. ResNet-29 used for CIFAR-I0 and ResNet-101 used for ImageNet. The standard deviation of five independent runs for each single model is reported. The best single model and ensemble model in each shift intensity highlighted in bold.
Single-model based
Ensemble-based
Methods
CIFAR-10-C
ImageNet-C
Methods
CIFAR-10-C
ImageNet-C
Vanilla 16.7:00.5
10.7=0.5
Ensemble of Vanilla
6,5
4.2
LS AdaLS
10.1=0.4
8.1=0.4
Ensemble of LS Ensemble of AdaLS
4.6
4.7
9.6+00.5
8.0+0.2
5.2
4.8
AR- AR-AdaLS
6.410..6
6.910..2
Ensemble of AR-AdaLS 5.5
5.1
AR-AdaLS of Ensemble
4.4
4.0
5.3 Improve calibration on shifted dataset
Table summarizes the mean calibration error (ECE) on the corrupted datasets: CIFAR-10-C and ImageNet-C (Hendrycks & Dietterich 2019). Looking at all the single-model based methods, see that AR-AS significantly outperforms other single-model based methods with the lowest ECE. Contrasting with LS and AdaLS, we see AR-AdaLS benefits greatly from the adversarial robustness slicing. As result, our model learns give smaller soft labels of the correct class those adversarially unrobust training data, which can also be considered as outliers of the underlying data distribution (Carlini et al. (0T9). Therefore, when tested the shifted data that deep networks have been shown to produce pathologically over-confident predictions (Hendrycks Dietterich (209)). our model correctly learns make a relatively lower-confidence prediction, resulting in better calibration performance.
In addition, also compare AR- AdaLS with "Ensemble of Vanilla" [akksmmnaaraaaaan et al., 2017 which is shown to be the best model for models' calibration under distributional shift (Snoek et al. 2019). The result of Ensemble of Vanilla is an ensemble of M 5 vanilla models independently trained with random initialization. We can see that AR-AdaLS achieves comparable calibration performance on CIFAR-10 and the ensemble is better under highly shifted data ImageNet.
Combination with deep ensembles We further discuss the following two ways to combine AR-AdaLS with ensembles:
Ensemble of AR-AdaLS: As in Lakshminarayanan et al. 2017). 7) ensemble AR-AdaLS by training multiple independent AR Ad-AS models with random initialization, and average their predictions at inference.
AR-AdaLS of Ensemble: Instead of computing soft labels independently for each AR AR-AdaLS, we perform AR-AdaLS on the ensembled predictions, i..., in Eqn (3) we compute confidence and accuracy based on the average of M model predictions. Each model is then supervised with the same soft labels. We will see this slight distinction in training is quite important.
As shown in Table naively combining deep ensembles with AR-AdaLS (Ensemble of AR- AR-AdaLS) could not effectively improve models' calibration (see more details in Appendix B In contrast, AR-AdaLS of Ensemble, which adaptively adjusts smoothing to keep the ensemble models well calibrated, performs the best under distributional shift on both CIFAR-10 and ImageNet.
5.4 Improve model stability
Since we observe in Figure that the most adversarially unrobust data points also have very unstable predictions, we test AR-AdaLS to see if it can help improve model stability, which is of great value in practice where high variance of a model bad for churn (Milani Fard et al.) 2016). can see that AR-AdaLS can effectively reduce the variance of a model compared to a vanilla model and label smoothing on CIFAR-10 and ImageNet. Please refer to Table in Appendix for numerical numbers on both datasets.
Figure we
8

---

0.30
Vanilla
AdoLS
CIFAR-10
Veriance
0.20
0.15
0.10
0.05
Test
Shift intensity
0.12
Vanilla
AdaLS
ImageNet
0.10
AlsAdaL.
Variance
0.08
0.04
Shift intensity
Figure 3: Variance on clean test and shifted data on CIFAR-I0 10 and ImageNet. For each shift intensity, show the results with box plot summarizing the 25th, 50th, 75th quartiles across 19 shift types on CIFARIO- I0-C and 15 shift types on ImageNet-C. The error bars indicate the min and max value across different shift types. ResNet-29 is used for CIFAR-10 and ResNet-101 is used for ImageNet.
CIFAR-DO(InCCFFAR-1000/00
CIFAR-COInNCIFAR--0000U0
CFFAR-10LIm/CIFARR-000CGU
Venille
AR-AdeLS
2000
Court
2000
Court
2000
1000
1000
1000
0.0
1.0
1.5
0
0.0
0.5
2.0
0
1.0
1.5
2.0
Entropy
Entropy
Entropy
Figure 4: Histogram of predictive entropy on out-of-distribution data. Each model is trained on CIFAR-10 and tested CIFAR- The network architecture is WRN-28-10 and adversarial robustness in AR- AdaLS generated on-the-fly.
5.5 Improvements on out-of-distributio data
We further study the performance of AR- Ad-AdaLS when predicting out-of-distribution (OOD) data. Following (Snock et al. 2019). we compare the performance Vanilla, Label Smoothing and AR- AR-AdaLS by plotting the histogram of the entropy the OOD data (higher entropy OOD better). As shown in Figure each model is trained on CIFAR-10 dataset and then tested on CIFAR- 100 dataset. We can clearly that AR-AdaLS significantly reduces the number of low- entropy predictions on OOD data. In addition, using of-distribution data, also report the Area under the ROC curve (AUROC) of label smoothing, mixup and AR-AdaLS. The AUROC score of standard label smoothing and mixup is 0.832+0.008 and 0.821 +0.003 respectively, whereas our AR- Ad-ALLS achieves 0.885+000033. This demonstrates the effectiveness of AR-AdaLS even on fully out-of-distribution data.
1AA.AR--O/IFRA--AA as in-distributionfouu.
5.6 Sensitivity analysis
Sensitivity to the number of adversarial robustness subsets We perform a sensitivity analysis for the number of adversarial robustness subsets R. Specifically, we plot the calibration error of AR-AS with varying R on the clean CIFAR-10 10 and corrupted CIFAR-10-C in Figure We can see that there a significant drop in calibration error (ECE) when we increase the number of adversarial robustness subsets R from I, where R denotes AdaLS, Further, the calibration ervor relatively stable when R is chosen within the range [10, 16). Thus, we choose 10 for all results shown in this paper for AR- AdaLS.

---

CIFAR-10 Callbration Error
CIFAR-10-C Calibration Error
0.16
0.015
AR-AdaLS
AR-AdaLS
0.14
0.009
0.012
0.003
12
10 12 14 16 Adversarial Robustress Subset
4
6
0.08
2
8 10
14
Adversarial Robustness Subset
Figure 5: ECE CIFAR-10 and CIFAR-10-C of AR-AdaLS with varying number of adversarial robustness subset R. Note that when R = L, AR-AdaLS becomes AdaLS. The results are based on 4 independent runs.
Table 4: Ablation study of AR-AdaLS on CIFAR-100 and LIFAR10-CC (corrupted), We report both accuracy (x10-²) and expected calibration enror (x10-3 3), denoted by Ace and ECE for the clean test set, and cAce and cECE for CIFAR100-C. Arrow indicates the better direction; best calibration is bolded.
Label Smoothing
Temperature Scaling
AR-AdaIA
AR-ASaLS (on-the-fly)
Method
Vanilla
(pre-compute)
AccleAce (f) ECE/CECE (1)
79.2/52.0
78.9/51.7 2.8/16.3
79.2/52.0 4.3/14.0
79.3/52.2 2.6/14.2
79.2/52.1
6.1/18.2
2.3/13.2
Sensitivity to the exactness of adversarial robustness To investigate this, we study the perfor- mance of AR-AdaLS using adversarial robustness generated via two different ways: One is "on-the- fly": we keep creating adversarial attacks during training, which provides a more precise adversarial robustness ranking but at the cost of great computing time. The other is to "pre-compute" adversarial robustness by attacking a vanilla model that is trained with one-hot labels. This is more efficient but at the sacrifice of the precision of adversarial robustness ranking. We perform experiments on CIFAR-100 as an example to compare the performance of AR-AaLSS based on the adversarial robustness that "pre-computed" or "on-the-fly", As shown in Table generating adversarial robustness "on-the-fly" can further help improve the calibration performance for AR- AdaLS on both clean and shifted datasets, compared to computinn adversarial robustness. Similar patterns are observed on CIFAR-101
Therefore,
conclude that 1) the exactness of adversarial robustness is helpful for AR-AdaLS, that is, more precise adversarial robustness leads to a better performance. 2) AR-AdaLS with an approximation of adversarial robustness (pre-computed) can already significantly improve label smoothing. Hence, all results in this paper related to "AR-AdaLS" without further specification are based on pre-computed adversarial robustness for efficiency. This is because our main target is to show that the idea of differentiating the training data based on their adversarial robustness is promising improve model calibration rather than pushing the results the best.
Conclusion
In this paper, we have explored the correlations between adversarial robustness and calibration. We find across three datasets that adversarially unrobust data points, where small adversarial perturbations likely have poorly
the input are able fool the classifier into wrong predictions, are calibrated and unstable predictions. Based on this insight, we propose AR-AdaLS to adaptively smooth the labels of the training data based their adversarial robustness. In our experiments see that AR- AdaLS is more effective than previous label smoothing methods in improving calibration, particularly for shifted data, and can offer improvements on top of already strong ensembling methods. We believe this is an exciting new use for adversarial robustness as a means to more generally improve model trustworthiness, not just by limiting adversarial attacks but also improving calibration and stability on unexpected data. We hope this spurs further work at the intersection of these areas of research.
We did not run on-th ot-the-fly fly AR-AdaLS for ImageNet due to the computational intensity.
10

---

References
Athalye, A., Carlini, N., and Wagner, D. Obfuscated gradients give a false sense of security: Cireumventing defenses to adversarial examples. In International Conference Machine Learning. 2018.
Blundell, C., Cornebise, J., Kavukcuoglu, K., and Wierstra, D. Weight uncertainty in neural network. volume 37 of Proceedings of Machine Learning Research, pp. 1613-1622, Lille, France, 07-09 Jul 2015. PMLR.
Carlini, N. and Wagner, D. Adversarial examples are not easily detected: Bypassing detection methods. In Proceedings of the 10th ACM Workshop on Artificial Intelligence and Security, AlSec '17, pp. 3-14. Association for Computing Machinery, 2017a. ISBN 9781 9781450352024.
Carlini, N. and Wagner, D. Towards evaluating the robustness of neural networks. 2017 IEEE Symposium on Security and Privacy (SP), pp. 39-57. IEEE, 2017b.
Carlini, N., Erlingsson, Ú. and Papernot, N. Distribution density, tails, and outliers in machine learning: Metries and applications. ArXiv, abs/1910.11422) 2019.
Gal, Y. and Ghahramani, Z. Dropout as a bayesian approximation:
Representing model uncertainty in deep learning. In International Conference on Machine Learning. pp. 1050-1059, 2016.
Goodfellow, I., Shlens, J., and Szegedy, C. Explaining and harnessing adversarial examples. In International Conference on Learning Representations, 2014.
Goodfellow, I., Qin, Y., and Berthelot, D. Evaluation methodology for attacks against confidence thresholding models. 2018
Graves, A. Practical variational inference for neural networks. Shawe-Taylor, J., Zemel, R. S., Bartlett, P. L., Pereira, F,, and Weinberger, K. Q. (eds.), Advances in Neural Information Processing Systems 24, pp. 2348-2356. Curran Associates, Inc., 2011.
Guo, C., Pleiss, G., Sun, Y., and Weinberger, K. Q. On calibration of modern neural networks. In International Conference on Machine Learning, 2017.
Hc, K., Zhang, X., Ren, S., and Sun, J. Deep residual learning for image recognition. In Proceedings of the IEEE conference on Computer Vision and Pattern Recognition, pp. 770-778, 2016a.
He, K., Zhang, X., Ren, S., and Sun, J. Identity mappings in deep residual networks. In European Conference on Computer Vision, pp. 630-645. Springer, 2016b.
He, W., Li, B., and Song. D. Decision boundary analysis of adversarial examples. In International Conference on Learning Representations, 2018
Hendrycks, D. and Dietterich, T. G. Benchmarking neural network robustness to common corruptions and perturbations. International Conference on Learning Representations, 2019.
Kingma, D. P., Salimans, T., and Welling, M. Variational dropout and the local reparameterization trick. In Advances in Neural Information Processing Systems, pp. 2575-2583, 2015.
Krizhevsky, A. Learning multiple layers of features from tiny images. Technical report, University of Toronto, 2009,
Kull, M., Perello Nieto, M., Kängsepp. M., Silva Filho, T., Song. H., and Flach, P. Beyond temperature scaling: Obtaining well-calibrated multi-class probabilities with dirichlet calibration. In Advances in Neural Information Processing Systems 32, pp. 12316-12326, 2019.
Lakshminarayanan, B., Pritzel, A., and Blundell, C. Simple and scalable predictive uncertainty estimation using deep ensembles. In Advances in Neural Information Processing Systems, pp. 6405-6416. Curran Associates Inc., 2017. ISBN 978151086096.
Madry, A., Makelov, A., Schmidt, L., Tsipras, D., and Vladu, A. Towards deep learning models resistant to adversarial attacks. In International Conference Learning Representations, 2017.
11

---

Milani Fard, M., Cormier, Q., Canini. K., and Gupta, M. Launch and iterate: Reducing prediction churn. In Lee, D. D., Sugiyama, M.. Luxburg, U. V., Guyon, I., and Garnett, R. (eds.), Advances in Neural Information Processing Systems 29, pp. 3179-3187. Curran Associates, Inc., 2016.
Müller, R., Kornblith, S., and Hinton, G. E. When does label smoothing help? Advances in Neural Information Processing Systems, pp. 4694-4703, 2019.
Qin, Y., Frosst, N., Raffel, C., Cottrell, G., and Hinton, G. Deflecting adversarial attacks. arXiv preprint arXiv:2002.074400 2020a.
Qin, Y., Frosst, N., Sabour, S., Raffel, C., Cottrell, G., and Hinton, G. Detecting and diagnosing adversarial images with class conditionl capsule reconstructions. International Conference Learning Representations, 2020b.
Russakovsky, O., Deng, J., Su, H.,, Krause, J., Satheesh, S., Ma, S., Huang, Z.,, Karpathy, A., Khosla, A., Bernstein, M. S., Berg, A. C., and Li, F.-F. Imagenet large scale visual recognition challenge. International Journal Computer Vision, 115:21 1-252, 2015.
Snoek, J., Ovadia, Y., Fertig, E., Lakshminarayanan, B., Nowozin, S., Sculley, D., Dillon, J., Ren, J., and Nado, Z. Can you trust your model's uncertainty? evaluating predictive uncertainty under dataset shift. In Advances in Neural Information Processing Systems, pp. 13969-13980, 2019.
Song, Y., Kim, T., Nowozin, S., Ermon, S., and Kushman, N. Pixeldefend: Leveraging generative models to understand and defend against adversarial examples. In International Conference on Learning Representations, 2017.
Stock, P. and Cissé, M. Convnets and imagenet beyond accuracy: Understanding mistakes and uncovering biases. In European Conference on Computer Vision, 2018.
Stutz. D., Hein, M., and Schiele, B. Confidence-calibrate adversarial training: Generalizing to unseen attacks. International Conference on Machine Learning, pp. 9155-9166. PMLR, 2020.
Szegedy, C., 7aremba, W., Sutskever, I., Bruna, J., Erhan, D., Goodfellow, I. J., and Fergus, R. Intriguing properties of neural networks. In International Conference on Learning Representations, 2014.
Szegedy, C., Vanhoucke, V., Ioffe, S., Shlens, J., and Wojna, Z. Rethinking the inception architecture for computer vision. 2016 IEEE Conference on Computer Vision and Pattern Recognition, pp. 2818-2821 2016.
Thulasidasan, S., Chennupati, G., Bilmes, J. A., Bhattacharya, T., and Michalak, S. On mixup training: Improved calibration and predictive uncertainty for deep neural networks. In Advances in 2019.
Neural Information Processing Systems, pp.
Welling, M. and Teh, Y. W. Bayesian learning via stochastic gradient langevin dynamics. In Pro- ceedings of the 28th International Conference on International Conference on Machine Learning, ICML' 11, 11, pp. 681 668- Omnipress, 2011. ISBN 9781450306195.
Wen, Y.. Tran, D., and Ba, J. Batchensemble: An alternative approach efficient ensemble and lifelong learning. In International Conference on Learning Representations, 2020.
Xin, D., Mayoraz, N., Pham, H., Lakshmanan, K., and Anderson, J. R. Folding: Why good models sometimes make spurious recommendations, In Proceedings of the Eleventh ACM Conference on Recommender Systems, pp. 201-209, 2017.
Yang, Y., Zhang, G., Katabi, D., and Xu, 7.. Me-net: Towards effective adversarial robustness with matrix estimation. International Conference on Machine Learning. 2019.
Zagoruyko, S. and Komodakis, N. Wide residual networks. In British Machine Vision Association, 2016.
Zhang, H., Cissé, M., Dauphin, Y., and Lopez-Paz, D. Mixup: Beyond empirical risk minimization. In International Conference on Learning Representation, 2018
Zhang. J., Kailkhura, B., and Han, T. Y.-J. Mix-n-match: Ensemble and compositional methods for uncertainty calibration in deep learning. In International Conference on Machine Learning, pp. 11117-11128. PMLR, 2020.
12


Reviews:
[{"review_id": "qd3DGAncOLO", "reviewer": "Reviewer_ceCu", "summary": "1) This paper points out a novel finding: Examples that can be adversarially attacked may also be calibrated worse.\n2) The author utilize the above finding to design the AR-AdaLS to enhance calibration. ", "questions": "", "limitations": "", "rating": 7, "confidence": 5, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "Generally speaking, I am satisfied with the topic and the method proposed in this work. In these two years, adversarial robustness has been found to be related to many other concurrent fields: Outlier Detection[1], Domain Adaption[2], Data Augmentation[3] etc. According to my knowledge, this is the first work that points out the relationship between adversarial robustness and calibration. Therefore, I think this can complement the current study about the concept of the general robustness. The proposed method is reasonable and consistent with the finding. The experimental setup aligns with common choices in other works. In particular, I consider this work as a good extension for [4]. However, I still have a few concerns:\n1) Although I do think the observation that \"Examples that can be adversarially attacked may also be calibrated worse\" is reasonable, I find that in this paper the authors did not provide sufficient information to properly explain this phenomena. Section 3 are basically composed of empirical findings and lack of analyses. I hope the authors can offer more insights in the rebuttal phase and perhaps provide evidences from other works, or maybe explain the phenomena based on the framework of [4].\n2) In Table 3, the author combine the proposed AR-AdaLS with ensemble based method. Can AR-AdaLS be combined with the data augmentation based methods like MixUp as in Table 1?\n3) Although the writing is rather clear and straightforward, there are many typos or mistakes. For instance, the appendix is submitted with the main content. Many section titles are not properly capitalized. Of course, these minor issues do not influence my general impressions on this paper. But I suggest the author to be more careful with these mistakes in the future.\n\n\n\n\n\n\n\n\n[1] ENHANCING THE RELIABILITY OF OUT-OF-DISTRIBUTION IMAGE DETECTION IN NEURAL NETWORKS\n\n[2] Adversarial Examples Improve Image Recognition\n\n[3] Adversarial Mutual Information for Text Generation\n\n[4] When does label smoothing help?", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "None", "review_text": "", "extra_scores": {"needs_ethics_review": "No", "ethics_review_area": "", "time_spent_reviewing": "8"}, "extra_text": {}}, {"review_id": "l0o1pW1RRRQ", "reviewer": "Reviewer_fyY9", "summary": "This paper investigates the relationship between calibration and adversarial robustness, showing that calibration error is larger among less robust images. Based on this, a training procedure is proposed where the labels of the images are smoothed based on the robustness level, to produce a better calibrated model. \n\nThe proposed method is evaluated on CIFAR-10/CIFAR-100, and their corrupted counterparts, comparing with other calibration and label smoothing approaches, as well as ensemble methods.", "questions": "", "limitations": "", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "This paper studies an interesting idea: how the less robust images are also responsible for higher calibration error. The idea of ranking the images based on their robustness and exploit the less-robust images to improve calibration is worth investigating. \nHowever there are many issues with the paper in its current version, as listed next:\n\n1. Until the last page of the paper, it was not clear if the robustness levels are pre-computed or update during training. This should be made clear earlier. Considering the \"on-the-fly\" computation, how does it change over the epochs? Do the images migrate from one level to another? Does the perturbation norm associated with each level change over the epochs? And by how much?\n2. How does the proposed methods AdaLS/AR-AdaLS affect the accuracy? In particular, for the comparison with the other approaches in Table 2, it would be useful to compare the accuracy as well, or clearly indicate that there's no significant change in accuracy, if that's the case.\n3. Although the authors indicate in the checklist that they present error bars in the plots, none of the plots or tables in the main paper have error bars or standard deviations, even though it is claimed in the paper that results come from 5 independent runs (also, multiple runs with distinct data splits would be more informative than relying on random initialization).\n4. Due to the lack of error bars/deviation measurements, it is hard to interpret the significance of the results. For example, in Table 2, Mixup or the ensemble methods have similar or better calibration for CIFAR-100. AR-AdaLS is better only for CIFAR-10 and by a margin of 0.2, which is hard to assess if it is significant.\n5. The units used for ECE are inconsistent throughout the paper. In Figures 1 and 2, ECE values are decimals (range from 0 to 1), while in Tables 2, 3, 5 and Figure 4 it is measured in percentages (from 0 to 100).\n6. For OOD experiments: CIFAR-10 and CIFAR-100 are derived from the same original dataset and have similar classes. It would be more meaningful to compare with more distinct datasets such as SVHN, LSUN or STL-10.\n7. For Figure 3, it could be improved by showing the entropy in-distribution as well. The current figure shows that AR-AdaLS increases entropy OOD, but it's not clear how it affects the in-distribution entropy. \n8. Another useful OOD analysis would be to have the ROC plot (or measure AUCROC) for the detection of images in/out of distribution.\n9. Notation is a bit confusing or unclear: it is never described if $\\tilde{p}_r$ is a vector or a function, or if it is distinct for each example.The $z$ in the superscript is never defined in the text. In the Algorithm, the epoch index should be updated for $t+1$ in the appropriate places in the for loop.\n\nOverall, the paper investigates an interesting idea, but, from the results presented, did not convince me that the proposed method is better than the competing baselines.", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "The paper does not discuss limitations or potential negative societal impact of their work. This could be improved by identifying the underlying assumptions (e.g. measuring robustness with CW attack) of the model and where they fall short. Or identifying cases of failure and what is their impact.", "review_text": "", "extra_scores": {"needs_ethics_review": "No", "ethics_review_area": "", "time_spent_reviewing": "4 hours"}, "extra_text": {}}, {"review_id": "evw03qfpL8y", "reviewer": "Reviewer_jdF6", "summary": "The authors first expose a link between robustness and expected calibration error (ECE), the less robust a data point is, the larger the ECE. They then propose to exploit this link by introducing an adaptive label smoothing method that improves the expected calibration error of less robust data points. They benchmark their new method showing better calibration metrics on standard datasets, as well on corrupted datasets and out-of-distribution data. ", "questions": "", "limitations": "", "rating": 6, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "The authors show that their adaptive label smoothing improves upon standard label smoothing. They also show how the adaptive label smoothing can further be improved by splitting the data into several adversarial robustness bins. However, I have several concerns/questions.\n\n\n1. Mixup and other baselines. From Tab. 2 we see that Mixup outperforms the proposed method on CIFAR-100, whereas on CIFAR-10 Mixup is competitive to the proposed method.\nHowever, in the rest of the paper, the authors do not show results with Mixup. Also, additional baselines are missing in the evaluation on corrupted datasets and out-of-distribution data, which prevents the reader from entirely assessing the proposed method.  \nIn my opinion, such benchmarks, and ideally both with mixup and with competitive methods to mixup, are necessary to indeed convey the message that the proposed method works well relative to existing methods.\n\n2. While the authors use CIFAR-100 and ImageNet in Fig. 1 to show the correlation between confidence and adversarial robustness, most of the results that follow focus solely on CIFAR-10. Could the authors show the results on CIFAR-100 and ImageNet (e.g. in Fig.2)? In my opinion, such results are necessary to indeed conclude that the performance gains hold across datasets.\n\n3. As the x-axis in Fig.1 is not a hyperparameter or one value that you can change to obtain the y-value, I assume you run multiple experiments to get the curves. Could you also report the standard deviation of the curves, or if that is problematic, maybe a scatter plot?\n\n\n4. As a minor comment, I think the authors should be more precise when referring to \"classifier confidence\", and should *not* be mixed with \"uncertainty estimation\". While I think it is common to mix the two, uncertainty estimation goes beyond the former, as it requires statistical tools to estimate the epistemic and aleatoric uncertainty of the model (for example, population of models). From line 121, I think the authors should use \"classifier confidence\" where needed (e.g. caption in Fig. 1).\n\n\n5. Regarding the writing, while I did not include it in my evaluation, I think the authors could improve it in terms of simplifying the sentences, as (in my opinion) some sentences are ambiguous and overloaded, making the manuscript harder to read. For example the sentences in lines 3, 29 can be simplified, and some terms such as \"unrobust data\", \"attackable data\" could be made more precise as these depend on the classifier and not solely the data points (e.g. data points whose perturbations fool the classifier into wrong predictions).\n\n\n\n--- Minor ---\n\nIn figure 1, consider mentioning what variance is referring to.\n\nline 133: as -> with the\n\nline 139: data -> datapoints\n\ninconsistent spacing between lines 157 and 158\n\n--- Recommendation --- \n\nI encourage the authors to benchmark their method against existing methods on corrupted and out-of-distribution datasets. I would be happy to raise my score if the authors could demonstrate that their method is on par with other baselines on those benchmarks. \n\n--- Post-discussion update --- \n\nThe authors addressed most of my concerns, thus I increase my score.", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "The limitations are not addressed.", "review_text": "", "extra_scores": {"needs_ethics_review": "No", "ethics_review_area": "", "time_spent_reviewing": "5"}, "extra_text": {}}, {"review_id": "KNrAeWpVr_", "reviewer": "Reviewer_dquZ", "summary": "This paper studies the connection between adversarial robustness (measured as the distance to the decision boundary using some adversarial attack) and model calibration (i.e how well the predicted probability indicates how much we can truth the model predictions). \n\n1. The authors show that there is a significant correlation between adversarial robustness and model calibration. That is, inputs that have smaller distance to the decision boundary are more likely to have poorly calibrated predictions. \n2. Based on this insight, the authors propose an algorithm called AR-AdaLS to learn how much to smooth the labels of the training data based on their adversarial robustness. They also discuss how this technique can be extended to an ensemble model. \n3. The authors thoroughly compare their results against many previous calibration techniques and show that AR-AdaLS results in improved performance (for the single-model based methods). The results are superior also under distribution shift. They show results on CIFAR-10, CIFAR-100 and Imagenet datasets. For results under distribution shift, they show results on CIFAR-10-C, CIFAR-100-C and Imagenet-C datasets.", "questions": "", "limitations": "", "rating": 7, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "The paper studies the important problem of model calibration and provides a novel viewpoint based on the idea of using adversarial robustness (measured using distance to the decision boundary). The authors clearly discuss how the work differs from the previous related works and in which cases the performance of their proposed method is better. \n\nThe submission is technically sound. The claims are well supported through extensive experiments on well studied datasets such as CIFAR-10, CIFAR-100 and Imagenet. The results under distribution shift are also extensively compared on CIFAR-10-C, CIFAR-100-C and Imagenet-C datasets. \n\nThe paper is well written and easy to understand. \n\nI believe the results showing link between the two seemingly different research areas of adversarial robustness and model calibration will be interesting for the research community. \nThe authors perform a very thorough comparison of their method against many single-model based methods and ensemble-based or data-augmentation based methods and find that their approach results in superior performance over single-model based methods. They show results on CIFAR-10, CIFAR-100 and Imagenet datasets.\n\nFor results under distribution shifts, they show results on CIFAR-10-C, CIFAR-100-C and Imagenet-C datasets and they are significantly better than the previous methods. The authors also show results on CIFAR-100 when the model is trained on CIFAR-10 and the number of low entropy predictions on OOD data is reduced. ", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "I see no foreseeable potential negative societal impacts of their work. ", "review_text": "", "extra_scores": {"needs_ethics_review": "No", "ethics_review_area": "", "time_spent_reviewing": "6"}, "extra_text": {}}]

Author rebuttal:

Meta-review: All reviewers agreed the method proposed in this submission is insightful and novel. The authors' rebuttal has successfully addressed the reviewers' concerns. However, the reviewers are also less satisfied by the fact that the proposed method seems to be, at best, on par with other baselines (e.g. Mixup). The robustness-based label smoothing is not consistently better and there are no theoretical arguments to favor their approach. This is a borderline paper but I recommend acceptance. Despite limited performance gain compared to baseline methods, I believe the technical contributions are sufficient.

Decision: Accept (Poster)