Improving Calibration through the Relationship with Adversarial Robustness
Authors: Yao Qin
Xuezhi Wang
Alex Beutel
Ed Chi
Venue:  (2021)
Keywords: Calibration
Uncertainty Estimates
Adversarial Robustness

Abstract: Neural networks lack adversarial robustness -- they are vulnerable to adversarial examples that through small perturbations to inputs cause incorrect predictions. Further, trust is undermined when models give miscalibrated uncertainty estimates, i.e.  the predicted probability is not a good indicator of how much we should trust our model.  In this paper, we study the connection between adversarial robustness and calibration on four classification networks and datasets. We find that the inputs for which the model is sensitive to small perturbations (are easily attacked) are more likely to have poorly calibrated predictions. Based on this insight, we examine if calibration can be improved by addressing those adversarially unrobust inputs. To this end, we propose Adversarial Robustness based Adaptive Label Smoothing (AR-AdaLS) that integrates the correlations of adversarial robustness and uncertainty into training by adaptively softening labels for an example based on how easily it can be attacked by an adversary. We find that our method, taking the adversarial robustness of the in-distribution data into consideration, leads to better calibration over the model even under distributional shifts. In addition, AR-AdaLS can also be applied to an ensemble model to further improve model's calibration.

Under review as a conference paper at ICLR 2021
IMPROVING CALIBRATION THROUGH THE RELATION- SHIP WITH ADVERSARIAL ROBUSTNESS
Anonymous authors Paper under double-blind review
ABSTRACT
Neural networks lack adversarial robustness they are vulnerable to adversarial examples that through small perturbations to inputs cause incorrect predictions. Further, trust is undermined when models give miscalibrated uncertainty estimates, i.e. the predicted probability not a good indicator of how much we should trust our model. In this paper, study the connection between adversarial robustness and calibration on four classification networks and datasets. We find that the inputs for which the model is sensitive small perturbations (are easily attacked) are more likely to have poorly calibrated predictions. Based on this insight, we examine if calibration can be improved by addressing those adversarially unrobust inputs. To this end, we propose Adversarial Robustness based Adaptive Label Smoothing AS) that integrates the correlations of adversarial robustness and uncertainty into training by adaptively softening labels for an example based on how easily can be attacked by an adversary. We find that our method, taking the adversarial robustness of the in-distribution data into consideration, leads to better calibration over the model even under distributional shifts. addition, AR-AdaLS can also be applied to an ensemble model to further improve model's calibration.
INTRODUCTION
The robustness of machine learning algorithms is becoming increasingly important as ML systems are being used in higher-stakes applications. In one line of research, neural networks are shown to lack adversarial robusiness small perturbations to the input can successfully fool classifiers into making incorrect predictions (Szegedy et al., 2014; Goodfellow et al., 2014: Carlini & Wagner, 2017b; Madry et al., 2017; Qin et al., 2020b). In largely separate lines of work, researchers have studied uncertainty in model's s predictions. For example, models are often miscalibrated where the predicted confidence is not indicative of the true likelihood of the model being correct (Guo et al., 2017: Thulasidasan et al., 2019; Lakshminarayanan et al., 2017; Wen et al., 2020; Kull et al., 2019). The calibration issue is exacerbated when models are asked to make predictions from the training distribution (Snoek et al., 2019), which becomes an issue in practical settings where it is important that we can trust model predictions under distributional shift.
data different
Despite robustness, in all its forms, being a popular area of research, the relationship between these perspectives has not been extensively explored previously. In this paper, we study the correlation between adversarial robustness and calibration. We discover that input data that are sensitive to small adversarial perturbations (are easily attacked) are more likely to have poorly calibrated predictions. This holds true on a number of network architectures for classification and on all the datasets that we consider: SVHN (Netzer et al., 2011), CIFAR- 10 (Krizhevsky, 2009), CIFAR- 100 (Krizhevsky, 2009) and ImageNet (Russakovsky et al., 2015). This suggests that the miscalibrated uncertainty estimates on those adversarially unrobust data greatly degrades the performance of model's calibration. Based on this insight, hypothesize and study if calibration can be improved by giving different supervision to the model depending on adversarial robustness of each training data.
To this end, we propose Adversarial Robustness based Adaptive Label Smoothing (AR Adaas) that integrates the correlations of adversarial robustness and calibration into training by adaptively smoothing the training labels conditioned on how unrobust an input is. Our method improves label smoothing (Szegedy et al., 2014) by explicitly teaching the model to differentiate the training data according to their adversarial robustness and then adaptively smooth their labels. By giving different

---

Under review as a conference paper at ICLR 2021
supervision to the training data, our method leads to better calibration over the model without an increase of latency during inference. In particular, since adversarially unrobust data can be considered as an outlier of the underlying data distribution (Carlini et al., 2019), our method, by taking the adversarial robustness of the in-distribution data into consideration during training, can even greatly improve model's calibration on held-out shifted data. Further, we propose "AR-AdaT. of Ensemble" to combine our AddLL and deep ensembles (Lakshminarayanan et al., 2017), which is the state-of-the-art method especially under distributional shift (Snoek et al., 2019), to further improve the calibration performance for shifted data. Last, we find an additional benefit of AR-Ada is improving model stability (i.e., decreasing variance), which is valuable in practical applications where changes in predictions across runs (churn) is problematic and deploying ensembles is too costly (Milani Fard et al., 2016).
In summary, our main contributions are as follows:
We find significant correlation between adversar- ial robustness and calibration: inputs that are unrobust to adversarial attacks more likely to have poorly calibrated predictions.
Relationship among Robustness Metrics:
Algorithm: We propose ALB-AdaIS to automatically Iearn how much to soften the labels of training data based on their adersarial robustness. Further, we introduce "AR-AdaLS of Ensemble" to show how to apply AR-AdaLS to an ensemble model.
Experimental Analysis: On CIFAR-10, CIFAR-100 and ImageNet, we find that AR-Ada is more effective than previous label smoothing methods in improving calibration, particularly for shifted data. Further, we find that while ensembling can be beneficial, applying AR- AR-AdaLS to adaptively calibrate ensembles offers further improvements over calibration.
2 RELATED WORK
Uncertainty Estimates How to better estimate a model's predictive uncertainty is an important research topic, since many models with a focus on accuracy may fall short in predictive uncertainty. A popular way to improve a model's predictive uncertainty is to make the model well-calibrated, e.g.. temperature
post-hoc calibration by temperature scaling (Guo et al., 2017), and multiclass Dirichlet calibration (Kull et al., 2019). In addition, Bayesian neural networks, through learning posterior distribution over network parameters, can also be used to quantify a model's predictive uncertainty, e.g., Graves (2011), 1); Blundell et al. (2015); Welling & Teh (201 1). Dropout-base based variational inference (Gal & Ghahramani, 2016; Kingma et al., 2015) can help DNN models make less over-confident predictions and be better calibrated. Recently, mixup training (Zhang al., 2018) has been shown improve both models' generalization and calibration (Thulasidasan et al., 2019), by preventing the model from being over-confident in its predictions. Despite the success of improving uncertainty estimates over in-distribution data, Snock et al. (2019) argue that it does not usually translate to a better performance on data that shift from the training distribution. Among all the methods evaluated by Snoek et al. (2019) under distributional shift, ensemble of deep neural networks (Lakshminarayanan et al., 2017), is shown to be most robust to dataset shift, producing the best uncertainty estimates.
Adversarial Robustness On the other hand, machine learning models are known to be brittle (Xin et al., 2017) and vulnerable to adversarial examples (Athalye et al., 2018; Carlini & Wagner, 2017a;b; He et al., 2018), Many defenses have been proposed improve model's 's adversarial robustness (Song et al., 2017; Yang et al., 2019; Goodfellow et al., 2018), however are further attacked by more advanced defense-aware attacks (Carlini & Wagner, 20:7b; Athalye et al., 2018). Recently, Carlini et al. (2019); Stock & Cissé (2018) define adversarial robustness as the minimum distance in the input domain required to change the model's output prediction by constructing an adversarial attack. The most recent work that is close to ours, Carlini et al. (2019), makes the interesting observation that easily attackable data are often outliers in the underlying data distribution and then use adversarial robustness to determine an improved ordering for curriculum learning. Our work, instead, explores the relationship between adversarial robustness and calibration. In addition, we use adversarial robustness as an indicator to adaptively smooth the training labels to improve model's calibration.
Label Smoothing Label smoothing is originally proposed in Szegedy et al. (2016) and is shown to be effective in improving the quality of uncertainty estimates in Müller et al. (2019); Thulasidasan
2

---

Under review as a conference paper at ICLR 2021
Table I: Network architecture and accuracy used for each dataset.
Dataset
SVHN
CIFAR-10
CIFAR-100
ImageNet
Network
CNN-7
ResNet-29 v2
Wide ResNet-28-10 v2
ResNet-101 v1
Accuracy
95.0%
91.4%
79.2%
77.7%
al. (2019). Instead of minimizing the cross-entropy loss between the predicted probability P and the one-hot label p, label smoothing minimizes the cross-ent cros--entropy between the predicted probability and a softened label y = p(1 - <) + 2. where
the number of classes in the dataset and is a hyperparameter which controls the degree of the smoothing effect. Our work makes label smoothing adaptive and incorporates the correlation with adversarial robustness to further improve calibration.
3 CORRELATIONS BETWEEN ADVERSARIAL ROBUSTNESS AND CALIBRATION
To explore the relationship between adversarial robustness and calibration, we first introduce the metrics to evaluate each of them. We use arrows to indicate which direction is better.
Ad Adversarial robustness t Adversarial robustness measures the minimum distance in the input domain required to change the model's output prediction by constructing an adversarial attack (Carlini al., 2019; Stock & Cissé, 2018). Specifically, given an input æ and a classifier f(-) that predicts the class for the input, the adversarial robustness is defined as the minimum ad adversarial perturbation 5 that enables f(z + 8) * fz)) Following the work (Carlini et al., 2019). we construct the l2 based CW attack (Carlini & Wagner, 2017b) and then use the lg norm of the adversarial perturbation || 2 to measure the distance to the decision boundary. Therefore, a more adversarially robust input requires larger adversarial perturbation to change the model's prediction.
Calibration metric Model's calibration measures the alignment between the predicted probability and the accuracy. Well calibrated uncertainty estimates convey the information about how much we should trust a model's prediction. We follow the widely used expected calibration error (ECE) to measure the calibration performance of network (Guo et al., 2017: Snoek et al., 2019). To compute the ECE, we need to first divide all the data into K buckets sorted by their predicted probability (confidence) of the predicted class. Let B₁ represent the set of data in the k-th confidence bucket. Then the accuracy and the confidence of B, are defined as acc(B,,) = [B&T
1 EiER, 1
w:) and (Bx) = [HL EiEB, P. where á and y represent the predicted class and the true class respectively, and G is the predicted probability of ÿ. The ECE is then defined ECE- EK_ [Bal | (B)) - conf(B:)l, where N is the number of the data.
N
3.1 CORRELATIONS
In this section, we perform experiments the clean test set across four datasets: SVHN (Netzer et al., 2011), CIFAR-10 (Krizhevsky, 2009), CIFAR-100 100 (Krizhevsky, 2009) and ImageNet (Russakovsky et al., 2015) with different networks, whose architecture and accuracy shown in Table 1. We refer to these models as "Vanilla" for each dataset in the following discussion. The details for training each vanilla network are included in Section A in the Appendix.
To explore the relationship between adversarial robustness and calibration, we start with the relation- ship between adversarial robustness and confidence together with accuracy. Specifically, we rank the input data according to their adversarial robustness and then divide the dataset into I equally-sized subsets (R 10 used in this paper). For each adversarial robustness subset, we compute the accuracy and the average confidence score of the predicted class. As shown in the first row in Figure 1, we can clearly see that both accuracy and confidence increase as adversarial robustness of the input data, and confidence is consistently higher than accuracy in each adversarial robustness subset across four datasets. This indicates that although vanilla classification models achieve the of-e-oofthheart accuracy, they tend to give over-confident predictions, especially for those unrobust data.
Based on this, to explore the relationship between adversarial robustness and calibration, we compute the expected calibration error (ECE) in each adversarial robustness subset, shown in the bottom row of Figure 1. In general, we find that those unrobust data in lower adversarial robustness levels are more likely to be over-confident and less well calibrated (larger ECE). For more robust examples, there is a better alignment between their confidence and accuracy, and the ECE over those examples is close to O. On larger-scale ImageNet, while still see the general trend that less robust examples
3

---

Under review as a conference paper at ICLR 2021
SVHN
CIFR-10
CIFAR-10D
Imageket
1.0
2.0
1.0
0.8
0.0
Score
Score
0.6
2.6
Accuracy
Accuracy Confidence
Accuracy
Confidence
Confidence
Confidence
2
10
6
30
2
10
10 Adversarial Robustness Level
6
Adversarial Robustness Level
Adversarial Robustness Level
Adversarial Robustness Level
SVHN
CIFAR-10
CIFAR-100
Imagelitt
ECE
ECE
ECE
ECE
02
Variance
0.2
Veriance
122
Varianca
022
Variance
Score
Score
0.2
0.1
D.1
D.1
0.0
20
0.0
20
0.0
10
0.0
10
Adversartal Robustness Level
Adversarial Robustness Level
Adversartal Robustness Level
Adversarial Robustress Level
Figure 1: Correlations between adversarial robustness and uncertainty estimates on the clean test set on SVHN CIFAR-10, CIFAR-100 and ImageNet. Top: Accuracy and confidence of the predicted class. Bottom: ECE and variance (lower is better)in each adversarial robustness subset. Higher adversarial robustness level means the input are more adversarially robust (harder to attack).
are less well calibrated, we see that the least robust examples are relatively well calibrated. We hypothesize this may be due to larger data and less overfitting.
Furthermore, we also find an interesting correlation between adversarial robustness and model stability, which is measured by the variance of the predicted probability across 5 independent runs. As shown in the bottom row of Figure 1, we see that those adversarially unrobust examples tend to have a much higher variance compared to the robust across all four datasets.
METHOD
Based on the correlation between adversarial robustness and calibration, we hypothesize and study calibration can be improved by giving different supervision to the model depending the adversarial robustness of training data. To this end, we propose a method named Adversarial Robustness based Adaptive Label Smoothing (AR-AdaLS), which performs label smoothing at different degrees to the training data based on their adversarial robustness. Specifically, we sort and divide the training data into R small subsets with equal size according to their adversarial robustness' and then use e, to soften the labels in each training subset Strain The soft labels can be formulated as:
pr = p. +1) 4
Z
where y²- = for the correct class y and p = 0 for the others, p, stands for the one-hot label, and Z is the number of classes in the dataset. The parameter e, controls the degree of smoothing effect and allows for different levels of smoothing in each adversarial robustness subset. Generally, a relatively larger 6, is desirable for lower adversarial robustness levels such that the model learns to make a lower confidence prediction. Instead of empirically setting the parameter 6, in each adversarial robustness subset, we allow it to be adaptively updated according the calibration performance on the validation set (discussed in Section 4. 1). In this way, we explicitly train a network with different supervision based on the adversarial robustness of training data.
4.1 ADAPTIVE LEARNING MECHANISM
To find the best hyperparameter e for label smoothing, previous methods (Szegedy et al., 2016; Thulasidasan et al., 2019) sweep e in a range and choose the one that has the best validation performance. However, in our setting, the number of combinations of 6, increases exponentially with the number of adversarial robustness subsets R. To this end, we propose an adaptive learning mechanism to automatically learn the parameter e, in each adversarial robustness subset. The overall training procedure is summarized in Algorithm 1.
First, we denote the soft label for the correct class in the r-th adversarial robustness subset as pX According to Eqn. (I), we can derive:
NNote, predicted confidence is not good indicator for splitting the training dataset as the model can easily overfit the training data and their predicted confidence are all close 100%.
4

---

Under review as a conference paper at ICLR 2021
Algorithm 1 Pseudocode of the training procedure for AR-AdaLS
Input: number of classes Z, number of training epochs 7', number of adversarial robustness subset R. learning rate of adaptive label smoothing a.
For each adversarial robustness training subset, we initialize the soft label as the one-hot label Pr.. Pr, and initialize the soft label for the correct class Pr, =1 1.
for ( = 1 to 7 do
Minimize cross-entropy loss between soft label and predicted probability 1 Σx ((rr..Pr.t))
for r
R do Update P +- Pr.t
{conf(Syaa) acc(Spal) )c
according to Eqn. (3)
Clip Pr.t to be within (z:
Update c, t (Pr,t 1) 1-2
according to Eqn. (4)
Update Pr.t - Pr [1 tr,t) + Set
according to Eqn. (1)
end for end for
pr 1 Ey
Since well-calibrated uncertainty estimates should be aligned with the empirical accuracy, we use the calibration performance in the validation set to help update pr for the training data. Specifically, we first rank the adversarial robustness of the validation data and split the validation set into R equally-sized subsets. Then, we use the difference between confidence and accuracy in the r-th adversarial robustness validation subset conf(S (Se(S to update the soft label for the correct class of training data in the r-th adversarial robustness training subset Strain
Pr.+1 = Pr. a {conf(S;")) - acc(S;al), )
where Pr,," is the soft label of the correct class in the r-th adversarial robustness training subset at time step L. The accuracy and the confidence of Syat are defined as aco(Spal) = [ Eiesyat 1 (ú: = y/)
and (S
Syuel Evesqol P; 22- 0; where ÿ and y is the predicted class and the true class
respectively, f denotes the the predicted probability of the predicted class. The hyperparameter plays a role learning rate update the soft label p based on the difference between the predicted confidence and accuracy in the valiadation set. Intuitively, if we assign a large f7 to training data, then the network tends to make a high confidence prediction and vice versa. Therefore, the confidence greater than the accuracy (Sonf() > acc(Spa()) ))) in the validation set, we should reduce p; to teach the network to be less confident. Otherwise, we should increase p " In addition, we also need to constrain p. " to be within (z. 1] after each update as stands for the true probability of the correct class, where Z is the number of classes in the dataset.
For given py- we can easily obtain c, by reversing Eqn. (2): %
6, (ロ 1) 1- 7'
(4)
and the soft labels for all the classes p, can be computed according Eqn. (1). We update the soft labels after each training epoch in our experiments.
Note that this adaptive learning mechanism can be easily applied standard label smoothing without adversarial robustness slicing (R = 1). In this case, we can replace sweeping the hyperparameter e with this adaptive learning method, named as "Adaptive Label Smoothing" (AdaLS). Our proposed AdaLs and AR AdaLS does not increase the inference time: we test AdaLS and AR-Adea AdaLS exactly the same as a vanilla model.
EXPERIMENTS
We now test our methods on both clean and shifted data for CIFAR- CIFAR-100 and ImageNet for calibration. The shifted dataset (Hendrycks & Dietterich, 2019) consists of different types (19 types for CIFAR-10, 10, 17 types for CIFAR- 100 and 15 types for ImageNet) of corruptions, e.g., noise, blur, weather and digital categories that are frequently encountered in natural images. Each type of corruption has five levels of shift intensity, with higher levels having more corruption. For each shift intensity, we report the results with box plot summarizing the 25th, 50th, 75th quartiles across the
5

---

Under review as a conference paper at ICLR 2021
Table 2: Ablation study of AR-AdaLS on CIFAR- 100 and CIFAR100-C (corrupted). We report both accuracy and expected calibration error, denoted by Ace and ECE for the clean test set, and cAce and cECE for CIFAR100-C. Arrow indicates the better direction; best calibration of single model is bolded.
Label Smoothing
Temperature Scaling
AR-AdaLS
AR-AdaLS (on-the-fly)
Ensemble Vanilla
Method
Vanilla
(pre-compute)
Acc/cAce (f) 79.2/52.0
78. 78.9/51.7
79.2/52.0 4.3/14.0
79.3/52.2 2.6/14.2
79.2/52.1 2.3/13.2
81.8/55.1
ECE/CECE (U) 6.1/18.2
2.8/16.3
2.2/10.5
(a) LS
(b) AR-AdaLS
(c)
1.0
1.0
0.3
Vanilla
0.8
0.3
Score
Score
AR-AdaLS
ECE
0.8
0.5
0.1
Arcuracy
Arcuracy
0.4
Confidence
0.4
Confidence
2
2
10
4
10
Adversarial Robustness Level
Adversarial Robustness Level
Adversarial Robustness Level
Figure 2: Comparison between LS and our AR AR-Ad on the clean test set of CIFAR-10. (a) and (b): Accuracy and confidence score of the predicted class each adversarial robustness subset. (c): ECE of Vanilla, LS and AR AR-AdaLS.
types of shift. We first test single-model approaches and then explore how our methods compose with deep ensembles (Lakshminarayanan et al., 2017).
5.1 BASELINES
We compare our proposed AR-AdaLS with the following methods: Vanilla model that is trained with one-hot labels, Temperature Scaling (Guo et al., 2017), label smoothing (I.S) (Szegedy et al., 2016) that softs labels by sweeping the hyperparameter which controls the smoothing degree in range to find the best hyperparameter e, Adaptive Label Smoothing (AdaLS): we use our proposed adaptive learning mechanism introduced in Section 4.1 to automatically learn the hyperparameter e rather than sweeping to find the best e. In addition, we also report the results of Ensemble of Vanilla (Lakshminarayanan et al., 2017) with M 5 vanilla models independently trained with random initialization. We will further discuss that our method is complementary to deep ensembles. All the methods trained with the same network architectures and training hyperparameters: e.g., learning rate, batch size, number of training epochs, for fair comparison. Please refer to Appendix A for all the training details and hyperparameters.
5.2 IMPROVEMENTS OVER SINGLE MODEL.
Ablation Study There are two options to compute adversarial robustness. One is "on-the-fly": to keep creating adversarial attacks during training, which provides precise adversarial robustness rank- ing but at the cost of great computing time. The other is to "pre-compute" adversarial robustness by attacking vanilla model. This is more efficient but at the sacrifice of the precision of adversarial ro- bustness ranking. We perform experiments on CIFAR- 100 as an example to compare the performance of AR-AdaLS based on the adversarial robustness that is "pre-computed" or "on-the-fly".
As shown in Table 2, generating adversarial robustness "onn-hheffl" helps improve the calibration performance further for AR-AdaLS both clean and shifted datasets compared to pre-computing adversarial robustness. When comparing AR AdaLS with other single-model based methods, we can that all models have similar accuracy on clean and shifted datasets. Both versions of AR-AdaLS significantly improve the calibration performance over label smoothing, especially under distributional shifts. Further, we observe that AR-AdaLS with pre-computed adversarial robustness has similar performance as Temperature Scaling but AR-AdaLS based on more precise adversarial robustness ("on-the-fly")) is significantly better". In the following sections, all results related "AR-Ada T.S" are based on pre-computed adversarial robustness for efficiency. This is because our main target
ASimilarly, we find that pre-computed AR-AdaLS improves on LS and performs nearly as well as TS on CIFAR-10 and ImageNet. We also see similar improvements from on-the-fly AR- Ae-Adal S on CIFAR-10, but we did not run on-the-fly AR-Ada I.S for ImageNet due to the computational intensity.

---

Under review as a conference paper at ICLR 2021
CIFAR-10
0.50
AR-AdaLS Ensemble Venille
LS
0.20
Test
Shift intensity
ImageNet
0.25
Vanilia
AR-AdaLS
0.20
AdeLS
0.15
0.05
0.00
Test
Shift intensity
Figure 3: ECE on clean test and shifted data on CIFAR- 10 and ImageNet. For each shift intensity, we show report box plot summarizing the 25th, 50th, 75th quartiles across 19 shift types on CIFARIO-C and 15 shift types on ImageNet-C. The error bars indicate the min and maz value across different shift types. Similar figures for Accuracy are shown in Figure in Appendix and we observe that all the single-model based methods have comparable accuracy while ensembles achieve higher accuracy.
is to show that the idea of differentiating the training data based on their adversarial robustness is promising improve model's calibration rather than pushing the results the best.
Visualization of improvement over label smoothing Since our proposed AR-AdaLS is built upon label smoothing (LS), Figure we visualize the effect label smoothing (LS) and our AR-AdaLS. Comparing Figure 2 (a) and (b). AR-AdaLS is better calibrating the data than label smoothing, especially on the unrobust examples (lower adversarial robustness level). Further, we show plots of ECE in Figure (c). Both label smoothing and AR- AdaLS improve model's calibration over vanilla model and AR-AdaLS has the best performance among three methods. This suggests that -ddLS better improving calibration unrobust regions, not just on average.
Generalization over shifted dataset Figure and Table 4 summarizes the ECE for CIFAR-10 10 and ImageNet for both clean and shifted data with different levels of corruptions (Hendrycks & Dietterich, 2019). On the clean test set, all non-vanilla methods achieve comparable low values of ECE. When the intensity of shift increases, ARR-Aaaa LS significantly outperforms other single- model based methods with the lowest ECE. Contrasting with LS and AdaLS, we see AR-AdaLS benefits greatly from the adversarial robustness slicing. As a result, our model learns to give smaller soft labels of the correct class those adversarially unrobust training data, which can also be considered as outliers of the underlying data distribution (Carlini et al., 2019). Therefore, when tested on the shifted data that deep networks have been shown to produce pathologically over- confident predictions (Hendrycks & Dietterich, 2019), our model correctly learns to make relatively lower confidence prediction, resulting in a better calibration performance. When we compare to see that AR-AdaLS achieves comparable calibration
an ensemble of five Vanilla models, performance on CIFAR-10 and the ensemble is better under highly shifted data on ImageNet.
Sensitivity analysis To analyze the effect the number of adversarial robustness subset R in AR-AdaLS, we plot the calibration enror of AR-AdaLS with varying R on the clean CIFAR-10 and corrupted CLFAR-10-CTO in Figure We can see that there is a significant drop in cali- bration error (ECE) when we increase the number of adversarial robustness subset R from 1, where notes AdaLS. Further, the calibration error is robust when Rt is chosen within the range [4, 16).
Calibration
ECF
1.0
ECE
10
0.5
0.0
17
10 12 14 Adversarial Robustnets Subset
12
12
Adversorial Robustnets Subset
Figure 4: ECE on CIFAR- 10 and CIFAR- AR-AS with varying number of adversarial robustness subset
de-
7

---

Under review as a conference paper at ICLR 2021
CIFAR-DDDIICCIFAA--000//00
CTTAR--DInnCCIFA---00OOOW
3000
Venille
3000
3000
2000
Court
2000
Court
2000
1000
0.0
0.5
1.0
1.5
2.0
0
0.0
0.5
1.0
1.5
2.0
0.0
0.5
1.0
1.5
2.0
Entropy
Entropy
Entropy
Figure 5: Histogram of predictive entropy on out-of-distribution data. Each model is trained on
CIFAR-10 and tested on CIFAR-100.
Table 3: Mean of variance (x x100²) across 19 types of shift for CIFAR- and 15 types of shift for ImageNet-C. The best model is shown in bold.
Dataset
CIFAR-IO-C
ImageNet-C
Shift Intensity
2
2
3
5
Vanilla
7.85
9.69
11.2
13.1
16.0
5.28
6.39
7.37
8.23
8.29
LS
5.54
6.95
8.11
9.65
11.8
4.86
5.84
6.78
7.55
7.41
AdaLS
5.47
6.87
7.95
9.44
11.5
4.79
5.77
6.66
7.51
7.56
AR AR-AdaLS
4.21
5.06
5.73
6.66
8.24
4.53
5.49
6.12
6.76
6.66
Improvements on Out-of-Distribution Data We further study the performance of AR-AdaLS when predicting on cuttof-distribution (OOD) data. In Figure 5, we compare the performance of Vanilla, Label Smoothing and AR-AdaLS by plotting the histogram of the entropy the OOD data (higher entropy on OOD is better). Each model trained on CIFAR-10 10 dataset and then tested on CIFAR-100 dataset. We low-entropy prediction on OOD data, which demonstrates the effectiveness of AR-AdaLS even on fully duitt--dist data.
clearly see that AR AR-Ad significantly reduces the number of
Improvements over stability Since we observe in Figure that the most adversarially unrobust data are also very unstable, we test AR-AdaLS to see if it can help improve model stability, which is of great value in practice where high variance of a model is bad for churn and deploying ensembles too costly (Milani Fard et al., 2016). Experiments show that AR-AdaLS can effectively reduce the variance model compared to a vanilla model and label smoothing on CIFAR- 10 and ImageNet. An extensive set of results are shown in Table 3 and Figure 11 in the Appendix.
5.3 COMBINATION WITH DEEP ENSEMBLES
We now discuss the two best models, deep ensembles and our AR-Ada are complementary. To this end, propose the following two ways to combine them: Ensemble of AR-AdaLS: As in Lakshminarayanan et al. (2017); Lee et al. (2015), we ensemble AR- AR-AdLLS by training multiple independent AR-AdaLS models with random initialization, and average their predictions inference.
AR-AdaLS of Ensemble: Instead of computing soft labels independently for each AR-AdaLS, perform AR-AdaLS the ensembled predictions, i.e., in Eqn (3) we compute confidence and accuracy based on the average of M = 5 model predictions. Each model is then supervised with the same soft labels. We will this slight distinction in training quite important.
Discussion We present the results for CIFAR-10 and ImageNet in Figure 6 and Table 4. At high level, we see that AR-AdaLS of Ensemble performs the best across both clean test data and all intensities of shifted data. Looking more closely, some trends emerge: all of the ensemble methods perform relatively well for highly shifted data (intensity 4-5), but Ensemble of AR AR-AdaLS performs much worse on less shifted and clean test data. Please refer to Figure 8 in Appendix for more extensive results on CIFAR-10 and Imagenet.
Digging deeper, we display the confidence of the predictd class and accuracy of each single model and the corresponding ensemble models on the clean test set of CIFAR-10 and ImageNet in Figure 10 in Appendix. We can clearly see that the ensemble models generally increase accuracy and decrease confidence compared to single model, which results from the disagreement of the prediction of each single model ensembles. Therefore. naive deep ensembles can improve calibration on highly shifted data where single-model is over-confident but harm calibration if applied to well-
8

---

Under review as a conference paper at ICLR 2021
0.5
Vanilla
Ensemble ARtAdtaLS
0.4
AR-AdaLS
0.2
0.1
Test
Shift intensity
Figure 6: Comparison of ensembled models: ECE both clean test data and shifted data CIFAR- 10. For each intensity of shift, we report a box plot summarizing the 25th, 50th, 75th quartiles across 19 types of shift in CIFARIO-C. The error bars indicate the min and max value across shift types.
Table 4: Mean of ECE (x 1010- 2) across 19 types shift for CIFAR-I0- -O-C and 15 types of shift for ImageNet-C. The best single model and ensemble model are shown in bold respectively.
Dataset
CIFAR-IO-C
ImageNet-C
Shift Intensity
2
2
3
4
5
Vanilla
9.59
12.9
15.6
19.5
25.7
4.67
6,94
9.60
13.2
16.6
LS AdaLS
4.45
7.06
9.15
12.1
17.5
2.38
4.62
7.39
11.4
15.1
4.06
6.65
8.85
11.7
16.5
2.37
4.48
7.18
11.1
14.6
AR-AdaLS
2.57
4.09
11.9
2.44
3.58
5.80
9.33
12.9
Ensemble Vanilla
2.47
4.13
5.73
8.03
2.77
2.51
2.87
4.64
8.26
Ensemble
2.77
3.24
3.96
5.20
4.96
4.18
3.65
3.65
6.94
Ensemble of Adaī
3.48
3.74
4.86
5.95
7.74
5.20
4.36
3.61
3.74
6,92
Ensemble of A I.S
4.23
4.52
5.23
5.68
7.68
5.76
5.23
4.51
3.69
6.33
AR-AdaLS of Ensemble
1.97
2.98
3.91
5.38
7.98
3.41
3,03
2.84
3.61
7.03
CIFAR-10
CIFAR-10
ImageNet
ImageNet
1.0
1.0
1.0
1.0
0.8
0.8
0.8
0.8
Accurac)
Accuracy
Accuracy
0.6
Accuraci
0.6
0.6
Wella
ARACCS
8.00
0.25
0.75
8860
0.25
0.75
1.00
0.0
0.25
0.50
1.00
8.00
0.5
aão 0.75
Confidence
Confidence
Confidence
Figure 7: Reliability diagram of accuracy versus confidence of single model and ensemble model on the clean test of CIFAR-10 10 and ImageNet. The perfect calibrated model should be aligned with the diagonal dotted line (above is under-confident, below is over-confident).
calibrated single-model. This is made clearer in Figure 7: while deep ensembles make over-confident vanilla model well calibrate, leads the well calibrated AR-Ach AR-AdaLS models be under-confident (similar patterns observed on LS and AdaLS, shown in Figure 9 in Appendix). From this perspective, AR-Ada LS of Ensemble avoids this issue by adaptively adjusting smoothing to keep the ensemble well calibrated. Taken together we see that AR AR-AdaLS improves calibration for both single-models and ensembles.
CONCLUSION
In this paper, we have explored the correlations between adversarial robustness and calibration. We find across four datasets that adversarially unrobust (casily attacked) data are more likely to have poorly calibrated and unstable predictions. Based on this insight, we propose AR-Ada to adaptively smooth the labels of the training data based on their adversarial robustness. In our experiments we see that AR- Ad-AdLLS is more effective than previous label smoothing methods in improving calibration, particularly for shifted data, and can offer improvements strong ensembling methods. We believe this is an exciting new use for adversarial robustness as means to more generally improve model trustworthiness. not just by limiting adversarial attacks but also improving uncertainty on unexpected data. We hope this spurs further work at the intersection of these areas of research.
top of already

---

Under review as a conference paper at ICLR 2021
REFERENCES
Anish Athalye, Nicholas Carlini, and David Wagner. Obfuscated gradients give false sense of security: Circumventing defenses to adversarial examples. In International Conference on Machine Learning, 2018.
Charles Blundell, Julien Cornebise, Koray Kavukcuoglu, and Daan Wierstra. Weight uncertainty in neural network. volume 37 of Proceedings of Machine Learning Research, pp. 1613-1622, Lille, France, 07-09 Jul 2015. PMLR. URL http:/ httt:::/ppoceeeingg.. mww.mmlr.......... wwl...pree//v317 blundel115 html.
Nicholas Carlini and David Wagner. Adversarial examples are not easily detected: Bypassing ten detection methods. In Proceedings of the 10th ACM Workshop Artificial Intelligence and Security, AlSec '17, pp. 3-14, New York, NY, USA, 2017a. Association for Computing Machinery. ISBN 978145035204,, doi: 10.1145/3128572.314000 URL https: /tw.i//oi.////// owg/1o.gg1110o 1145/ 3128572. 3140444
Nicholas Carlini and David Wagner. Towards evaluating the robustness of neural networks. In 2017 IEEE Symposium on Security and Privacy (SP), pp. 39-57. IEFE, 2017b.
Nicholas Carlini, Ülfar Erlingsson, and Nicolas Papernot. Distribution density, tails, and outliers in machine learning: Metrics and applications. ArXiv, abs/1910. 13410..117,,... 2019.
Yarin Gal and Zoubin Ghahramani. Dropout as a bayesian approximation: uncertainty in deep learning. In International Conference Machine Learning, pp. 1050-1059, 2016.
Representing model
Ian Goodfellow, Jonathon Shlens, and Christian Szegedy. Explaining and harnessing adversarial examples. In International Conference on Learning Representations, 2014.
Ian Goodfellow, Yao Qin, and David Berthelot. Evaluation methodology for attacks against confidence thresholding models. 2018.
Alex Graves. Practical variational inference for neural networks. In J. Shawe-Taylor, R. S. Zemel, P. L. Bartlett, Percira, and K. Q. Weinberger (eds.), Advances in Neural Information Processing Systems 24, pp. 2348-2356. Curran Associates, Inc., 2011. URL http: /ttppaaapaeae///a//e/e/eeea/aaea nips cc/ paper/4329 ptpeer/429-prrccccalll wawiicallllaaiiiooo inffrenneeeeeee pdf.
Chuan Guo, Geoff Pleiss, Yu Sun, and Kilian Q. Weinberger. On calibration of modern neural networks. In International Conference on Machine Learning, 2017.
Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition. In Proceedings of the IEEE conference on Computer Vision and Pattern Recognition. pp. 770-778, 2016a.
Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Identity mappings in deep residual networks. In European Conference on Computer Vision, pp. 630-645. Springer, 2016b.
Warren He, Bo Li, and Dawn Song. Decision boundary analysis of adversarial examples. In International Conference Learning Representations, 2018.
Dan Hendrycks and Thomas G. Dietterich. Benchmarking neural network robustness common corruptions and perturbations. International Conference on Learning Representations, 2019,
Durk P Kingma, Tim Salimans, and Max Welling. Variational dropout and the local reparameterization trick. In Advances Neural Information Processing Systems, pp. 2575-2583, 2015.
Alex Krizhevsky. Learning multiple layers of features from tiny images. Technical report, University of Toronto, 2009.
Meelis Kull. Miquel Perello Nieto, Markus Kängsepp, Telmo Silva Filho, Hao Song, and Peter Flach. Beyond temperature scaling: Obtaining call-calibrated multi-class probabilities with dirichlet calibration. In Advances in Neural Information Processing Systems 32, pp. 12316-12326, 2019.
10

---

Under review as a conference paper at ICLR 2021
Balaji Lakshminarayanan, Alexander Pritzel, and Charles Blundell. Simple and scalable predictive uncertainty estimation using deep ensembles. In Advances in Neural Information Processing Systems, pp. 6405-6416. Curran Associates Inc., 2017. ISBN 9781510860964
Stefan Lee, Senthil Purushwalkam, Michael Cogswell, David Crandall, and Dhruv Batra. Why m heads are better than one: Training a diverse ensemble of deep networks. arXiv preprint arXiv:/511.063 14, 2015.
Aleksander Madry, Aleksandar Makelov, Ludwig Schmidt, Dimitris Tsipras, and Adrian Vladu. Towards deep learning models resistant to adversarial attacks. In International Conference on Learning Representations, 2017.
Mahdi Milani Fard, Quentin Cormier, Kevin Canini, and Maya Gupta. Launch and it- erate: Reducing prediction churn. In D. D. Lee, M. Sugiyama, U. V. Luxburg. Guyon, and R. Garnett (eds.), Advances in Neural Information Processing Systems 29, pp. 3179-3187. Curran Associates, Inc., 2016. URL http: http://papers niis. cc /aper/ pdf.
Rafael Müller, Simon Komblith, and Geoffrey E Hinton. When does label smoothing help? In Advances in Neural Information Processing Systems, pp. 4694-4703, 2019.
Yuval Netzer, Tao Wang, Adam Coates, Alessandro Bissacco, Bo Wu, and Andrew Y. Ng. Reading digits in natural images with unsupervised feature learning. In NIPS workshop on deep learning and unsupervised feature learning, 2011.
Yao Qin, Nicholas Frosst, Colin Raffel, Garrison Cottrell, and Geoffrey Hinton. Deflecting adversarial attacks. arXiv preprint MiXi::0002.07405: 2020a.
Yao Qin, Nicholas Frosst, Sara Sabour, Colin Raffel, Garrison Cottrell, and Geoffrey Hinton. Detect- ing and diagnosing adversarial images with class-conditional capsule reconstructions. International Conference on Learning Representations, 2020b.
Olga Russakovsky, Jia Deng, Hao Su, Jonathan Krause, Sanjeev Satheesh, Sean Ma. Zhiheng Huang, Andrej Karpathy, Aditya Khosla, Michael S. Bernstein, Alexander C. Berg, and Fei-Fei Li. Imagenet large scale visual recognition challenge. International Journal of Computer Vision, 1 115: 211-252, 2015.
Jasper Snoek, Yaniv Ovadia, Emily Fertig, Balaji Lakshminarayanan, Sehastian Nowozin, D Sculley, Joshua Dillon, Jie Ren, and Zachary Nado. Can you trust your model's uncertainty? evaluating predictive uncertainty under dataset shift. In Advances in Neural Information Processing Systems, pp. 13969-13980, 2019.
Yang Song, Taesup Kim, Sebastian Nowozin, Stefano Ermon, and Nate Kushman. Pixeldefend: Lever- aging generative models to understand and defend against adversarial examples. In International Conference on Learning Representations, 2017.
Pierre Stock and Moustapha Cissé. Convnets and imagenet beyond accuracy: Understanding mistakes and uncovering biases. In European Conference on Computer Vision, 2018.
Christian Szegedy, Wojciech Zaremba, Ilya Sutskever, Joan Bruna, Dumitru Erhan, Ian J. Goodfellow, and Rob Fergus. Intriguing properties of neural networks. In International Conference on Learning Representations, 2014. URL http: //tpvv/rvvvvv//aa////a//a../..wa org/ abs / 1112. 6199.
Christian Szegedy, Vincent Vanhoucke, Sergey loffe, Jon Shlens, and Zbigniew Wojna. Rethinking the inception architecture for computer vision. 2016 IEEE Conference on Computer Vision and Pattern Recognition, pp. 2818-22888800 2816-2826. 2016.
Sunil Thulasidasan. Gopinath Chennupati, Jeff A. Bilmes, Tanmoy Bhattacharya, and Sarah Ellen Michalak. On mixup training: Improved calibration and predictive uncertainty for deep neural networks. In Advances in Neural Information Processing Systems, 2019.
Max Welling and Yee Whye Teh. Bayesian learning via stochastic gradient langevin dynamics. In Proceedings of the 28th International Conference on International Conference on Machine Learning, ICML'11 11, pp. 681-688 Madison, WI, USA, 2011. Omnipress. ISBN 97819981111 150036619.
11

---

Under review as a conference paper at ICLR 2021
Yeming Wen, Dustin Tran, and Jimmy Ba. Batchensemble: An alternatve approach to efficient ensemble and lifelong learning. In International Conference on Learning Representations, 2020.
Doris Xin, Nicolas Mayoraz, Hubert Pham, Karthik Lakshmanan, and John R Anderson. Folding: Why good models sometimes make spurious recommendations. In Proceedings of the Eleventh ACM Conference on Recommender Systems, pp. 201-209, 2017.
Yuzhe Yang, Guo Zhang, Dina Katabi, and Zhi Xu. Me-net: Towards effective adversarial robustness with matrix estimation. In International Conference on Machine Learning, 2019.
Sergey Zagoruyko and Nikos Komodakis. Wide residual networks. ArXiv, abs/ 0als/1605.01 146, 2016.
Hongyi Zhang, Moustapha Cissé, Yann Dauphin, and David Lopez-Paz. Mixup: Beyond empirical risk minimization. In International Conference on Learning Representation, 2018.
A IMPLEMENTATION DETAILS
A.1 SVHN
We use a simple network architecture with 7 convolutional layers and follow all the training details introduced in Qin al. (2020a) for SVHN. This network architecture achieves the otate-of-the-art accuracy on SVHN.
A.2 CIFAR-10
All the experimental results on CIFAR-I0 were obtained with a ResNet-29 v2 (He et al., 2016b) with batch size of 256. The network is trained with Adam optimizer (Kingma et al., 2015) for 200 epochs. The initial learning rate is 10-3 and decayed down to 10-4 after 80 epochs, 10-6 after 120 epochs, 10 after 160 epochs and 0.5 10 after 180 epochs. We adapted the following data augmentation and training script at https: fttt::///ereee /keras... io/ 1o/examp) exampl es / ci 2aar00rreeee//... The training mechanism is the same for all the methods that we compare in the main paper. We randomly split the training dataset into training data of 45000 images and 5000 images the validation set. The test set has 10000 images.
For label smoothing (LS), sweep the hyperparameter within the range [0, 0.1] with step size 0.01 and find that the network has the best calibration performance on the validation set when 0.02.
For Adaptive Label Smoothing (AdaL-)) there is a hyperparameter a which plays a role as learning rate in the adaptive learning mechanism. We choose hyperparameter based the calibration performance on the validation set. Specifically, we run experiments with a E {0.005, 0.01, 0.05, 0.1) and find that 0.05 achieve the best calibration performance.
Similarly, for Adversarial Robustness based Adaptive Label Smoothing (AR- -d),LS, we choose the hyperparameter a from the (0.005, 0.01, 0.05) and empirically set 0.005 which has the best calibration performance on the validation set. We use the same hyperparameter a 0.005 without further tuning for AR-AdaLS of Ensemble.
All the results of ensemble models are obtained via training 5 independent models with random initializations.
A.3 CIFAR-100
We train a Wide ResNet-28-10 v2 (Zagoruyko & Komodakis, 2016) to obtain the state-of- the-art accuracy for CIFAR-100. We adapt the same training details and data augmenta- tion at
com/ http//ooole/eeeee http://////eeeaaa////////ttt////
deterministie.pa
For label smoothing, we e sweep the hyperparameter within the range [0, 0.1] with a step size 0.01 and find that the network has the best calibration performance the validation set when 0.07.
12

---

Under review as a conference paper at ICLR 2021
The hyperparameter is set be 0.005 in AR-AdaLS, For AR-AdaLS that generated with on-the- fly adversarial examples, we recompute the adversarial robustness for training and validation sets after 65, 130 epochs.
A.4 IMAGENET
All the experiments on ImageNet were obtained via training ResNet- 101 v1 (He et al., 2016a) following the training script at https://githut::
www/googgeeeeeeeeeooooooooeoeeee hetpl///ddwrd22/ / /bwob//////oooooc allobmma /ma er basel nes/ / imagenet/ weeennee/deeeeinnnnne ristiiiiiiiiii.immmmmmmmmmmmmiit py. The network is trained with a batch size of 128 for each TPU core with SGD optimizer for 90 epochs. The input image is normalized (divided by 255) to be within [0.1]. We randomly divide 50000 validation images into validation set with 25000 images and test set with 25000 images. Note that the same dataset and training mechanisms are used for all the methods that we compare in the main paper.
For Label Smoothing (LS), we sweep the hyperparameter e within the range [0, 0.1] with a step size 0.01 and find that the best calibration performance the validation set is achieved by setting = 0.02.
For Adaptive Label Smoothing (AdaLS), sweep the hyperparameter in the set {0.005, 0.01, 0.03, 0.05, 0.1} and set it to be a = 0.03 for the best calibration performance on the validation set.
We empirically set a = 0.001 for AR-AdaLS in the first 60 epochs of the training and then increase it 0.05 for the next 30 epochs. The same hyperparameter is used for AR-AdaLS of Ensemble without further tuning.
All the ensemble models combination of independent models with random initializations.
A.S CW ATTACKS
To compute the adversarial robustness, we construct & based CW attacks (Carlini & Wagner, 2017b) following the code at ht.pp:: //gi thub f low wwt.:lloo//c//ooeooeeeoooee weleeveaanss//ee //bl ob/ master/ cww.eeeaasss///eecceeeee http//tttakk///ee/e carkini _eriiiiaaaer_____ 12 py, Specifically, we set the binary search steps to be 3, max iterations to be 500 and learning rate to be 0.005. The generated untargeted CW attacks achieve 100% success rate for all the datasets that consider: SVHN, CIFAR- 10. CIFAR- 100 and ImageNet. We set the number of adversarial robustness training subset and validation subset be R 10 respectively.
B CALIBRATION PERFORMANCE. ADDITIONAL RESULTS
In Figure 8, show ECE and accuracy of all the single-models and their corresponding ensembles on the clean test and shifted CIFAR-10 10 and ImageNet. The high-level conclusion is: 1) all the ensemble models that we compare have similar accuracies, which are higher than single-models. 2) all the ensemble methods perform relatively well for highly shifted data but ensembles of well calibrated models (Ensemble of LS, Ensemble of AdaLS, Ensemble of AR AdaLs) perform much worse on less shifted and clean test data. 3) AR-AdaLS of Ensemble successfully learns to adaptively adjust its smoothing level keep the ensemble well calibrated on both clean and shifted dataset.
Figure 9 is to validate the observation that naive ensembling of single model can improve calibration over-confident but can harm calibration on clean test
on highly shifted data where single- data where single model is well calibrated. As shown in Figure 9, we can see that if the single-model over-confident (Vanilla), Ensemble of Vanilla greatly improves the model's calibration. However, when the single model is well-calibrated (LS, AdaLS, AR ARRAA,LS), naive ensembling of these models leads to an under-confident model with worse calibration performance. In contrast, AR AR-AdaLA of Ensemble successfully avoids this issue by adaptively smoothing the training labels according to adversarial robustness of training data.
13

---

Under review as a conference paper at ICLR 2021
LS
0.4
L5
Ensemble AdsLS
Eroemble
0.3
AR AdaLL
All AdaLS Ensemale
Ensamble Vanillo
CIFAR-10
0.0
Test
2
1.0
Accuracy
0.8
0.6
0.2
Ensemble LS
0.3
L5
Ensemblee AdsLS
troemble Alt-AclaL3
AR AdaLL
All AdaLs Ensomaae
0.2
Ensomble Vanilio
ImageNet
0.1
Test
0.8
0.6
0.4
0.2
Test
7
Shift intensity
Figure 8: Comparison of ensembled models: ECE and Accuracy on both clean test data and shifted data on CIFAR-10 10 and ImageNet. For each intensity of shift, we show the results with a box plot summarizing the 25th, 50th, 75th quartiles across 19 types of shift CIFAR-10-C and 15 types of shift on ImageNet-C. The error bars indicate the min and max value across different shift types.
GIFAR-10
1.0
CIFAR-10
1.0
CIFAR-10
1.0
CIFAR-10
CIFAR-10
Acturacy
Accurocy
AcCuracy
0.5
0.8
0.8
0.4
0.4
0.
0.4
02
0.
82
18.00
0.25
0.50
0.7
1000
88.00
0.29
0.50 als
1000
8.00
0.25
050
0.75
1000
88.00
0.25
0.50
0
1,00
8.00
025
0.0 0.25 1000 Confidence
Confidence
Confidence
Conficence
Confidence
ImageNNtt
Imageket
ImageNNtt
ImageNett
Imagetet
1.0
1.0
1.0
1.0
1.0
0.5
0.5
uocuracy
Accuracy
AMC
4565
00
0.25
0.50
075
60
0.50
075
100
0.25
0.50
075
1,00
8.00
0.25
0.50
075
1,00
6.35
0.0
075
100
Figure 9: Reliability diagram of accuracy versus confidence of single model and ensemble model the clean test CIFAR-10 and ImageNet. The perfect calibrated model should be aligned with the diagonal dotted line (above under-confident, below is over-confident).
14

---

Under review as a conference paper at ICLR 2021
CI
CIFAR-10
ImageNet
ImageNet
1.00
Smgle
Single
Sinle
0.54
Sngle
trsemiie
troemtie
treembre
traemble
Centr decce
Gence
0.02
0.75
0.78
0.76
0.76
AduLS
Emmenbie
885
45
Adas
An treemblle
AMRS
Ennenblee
L5
Addl5
Emaunble
Figure 10: Comparing accuracy and confidence of the predicted class between single model and the corresponding ensemble model for each method.
0.30
Vanilla
AdaLS
0.25
CIFAR-10
Veriance
0.20
0.15
0.05
Test
Shift intensity
0.12
Verilla
AdaLS
ImageNet
AR-AdaLS
0.10
Variance
0.08
0.06
0.04
Test
3
Shift intensity
Figure 11: Variance on clean test and shifted data on CIFAR-10 and ImageNet. For each shift intensity, we show the results with box plot summarizing the 25th, 50th, 75th quartiles across 19 shift types on CIFARIO-C 10-C and 15 shift types on ImageNet-C. The error bars indicate the min and max value across different shift types.
15


Reviews:
[{"review_id": "wfenQgdNXcc", "reviewer": "AnonReviewer4", "summary": "", "questions": "", "limitations": "", "rating": 5, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "### Main contribution\n1. The author find that unrobust data may cause worse calibration.\n2. This work propose to encourage models to be adversarial robust to help calibration with a modified version of label smoothing: AR-AdaLS\n\n### Clarity\n1. The introduction of the backgrounds for uncertainty estimates is not very clear\n2. The proposal of AR-AdaLS is clear and reasonable\n3. The argument of \"that unrobust data may cause worse calibration\" is not fully demonstrated in Sec 3. \n\n### Questions\n1. In adversarial learning area, people usually refer \"robustness\" for models rather than data. We can say a model is robust or not. But it is a little bit confusing when you apply the term on data. Can the author provide references of other canonical works that also use this phrase?\n2. In Sec 3, the author stated they consider adversarial perturbation δ as a measure of robustness. Then why bother to divide the data into ten sets based on δ in your most import illustrations, which is Fig 1. What will happen if you directly present the relationship between δ and score? Does the 10th set represents the most robust data or does the 1st set do?\n3. What conclusion can we draw from the first row of Fig 1? How does it relate to the argument that \"the model is sensitive to small perturbations are more likely to have poorly calibrated predictions\"?\n4. I am familiar with the area of adversarial robustness and out-of-distribution detection, but not very confident about uncertainty estimates. I also did not find any specific introduction or helpful information in Sec 2. It seems to me that this area is not very popular and you can only compare your method with a few of other works, most of which are not even proposed for calibration in the first place (Label Smoothing/Mixup/Temperature Scaling). So can the author illustrate how the area of CALIBRATION differs from OOD detection? What is the most important evaluation metric for this problem? And maybe tell me which one is the most canonical work in this area so that I can refer to it as a comparison.\n\nI will raise my score if the author can fully address my questions.", "extra_scores": {}, "extra_text": {}}, {"review_id": "h9-gyachAjU", "reviewer": "AnonReviewer2", "summary": "", "questions": "", "limitations": "", "rating": 7, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "Summary: This paper proposes a new method (AR-AdaLS) for label smoothing to improve deep network calibration. In particular, the authors draw a connection between lack of calibration (overconfidence) and examples which are prone to adversarial attacks. They show that by generating smoothed targets based on the adversarial robustness of an example, they can further improve model calibration beyond traditional label smoothing.\n\nPros: The paper demonstrates a clear connection between per-example calibration error (in general, overconfidence) and a lack of adversarial robustness. \n\nMethod and results are clear and seem to be well situated in the literature.\n\nThey demonstrate the outperformance of their method relative to Label Smoothing in calibrating across several datasets, model architectures, and domain shifts. \n\nCons: I'm not an expert in this field, but the novelty of the adversarial robustness - overconfidence connection is not entirely clear to me. Muller et al. describe the behavior of label smoothing as necessarily reducing the difference between logits of two classes and thereby the confidence of the prediction, particularly for outliers. This would seem to be the same effect you describe, but perhaps less explicitly.\n\nThe methods used as baselines, label smoothing and temperature scaling, are attractive for their simplicity and ease of training. While you note that your model has the same latency at inference, it seems important to know how much extra computation is required in training.\n\nThe improvement of AR-AdaLS relative to vanilla ensemble (and ensemble vanilla relative to AR-AdaLS of ensemble) is meaningfully different for ImageNet. You mention this may be due to larger dataset/less overfitting but it would be useful to better understand this - how do other forms of regularization relate to AR-AdaLS?\n\nI find the discussion of Figure 5 somewhat unsatisfying - it's not entirely clear to me why averaging the predictions of 5 well-calibrated models results in a miscalibrated model\n\nWhat is sensitivity to hyperparameter R? \n\nMinor:\n\nComplementary, not complimentary\n\nIn related work: multiclasss", "extra_scores": {}, "extra_text": {}}, {"review_id": "LV8EMmAsEY6", "reviewer": "AnonReviewer3", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "\nThe author addressed my concerns. I’ll keep the score 6.\n\n=======================\n\nSummary:\n\nThe paper studied the relationship between adversarial robustness and calibration, then use the findings to improve label smoothing method. An adaptive label smoothing method (AR-AdaLS) is proposed to improve the calibration performance. Combining AR-AdaLS and deep ensemble can further improve the performance of deep ensemble method.\n\nStrength:\n\n1. The idea is novel and easy to understand. Experiments show the positive relationship between high adversarial robustness and better calibration. This work connects two field of studies.\n2. AR-AdaLS improves the performance of label smoothing. AR-AdaLS can also be used to improve deep ensemble.\n3. Improving calibration of deep neural networks is an important task.\n\nWeakness:\n\n1. Comparing the performances of LS and AR-AdaLS on CIFAR100 and CIFAR100-c, the improvements are actually not significant on CIFAR100, but AR-AdaLS is more computationally expensive. AR-AdaLS (on-the-fly) generates adversarial examples during training. This is very computationally expensive and will restrict the ability of the method to scale up to large dataset.\n2. AR-AdaLS does not perform better than deep ensemble, which is the state-of-the-art (claimed by the author).\n3. Three datasets are used in the paper but it seems that not all datasets are used to compare the performances of proposed model and other baselines.\n\n\nClarity and Correctness:\n\nThe paper is well written and easy to follow. The experiments look convincing. Boxplots comparison of ECE on CIFAR10 and ImageNet are provided.\n\nReproducibility:\n\nDetails of training and pseudocode is given but code is not available.\n\nQuestions:\n\nThe comparison of performances of ensemble models on CIFAR10 is provided but the comparisons on CIFAR100 and ImageNet are not. Why is that?\n\nConclusion:\n\nThe idea is novel and interesting but the empirical performance is not outstanding. Overall, I'm ok to accept the paper. I like the idea of using adversarial robustness to improve calibration. This paper reveals some interesting connections between the two things. \n", "extra_scores": {}, "extra_text": {}}, {"review_id": "141LpOkuuxm", "reviewer": "AnonReviewer1", "summary": "", "questions": "", "limitations": "", "rating": 2, "confidence": 5, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "The authors propose an extension to label smoothing, where the smoothing parameter is determined based on the miscalibration of the validation set. They then compare the performance of their method in terms of calibration under domain shift to a small set of baselines. \n\nI have 3 main concerns regarding missing baselines, missing experiments and only marginal benefits over state-of-the-art.\n\nMy first main concern is that the authors limit their comparison to rather dated baselines (2018 and older), when there has been a lot of active research in this field in the past year. In particular, I am disappointed that while the authors cite recent work on MixUp showing its benefits for calibration (Thulasidasan et al., NeurIPS 2019) they do not include MixUp as baseline (where also inputs are smoothed rather than labels only). Other notable recent work that should be included as baseline is Verified Uncertainty Calibration, Kumar et al., NeurIPS 2019, which has been shown to be superior to Temperature Scaling.  In addition, a baseline which also exploits links between calibration and adversarial, Stutz et al., ICML 2020, should be included as prior work.\nAnother aspect of the presented work is the exploration of constructing an ensemble of neural nets trained with label smoothing. In light of this, it is crucial to also compare the presented approach to Mix-n-Match, Zhang et al., ICML 2020, who present work on ensemble methods for uncertainty calibration using label smoothing. \n\n My other main concern is a whole set of missing experiments investigating calibration in truly OOD scenarios. Snook et al, on whose work this paper heavily builds, point out that in addition to the domain drift scenarios explored by the authors, it is crucial to investigate performance in truly OOD scenarios, where the test data is drawn from a distribution far a way from the training distribution. Since the model is per definition not able to make a correct prediction, in this scenario entropy is used to quantify model the quality of the predictive uncertainty. The authors should add these experiments for all datasets. \nIn this context, it would also interesting to investigate the quality of predictive uncertainty of OOD detection methods. While such comparisons are not always meaningful, a model strongly related to label smoothing/MixUp is Hendrycks et al., ICLR 2019, where a GAN is trained to learn OOD samples, where in MixUp inputs are smoothed to generate OOD samples. A comparison to this approach would also be interesting. \n\nFinally, even though important baselines as well as experiments for truly OOD scenarios are missing, benefits over a simple ensemble of vanillas remain unclear. For large-scale data (i.e. Imagenet), ensemble of Vanilla perform consistently better than the proposed method in terms of ECE under domain shift. For CIFAR-10, performance is comparable to deep ensembles and only for AR-AdaLS of Ensemble for CIFAR-10 there is a marginal improvement for a subset of perturbation strengths. I would have liked to see also evaluation of ECE under domain shift for CIFAR-100 and SVHN. Also missing is an evaluation of AR-AdaLS of Ensemble for Imagenet. Finally, results for non-image data, e.g. using a recurrent architecture (as in Snoek et al) are missing. \n\nOther concerns and open questions include: \nAlthough the authors explain the hyperparameters they use in the appendix, a detailed sensitivity analysis is missing to understand how sensitive the method is with respect to alpha and R. \n\nThe authors use „Ensemble of Vanilla“ as baseline - however, Snoek et al. have shown that it is actually deep ensembles that perform best; in addition to the ensemble effect they are also trained using adversarials. What is the performance of actual deep ensembles rather than ensemble of vanilla?\n\nA conceptual concern is that in the proposed approach the validation set becomes part of the training set since it is used during training to update epsilon and thus is not anymore the independent set that may be necessary to tune other hyperparameters/for early stopping; I wonder whether this data leak may lead to problems related to overfitting?\n\nPlease report a proper scoring rule in addition ECE (e.g. Brier score).\n\nA minor point ist that different colors are used for AR-AdaLS  and Ensemble of Vanilla in figure 3 and figure 4.\n\nUnfortunately the authors do not provide any code, which make reproducibility difficult. \n\n\n\n####post rebuttal####\nThe contribution of this paper is marginal only. The link between adversarial robustness and calibration has been explored previously: Snoek et al. NeurIPS 2019 have shown that adversarial training as part of deep ensembles leads to better calibration under domain shift. Unfortunately the authors do not compare their approach to these deep ensembles, but only an ensemble of differently initialised vanilla networks without adversarial training, which they call deep ensembles (section 5.1). Also in terms of label smoothing the contribution is marginal: in their rebuttal the authors show that MixUp training - a different implementation of label smoothing combined with input smoothing - has a better performance than their method (ECE of 1.8 MixUp vs 2.3 their method for CIFAR-100); they do show that further post-processing improves their method, but this is likely true for MixUp too (results not shown). For ImageNet results for MixUP are not shown, nor for calibration under domain shift where MixUp is likely to perform well too.\n Taken together, this suggests that the link between adversarial robustness and calibration is mainly a link between OOD samples and calibration: generating OOD samples with input smoothing in MixUp works very well compared to the proposed approach, as does adversarial training in deep ensembles (both of which was shown in prior work). In summary, the proposed approach lacks novelty and performs worse than baselines for complex datasets.\n\nLack of code during the reviewing phase means it is not possible to review reproducibility of results.", "extra_scores": {}, "extra_text": {}}]

Author rebuttal:

Meta-review: This paper improves calibration of neural networks by investing its connection to adversarial robustness. Two reviewers suggested acceptance, and two did rejection. As the authors and some reviewers highlighted, AC also agreed that the correlation between adversarial robustness and calibration is interesting to explore. However, as R1 pointed out, AC also thinks that the experimental results are not strong enough to meet the high standard of ICLR, e.g., Mixup often outperforms the proposed method (without further post-processing) and the proposed method does not outperform the deep ensemble (although deep ensemble is expensive and both method can be combined). Due to this, AC doubts whether adversarial robustness is indeed the best way to improve calibration (it can be useful though). Hence, AC recommends rejection.
Decision: Reject