Dataset Curation Beyond Accuracy
Authors: Johan Bjorck
Carla P Gomes
Venue:  (2021)
Keywords: crowd-sourcing
calibration
dataset
uncertainty

Abstract: Neural networks are known to be data-hungry, and collecting large labeled datasets is often a crucial step in deep learning deployment. Researchers have studied dataset aspects such as distributional shift and labeling cost, primarily using downstream prediction accuracy for evaluation. In sensitive real-world applications such as medicine and self-driving cars, not only is the accuracy important, but also the calibration -- the extent that model uncertainty reflects the actual correctness likelihood. It has recently been shown that modern neural networks are ill-calibrated. In this work, we take a complementary approach -- studying how dataset properties, rather than architecture, affects calibration. For the common issue of dataset imbalance, we show that calibration varies significantly among classes, even when common strategies to mitigate class imbalance are employed. We also study the effects of label quality, showing how label noise dramatically increases calibration error. Furthermore, poor calibration can come from small dataset sizes, which we motive via results on network expressivity. Our experiments demonstrate that dataset properties can significantly affect calibration and suggest that calibration should be measured during dataset curation.

Under review as a conference paper at ICLR 2021
DATASET CURATION BEYOND ACCURACY
Anonymous authors
Paper under double-blind review
ABSTRACT
Neural networks are known to be data-hungry, and collecting large labeled datasets often a crucial step in deep learning deployment. Researchers have studied dataset aspects such as distributional shift and labeling cost, primarily us- ing downstream prediction accuracy for evaluation. In sensitive real-world appli- cations such as medicine and self self-driving cars, not only is the accuracy important, but also the calibration the extent that model uncertainty reflects the actual cor- rectness likelihood. It has recently been shown that modern neural networks are ill-calibrated. this work, we take a complementary approach studying how dataset properties, rather than architecture, affects calibration. For the common issue of dataset imbalance, we show that calibration varies significantly among classes, even when common strategies to mitigate class imbalance employed. We also study the effects of label quality, showing how label noise dramatically increases calibration error. Furthermore, poor calibration can come from small dataset sizes, which we motive via results on network expressivity. Our experi- ments demonstrate that dataset properties can significantly affect calibration and suggest that calibration should be measured during dataset curation.
INTRODUCTION
Neural networks often require large amounts of labeled data to perform well, making data curation crucial but costly aspect of deployment. Thus, researchers have studied dataset properties such as distributional shift (Miller et al. 2020) and the bias in crowd-sourced computer vision datasets [Tsipras et al. 2020) among others. Often, the evaluation criteria in such studies downstream prediction accuracy. However, neural networks are increasingly deployed in sensitive real-world applications such as medicine (Caruana al.) 2015), self-driving cars (Bojarski et al., 2016), and scientific analysis (Attia et al.] 2020). where not only accuracy matters but also calibration. Cali- bration the extent to which model certainty reflects the actual correctness likelihood. Calibration can be important when costs of false positives and false negatives are asymmetric, e.g., for deadly disease with cheap treatment, doctors might initiate treatment when the probability of being sick exceeds 10%. Beyond simple classification, calibration can be important for beam-search in NLP (Ott et al. 2018) and algorithmic fairness (Pleiss et al. (017)). Calibration in machine learning has been studied by e.g Zadrozny & Elkan (2001); Nacini al. (2015). Niculescu-Mizzi Mizil & Caruanal (2055) have shown that small scale neural networks can yield well-calibrated predictions. However, has recently been observed by (Guo al. (017)) that modern neural networks are ill-calibrated, whereas the now primitive Lenet (LeCun et al. 1998) achieves good calibration.
In this work, we take a complementary approach; instead of focusing on network architecture, we study how calibration is influenced by dataset properties. We primarily focus on computer vision and perform extensive experiments across common benchmarks and more exotic datasets such as satel- lite images (the eurosat dataset (Helber et al. 2019)) and species detection (the iNaturalist dataset (Van Horn et al. 2018)). We consistently find that dataset properties can significantly affect calibra- tion, causing effects comparable to network architecture. For example, we consider the ubiquitous problem class imbalanced datasets, a common issue in practice (Van Horn et al. 2018 Krishna al. 2017; Thomee al. 2016). For such datasets, the miscalibration is not uniform but instead varies across the different classes. This problem persists even when common strategies to mitigate class imbalanced are employed. Another practical concern is generating high- high-quality labels via e.g. crowdsourcing (Karger et al.) 201T). We demonstrate how labeling quality affects calibration, with noisier labels resulting in worse calibration. Additionally, show that just the size of the dataset

---

Under review as a conference paper at ICLR 2021
cifar10
cifar100
callbration (eca)
0.6
0.6
0.4
0.3
0.2
0.0
0.0
eurosat
INaturalist
0.24
calibration
0.08
0.00
0
0
class
class
Figure 1: Calibration error for individual classes under class-imbalance. The classes are ordered from the most (left) the least (right) amount of samples. Fewer samples result larger calibration errors. Imbalance is injected in CIFAR 10/100 and eurosat randomly, removing any correlation with class-specific properties. We do modify Inaturalist, which already suffers from imbalance; thus classwise calibration is correlated with class-specific properties.
has strong effect on calibration. This also holds when one artificially increases the dataset size by data augmentation. We motivate our findings by considering geometry of the cross-entropy loss and utilizing recent results on network expressivity (Yun et al. 2019). If the dataset is suffi- ciently small compared to the number of parameters, we argue that the lack of minimizer for the cross-entropy loss biases the network to high confidence and poor calibration. Our results highlight an underappreciated aspect of calibration and suggest that for sensitive applications, one should measure calibration during dataset curation.
BACKGROUND
Calibration. Calibration has traditional place in machine learning (Zadozzyy & Elkan 2001
Naeini et al. 2015). Before the advent of modern deep learning. Niculescu- Mizil & Caruana (0055)
showed that neural networks yield well well-calibrated predictions for classification. However, Guo ctt al. (017)) showed that modern neural networks are ill-calibrated. Modern neural networks are modeled c.g. resnet (He al. 2016) densenets (Huang al. 2016). It is important note that accuracy and calibration do not necessarily follow each other, but can move independently modern neural networks ill-calibrated, but still yield excellent accuracy. Beyond image classification, the importance of calibration in NLP has further been studied by Ott et al. (2018) and its relationship to fairness by Pleiss al. (2017).
We let {x;} E Raxd, be dataset of n datapoints with d, features and take {y: be the labels. Following Guo al. (2017), we assume that neural network h outputs h(x,)) = (p., (.). where ÿ, is the predicted class and p. is the estimated probability that the prediction correct. For evaluating calibration, we divide the interval [0, 1] into M equally sized bins and assign predictions to bins based upon p. Within each bin B,, we define the accuracy as acc Bm) B Zie 1(;: = v.). Similarly, we define the confidence as conf Bm,) Bm Eie B.. P.. For a wall-calibrated model, we would expect the confidence and accuracy of each bin to be close to each other. Calibration
Metrics for Calibration.
be measured by their difference, evaluated the test The
2

---

Under review as a conference paper at ICLR 2021
resulting metric is known as the expected calibration error (Nacini et al. 2015), often abbreviated as ece. Mathematically. it is defined as follows:
M
Bml
ece
acc( conf(Bmm)
(1)
3 EXPERIMENTS
Experimental setup. We consider the following computer vision datasets. Cifar10 & Cifar100 (Krizhevsky & Hinton 2010) which contains 50,000 RGB images spanning ten or hundred classes balanced. Eurosat (Helber al. 2019), which is dataset of satellite images over continental Europe; there are ten balanced classes and 27,000 images in total. iNatu- ralist (Van Horn al. 2018) which dataset for species detection. We use the FGVC6 version (FGVC6 2019), compromising over 260,000 images and an imbalanced hierarchical class system compromising e-g., species and phylum. We perform classification at the "class" level, resulting in nine classes. Across all datasets, we use the same architecture, Resnet50 (He et al. 2016). We use hyperparameters from the original resnet paper (He al.) 2016): cross-entropy loss optimized with SGD using a learning rate 0.1 and decreased by factor 0.1 after 50 % and 75 % of the training, batch size of 128, weight decay of 0.0001, and momentum of 0.9. For the cifar/eurosa//inaturiisst networks are trained over 22/0//31 10 gradient steps, corresponding dataset. We use randomized cropping and random horizontal flipping for data augmentation, see Appendix A for data preprocessing. Experiments are repeated five times, with mean and standard deviations reported. Calibration error is measured expected calibration error(ece) as in eq. (T) using M 15 and evaluated on the test set.
respectively. Classes
160 epochs for each
Inbalanced dataset. A common problem in practice, not necessarily found in benchmark datasets, class imbalance (i.e., the number available samples varies between classes), see Van Horn al. Krishna et al. (2017): Thomee et al. (2016). Here we study how imbalanced datasets affect
2018:
cifar10
cifar100
0.27
callbraion (oco)
0.18
0.14
0.07
0.0
0.1
0.2
0.3
0.4
0.0
0.1
0.2
0.3
0.4
eurosat
INaturalist
0.27
calibration (ace)
0.10
0.18
0.00
0.00
0.0
0.1
0.2
0.3
noisy labels (fraction)
noisy labels (fraction)
Figure 2: Calibration error under label noise, simulated by randomly reassigning labels for a fraction of the training labels. Across datasets, label noise degrades network calibration. Thus, label noise from e.g. crowd sourcing can affect not only accuracy, but also calibration (Karger et al. 2011).
3

---

Under review as a conference paper at ICLR 2021
cfar10
cHtr1100
0.27
calbratin ener (600)
0.38
0.18
0.2
0.09
0.00
60
300
0.00
Figure 3: Calibration error under non-uniform label noise. We linearly increase label noise from 0 to 0.5 among classes, and sort them thereafter. Increased noise leads to worse calibration.
the calibration error for individual classes. Whereas the iNaturalist dataset is naturally imbalanced, the cifar and eurosat datasets are not. For these datasets, we simulate long-tailed class imbalance following Cao et al. (2019). We randomly reorder the classes from to and only keep a fraction of examples for class Given some desired ratio p between the class with the most and fewest samples, one picks such that a=11 p. Following Buda al. 2018), we also consider a step-imbalance, where half of the classes are downsampled by a factor p. We consider 100 done by e.g. Cao et al. (2019) and keep the test set balanced. For cifar/eurosat, randomly chose what classes to subsample to eliminate class specifcc properties. Since iNaturalist is already imbalanced, we keep
is, but note that the class-specific properties class-specific imbalance. After this procedure, we train the DNNs as normal and give the average calibration error for individual classes. The results are shown in Figure II Generally, classes with fewer examples have significantly higher ECE, showing how imbalance can have significant effect on model calibration. For the iNaturalist dataset, we have some outlier classes which likely due to class- class-specific effects, e.g., the class with the most labels might be unusually hard to calibrate.
correlated with
Methods for Imbalanced Datasets. As class imbalanced is a problem of practical importance, there is ample work on mitigating this issue. One common strategy is to sample the dataset un- evenly when generating mini-batches, attempting to obtain a roughly balanced dataset. One can both oversample minority classes (Buda et al. 2018) and undersample the majority class (Japkow- icz & Stephen 2002). Another strategy is instead to weight the objective function to give all classes
Table 1: Calibration error for various mitigation strategies used in imbalanced datasets. We give the calibration error for the class with the most/fewest labels (referred to as min/max), and the ratio of these two emrors. Two types of imbalance are considered, exponential and step. While improving in some cases, classwise imbalance remains even when mitigation strategies are used.
exp-inbalance
ci
cof
eurosat
method
min
max
ratio
min
max
ratio
min
max
ratio
original
0.12
0.48
4.24
0.34
0.61
1.82
0.05
0.3
7.06
sampling
0.16
0.62
3.99
0.39
0.66
1.68
0.05
0.22
4.85
weighted
0.1
0.35
3.67
0.22
0.4
1.79
0.08
0.14
1.75
label smooth
0.08
0.35
4.31
0.16
0.27
1.73
0.1
0.29
2.96
focal [Lin et al.
2
0.1
0.46
4.88
0.29
0.56
1.91
0.06
0.28
4.5
CB (Cu et al. 2019)
0.09
0.34
3.98
0.2
0.32
1.57
0.08
0.14
1.84
step-imbalance
cifar10
c00
eurosat
method
min
max
ratio
min
max
ratio
min
max
ratio
original
0.04
0.63
16.57
0.16
0.73
4.6
0.02
0.43
19.62
sampling
0.06
0.77
13.17
0.19
0.74
3.85
0.02
0.43
19.94
weighted
0.08
0.34
4.62
0.15
0.3
2.01
0.1
0.18
1.94
label smooth
0.08
0.47
5.58
0.12
0.48
3.95
0.09
0.33
3.47
focal (Lin et al.
2017)
0.03
0.56
20.54
0.14
0.66
4.77
0.03
0.37
16.72
CB (Cu et 2019
0.07
0.41
6.24
0.15
0.3
2.04
0.08
0.17
2.22
4

---

Under review as a conference paper at ICLR 2021
cifar10
cifar100
0.40
0.16
calibration (oco)
0.32
0.12
0.24
0.04
0.08
1.0
0.8
0.6
0.4
0.2
0.1
1.0
0.8
0.6
0.4
0.2
0.1
iNaturalist
0.15
calibration (oce)
0.06
0.10
0.04
0.05
0.02
I
0.00
1.0
0.8
0.2
0.00
1.0
dataset size (fraction)
dataset size (fraction)
Figure 4: Calibration error under different dataset sizes. We subsample the datasets, and give the size as a fractions the original size. Across all tasks, smaller datasets consistenly yield poorer calibration, highlighting how dataset size influences not only accuracy but also calibration.
approximately the same weight in the objective function. A common strategy is weight classes in- versely proportional to their frequency (Wang et al. 2017). Recently, Cui et al. (2019) has proposed reweight based upon the "effective" number of samples, which defined per mathematical formula. We here investigate the calibration issues of an imbalanced dataset persist when us- ing such mitigation strategies. Thus, consider the standard cross-entropy (original), resampling inversely proportional to the frequency (sampling). reweighting inversely proportional to the fre- quency (weighted), the weighting scheme of Cfi et al. (2019) (CB), and the focal loss Lin et al.) (017) (focal). Additionally we consider label smoothing (Szegedy et al. 2016) (label smooth). We construct imbalanced datasets in the previous section. Due limited computational resources, only consider the three smallest datasets for these experiments. Calibration errors are given in Table T and we give largest and smallest calibration error among classes the average ratio of the two. We that issues of imbalanced calibration errors, while sometimes improving, still persist. Standard deviations are given Table 2 in Appendix(A)
Label Quality. When collecting labels, for example via crowdsourcing, common issue is label quality (Patterson & Hays 2012 Su et al. 2012] Kallison-Buucee & Dredze 2010). For example, workers might have poor incentives to perform well or lack the necessary skills for quality labeling. To study the effects of potentially mislabeled data, we artificially inject symmetric noise into the training set. This done selecting random subset of the training corresponding some fixed fraction, and then shuffling the labels of this set. This setup follows conventions label noise literature (Patrini al. 2017 Han al. 2018). Given these noisy labels, we train the networks and evaluate them on the original test-set (which has no noise). We consider five levels of label noise in increments of 0.1, starting 0.0. The resulting calibration errors for various noise levels are given in Figure 2 where we see that label noise increases the calibration error across all datasets. Addi- tionally, we consider the effects of non-uniform noise, studied by e.g. Crammer et al. (2006). For class we linearly increase noise level p; from 0.0 to 0.5. Classes are randomly ordered. For each image with original class i, with probability p: we assign it to a new random class. The reassignment probability class is proportional p.. Results from worse calibration. This underscores how label quality control e.g. crowdsourcing (Su et al, 2012) can be not only important for accuracy, but also for calibration of downstream models.
given in Figure 3] where noisier classes suffer
5

---

Under review as a conference paper at ICLR 2021
cifar10
cifar100
0.18
calibration errrr
0.06
0.12
0.03
0.06
original
no flip
no crop
no crop/flip
original
no flip
crap no crop/flip
INaturalist
calibration (oco)
0.09
0.04
0.02
0.03
0.00
0.00
original
crop/fipp
original
crop/flip
Figure 5: Calibration error under combinations of data augmentations. Following He et al. (2016). we consider randomized cropping and flipping. Removing these components, often used to artifi- cially enlarge the dataset, increases the calibration error.
Dataset size. The perhaps most common concern for collecting data is the dataset size, and model accuracy typically grows with this size (Hestness et al.] (017). Crowdsourcing labels and bound- ing boxes for images is common practice, with many researchers investigating strategies reduce needed queries (Su et al. 2012) In practice, dataset size can be limited by costs of labeling, but also by obtaining the actual data (Suram et al.) 2017). Motivated by this, we study the effect of dataset size upon the calibration error. We simply subsample the training sets of the datasets uniformly at random and thereafter train on them, comparing different sizes of the resulting dataset. We consider subsampled sizes, measured in fractions of the original size, from 1.0 to 0.2 in increments of 0.2, subsampled. The results of these experiments are given in Figure 4 We see that smaller datasets have substantially larger calibration errors, demonstrating the dramatic effect that dataset size can have not only on accuracy, but also on calibration error.
and also consider 0.1. The test set
Beyond actual dataset size, it common artificially increase the size of the dataset by augmenting it, e.g., randomly crop- ping the images or slightly shifting the color balance (Cubuk al.] 2018). We have seen that the size of the dataset influences calibra- tion, and now consider the effect of increasing the effective dataset size via augmentations. We use both randomized cropping and horizontal flipping for our training and consider removing these components while keeping other training parameters fixed. The outcome of this exper- iment is shown Figure 5 and we see that removing data augmentations significantly in- creases the calibration error. Viewing data aug- mentation strategy of extending the training set, we again see how smaller training sets in- crease the calibration error, just as in Figure 4
Augmentations.
0.3
calibration error (ece)
1.0
0.2
0.2
0.1
0.0
0.5
1.0
fraction training
Figure 6: Calibration enror of NLP task during training for different dataset sizes. The dataset is subsampled, and give the relative size.

---

Under review as a conference paper at ICLR 2021
NLP. While primarily focus on computer vision, we here consider experiments in NLP. Language models often generate text via beam- seam-,,arch, and has been observed that calibration is important for this process (Ott et al. 2018). Here we investigate the effect of dataset size on calibration in NLP. We considering translation of the IWSLT" 14 German to English dataset (Cettolo et al. 2014) using transformer architectures (Vaswani al.] 2017) with code and default hyperparameters from the publicly available fairseq codebase (Ou et al., 2019).. As before, we simply subsample the training set uniformly at random for variable sizes and train the transformer with all other training parameters fixed. Figure [ shows how the mean calibration error, with standard deviations as error bars, varies during training. Again we see how the dataset size influences the calibration. It is natural to guess that there might be word-level calibration issues too, e.g., rare words might have worse calibration.
4 THEORETICAL MOTIVATION
Figure 4 and Figure 5 show that the size of the dataset affects calibration, with smaller datasets resulting in worse calibration. To explain this, let us consider the cross-entropy loss. We will let bi denote the logit for image and class j. Furthermore, let o₁ be the index of the correct class for image i. The soft-max cross-entropy loss function is then defined as
f= <=Σes 4 - - log exp(lia.) -E
-
exp(la)) Σ C(4e (eer + log Σ exp((+))
(2)
We note that this loss function decreases monotonically as the logit 4e, increases. This implies that there is no global minimizer, but instead that if the other logits are fixed, we have & + 0 as lic, -> oc. The logit tending to infinity implies that the confidence of the prediction tends to 1. The lack of minimizer for soft-max cross-entropy is in stark contrast with e.g. label smoothing, which penalizes large confidence, see Figure Let us imagine that the network has infinite capacity. If we optimize for a sufficient amount of time, we would expect to tend to zero, which implies that the logits tend to infinity. This corresponds to the confidence on the training set tending to 100%. which most likely implies overconfidence and poor calibration. We can formalize this observation, but we first need state some assumptions.
Assumption Let {2,} E RRnd be a dataset of n datapoints with dy features each. Let {u:} E (0. 1)axe be a one-hot label encoding that assigns each image one out of e classes, where e is constant. We assume that all datapoints {æi) are distinct, iee. T: F Fi, vi Vi + j.
Under such assumptions, recent results in network expressivity say that one can essentially memo- rize a training set (Yun et al. 2019) if the width is at least of order (Vvn). In an idealized setting, where we optimize the function without computational considerations, such expressivity means that the loss function can be optimized towards its minimizer O. This means train set confidence growing to 100%, likely translating to poor calibration. We formalize this line of argument in Theorem [
Theorem Let Assumption | hold. Let f be a Relu networks with four or more layers and with width at least ((yn and parameters w. Let be equal to the loss function in eq. 2 Then (i) min. (w)) no global minima; (ii) the confidence tends to 1 as l + 0.
The formal proof is given in Appendix we here provide some intuition. To prove (i) and (i))
suffices that the network can fit the training set with 1.0 accuracy. This is typically the condition in practice (Zhang et al., (016). and whereas we consider an idealized argument without computation costs, the conclusions agree with our experimental results. For the sake of contradiction, let us assume that we are in a global minima with parameters w and 100% accuracy. Now consider i when we scale the final layer by (1 + o:) for > 0. The network output is then (1 a)ly.. and E, is - llog ( (exp((1 + ax)ii,)) )/ E, exp((1 + a))4))) = log(1 + Ejee exp ((1 + a))(k
)). The fact that we have perfect train accuracy means that (4g - 4c,) < OVj # q. Thus, the loss must shrink, as Σjt, exp ((1 + a)44, - 4cc))) decreases with and as log is monotone. By contradiction, we are not in such global minima. The results Yun et al. 2019) say that we can always find weights which achieves perfect accuracy using ((vn) parameters, and thus that there are no issues with fitting that dataset that prevents the loss from tending to O. Thus, if the dataset is small compared to the number of parameters, we expect overconfidence and poor calibration. This conclusion agrees with observations of Guo et al. (2017), who show that depth and width increases miscalibration.
7

---

Under review as a conference paper at ICLR 2021
5 RELATED WORK
There is much recent work on how datasets influence the behavior of neural networks. Tsipras al. (2020) shows how the process used to collect labels for imagenet can introduce bias into the resulting dataset. Miller et al. (2020) studies how the shift between different datasets can influence the performance question and answering systems. |Recht et al.] (2019) construct new test sets for imagenet and cifar10, and observe differences in generalization compared to the original test sets, Imbalanced datasets is common issue when applying machine learning in practice (Van Horn jet al. 20188 Krishna al. 2017 Thomee al., 2016), and researcher often describe the "heavy- tail" of class labels (Cui et al. 2019). Traditional work on class imbalance includes Japkowicz & Stephen 2002) which investigates different sampling strategies, applicable most machine learning models. For models of empirical risk minimization, one can instead reweight samples. A relatively recent reweighting scheme is proposed by Cui et al. (20T9), where one uses the effective number of samples, which can be calculated from a simple formula.
For generating datasets, common strategy is to employ crowdsoureing, where one lets ordinary people assign labels in a large-scale automated fashion, commonly via Amazon's Mechanical Turk system (Keith et al. 2017). Typical applications of crowdsourcing include analyzing images and providing bounding boxes (Patterson & Hays 2012 al. (012)) providing linguistic annotations for natural language (Callison-Buree
Dredze (210)), or evaluating the relevance of search engines results (Alonso) 2013). Another application is machine learning debugging Ribeiro et al. (2016). The idea of cliciting and aggregating crowdsoureed labels efficiently has inspired much algorithmic work (Khetan & Oh, 2016 Zhang et al. 2014). Common issues include finding tasks that result in high-quality labels, dealing with inconsistent labels (Karger et al. 2011 heterogenous workers (Ho et al. 2013).
Zheng et al. 2017) and
Calibration in machine learning has been stud- ied for a long time (Zadrozny & Elkan 2001 Naeini et al. 2015) due to its practical impli- cations. For neural networks, Caruana al. (2015) demonstrated that shallow neural net- works can yield well-calibrated predictions on classification tasks. In contrast, Guo et al. (2017) show how modern neural networks are ill-calibrated, with width and depth resulting in worse calibration scores, and investigate mit- igation strategies. Neural network calibration has implications in NLP (Ott et al. 2018),, fair- ness (Pleiss et al., 2017) and reinforcement learning (Kuleshov et al. 2018). For applica- tions such medicine (Miner et al. 2020). me- teorology (Ren et al.] 2015) and autonomous vehicles (Bojarski al. (0166) it can be impor- tant for performance. Reliable uncertainty esti- mates also allow one to integrate DNNS with other probabilistic models, incorporating e.g. camera information (Kendall & Cipolla 2015).
2.4
cress entropy
label smoothing
1.6
loss
0.8
0.0
0
logit
Figure 7: The softmax-cross entropy and label smoothing as a function of the logit of the cor- rect class (other logits are zero). Cross-entropy decreases monotonically, resulting in large logits after optimization.
CONCLUSIONS
We have investigated the effects that datasets can have on network calibration. By generating label noise and class imbalance synthetically, we show how calibration error increases with label noise and few samples. We also study how calibration changes with dataset size. Our work points towards the importance of high-quality dataset curation for generating well-calibrated predictions, and high- light issues that are relevant in high-stakes applications such as autonomous vehicles and medical applications. These calibration issues can potentially be mitigated both at dataset curation time and training time; we defer such studies to future work. A practical takeaway from this work is that for sensitive applications, one should evaluate calibration when collecting datasets.
8

---

Under review as a conference paper at ICLR 2021
REFERENCES
Omar Alonso. Implementing crowdsourcing--ased relevance experimentation: an industrial per- spective. Information retrieval, 16(2):011-120, 2013.
Peter M Attia, Aditya Grover, Norman Jin, Kristen Severson, Todor Markov, Yang-Hung, Liao, Michael H Chen, Bryan Cheong, Nicholas Perkins, Zi Yang. et al. Closed-loop optimization of fast-charging protocols for batteries with machine learning. Nature, 578(7795):397-402 2020.
Mariusz Bojarski, Davide Del Testa, Daniel Dworakowski, Bernhard Firner, Beat Flepp, Prasoon Goyal, Lawrence D Jackel, Mathew Monfort, Urs Muller, Jiakai Zhang, et al. End to end learning for self-driving cars. arXiv preprint arXiv: 1604.03316 2016.
Mateusz Buda, Atsuto Maki, and Maciej A Mazurowski. A systematic study of the class imbalance problem convolutional neural networks. Neural Nerworks, 016:249-259 2018.
Chris Callison- Burch and Mark Dredze. Creating speech and language data with amazon's mechan- ical turk. In Proceedings of the NAACL IILT 2010 workshop creating speech and language data with Amazon's Mechanical Turk, pp. 1-12, 2010.
Kaidi Cao, Colin Wei, Adrien Gaidon, Nikos Arechiga, and Tengyu Ma. Learning imbalanced datasets with aabel-distrbuttin-aaaal margin loss. In Advances in Neural Information Processing Systems, pp. 1567-1578, 2019.
Rich Caruana, Yin Lou, Johannes Gehrke, Paul Koch, Mare Sturm, and Noemie Elhadad. Intel- ligible models for healthcare: Predicting pneumonia risk and hospital 30-day readmission. In Proceedings of the 2/th ACM SIGKDD international conference on knowledge discovery and data mining, pp. 1721-1730, 2015.
Mauro Cettolo, Jan Nichues, Sebastian Stüker, Luisa Bentivogli, and Marcello Federico. Report on the 11th iwslt evaluation campaign, iwslt 2014. In Proceedings of the International Workshop Spoken Language Translation, Hanoi, Vietnam, volume 57, 2014.
Koby Crammer, Michael Kearns, and Jennifer Wortman. Learning from data of variable quality. In Advances in Neural Information Processing Systems, pp. 219-226, 2006.
Ekin D Cubuk, Barret Zoph, Dandelion Mane, Vijay Vasudevan, and Quoc Le. Autoaugment: Learning augmentation policies from data. arXiv preprint arXiv: 1805.00000, 2018.
Yin Cui, Menglin Jia, Tsung- Yi Lin, Yang Song, and Serge Belongie. Class-balanced loss based on effective number of samples. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 1168-9277,, 2019.
FGVC6. iNaturalist 2019 FGVC6, 2019.
inaturalist 2019- fgvc6
Chuan Guo, Geoff Pleiss, Yu Sun, and Kilian Q Weinberger. On calibration of modern neural networks. arXiv preprint httl::/70.045555 2017.
Bo Han, Jiangchao Yao, Gang Niu, Mingyuan Zhou, Ivor Tsang, Ya Zhang, and Masashi Sugiyama. Masking: A new perspective of noisy supervision. In Advances in Neural Information Processing Systems, pp. 5836-5846, 2018
Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recog- nition. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp- 770-778, 2016.
Patrick Helber, Benjamin Bischke, Andreas Dengel, and Damian Borth. Eurosat: A novel dataset and deep learning benchmark for land use and land cover classification. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, 12(7):2217-225 2019.
Joel Hestness, Sharan Narang, Newsha Ardalani. Gregory Diamos, Heewoo Jun, Hassan Kianinejad, Md Patwary, Mostofa Ali, Yang Yang, and Yanqi Zhou. Deep learning scaling is predictable, empirically. arXiv preprint
2017.

---

Under review as a conference paper at ICLR 2021
Chien-Ju Ho, Shahin Jabbari, and Jennifer Wortman Vaughan. Adaptive task assignment for crowd-
sourced classification. In International Conference on Machine Learning, pp. 534-55444440 542, 2013.
Gao Huang, Zhuang Liu, and Kilian Q Weinberger. Densely connected convolutional networks. corr abs//608.0000 06993 (2016). arXiv preprint arXiv: 06993, 2016.
Nathalic Japkowicz and Shaju Stephen. The class imbalance problem: A systematic study. Intelli- gent data analysis, 6(5):429-444) 44942994444400 2002.
David Karger, Sewoong Oh, and Devavrat Shah. Iterative learning for reliable crowdsourcing systems. In Advances in neural information processing systems, pp. 1953-1961, 2011.
Melissa G Keith, Louis Tay, and Peter D Harms. Systems perspective of amazon mechanical turk for
organizational research: Review and recommendations. Frontiers in Psychology, 8:1359, 2017.
Alex Kendall and Roberto Cipolla. Modelling uncertainty in deep learning for camera relocalization. сorr abs//150900535 .09999 (2015), 2015.
Ashish Khetan and Sewoong Oh. Achieving budget-optimality with adaptive schemes in crowd- sourcing. In Advances Neural Information Processing Systems, pp. 4844 2016.
Ranjay Krishna, Yuke Zhu, Oliver Groth, Justin Johnson, Kenji Hata, Joshua Kravitz, Stephanie Chen, Yannis Kalantidis, Li-Jia Li, David A Shamma, et al. Visual genome: Connecting lan- guage and vision using crowdsourced dense image annotations. International journal of computer vision, 123(1):32-7,, 2017.
Alex Krizhevsky and Geoff Hinton. Convolutional deep belief networks on cifar-10. Unpublished manuscript, 40,7):1-9, 2010.
Volodymyr Kuleshov, Nathan Fenner, and Stefano Ermon, Accurate uncertainties for deep learning using calibrated regression. arXiv preprint arXiv:1807.00263 2018.
Yann LeCun, Léon Bottou, Yoshua Bengio, and Patrick Haffner. Gradient-based learning applied to document recognition. Proceedings of the IEEE, 86(11):2288-2324, 1998.
Tsung-Yi Yi Lin, Priya Goyal, Ross Girshick, Kaiming He, and Piotr Dollár. Focal loss for dense object detection. In Proceedings of the IEEE international conference on computer vision, pp. 2980-2988, 2017.
John Miller, Karl Krauth, Benjamin Recht, and Ludwig Schmidt. The effect of natural distribution shift question answering models. arXiv preprint 2rXiv:2004.14444, 2020.
Adam S Miner, Albert Haque, Jason A Fries, Scott L Fleming, Denise E Wilfley, Terence Wilson, Arnold Milstein, Dan Jurafsky, Bruce Arnow, W Stewart Agras, et al. Assessing the accuracy of automatic speech recognition for psychotherapy. npj Digital Medicine, 3(1):1-8, 2020.
Mahdi Pakdaman Naeini, Gregory F Cooper, and Milos Hauskrecht. Obtaining well calibrated probabilities using bayesian binning. In Proceedings of the... AAAI Conference on Artificial Intel- ligence. AAAI Conference Artificial Intelligence, volume 2015, pp. 2901. NIH Public Access, 2015.
Alexandru Niculescu-Mizil and Rich Caruana. Predicting good probabilities with supervised learn- ing. In Proceedings of the 22nd international conference on Machine learning, pp. 625-632 025- 2005.
Myle Ott, Michael Auli, David Grangier, and Mare' Aurelio Ranzato. Analyzing uncertainty in neural machine translation. arXiv preprint /803000000 2018.
Myle Ott, Sergey Edunov, Alexci Bacvski, Angela Fan, Sam Gross, Nathan Ng. David Grangier, and Michael Auli. fairseq: A fast, extensible toolkit for sequence modeling. In Proceedings of NAACL-HLT 2019: Demonstrations, 2019.
Giorgio Patrini, Alessandro Rozza, Aditya Krishna Menon, Richard Nock, and Lizhen Qu. Making deep neural networks robust to label noise: A loss correction approach. In Proceedings of the IEEE Conference Computer Vision and Pattern Recognition, pp. 1944-1952, 2017.
10

---

Under review as a conference paper at ICLR 2021
Genevieve Patterson and James Hays. Sun attribute database: Discovering, annotating, and recog- nizing scene attributes. In 2012 IEEE Conference on Computer Vision and Pattern Recognition, pp. 2751-2758. IEEE, 2012.
Geoff Pleiss, Manish Raghavan, Felix Wu, Jon Kleinberg, and Kilian Q Weinberger. On fairness and calibration. In Advances in Neural Information Processing Systems, pp- 5680-5689, 2017.
Benjamin Recht, Rebecca Roelofs, Ludwig Schmidt, and Vaishaal Shankar. Do imagenet classifiers generalize to imagenet? arXiv preprint arXiv:1902.10S100 2019.
Ye Ren, PN Suganthan, and N Srikanth. Ensemble methods for wind and solar power forecasting-4 state-of-the-art review. Renewable and Sustainable Energy Reviews, 50:82-91, 2015.
Marco Tulio Ribeiro, Sameer Singh, and Carlos Guestrin. " why should i trust you?" explaining the predictions of any classifier. In Proceedings of the 22nd ACM SIGKDD international conference on knowledge discovery and data mining, pp. 1135-1144, 2016.
Hao Su, Jia Deng, and Li Fei-Fei. Crowdsourcing annotations for visual object detection. In Work- shops al the Twenty-Sixth AAAI Conference on Artificial Intelligence, 2012.
Santosh Suram, Yexiang Xue, Junwen Bai, Ronan Le Bras, Brendan Rappazzo, Richard Bernstein, Johan Bjorck, Lan Zhou, R Bruce van Dover, Carla P Gomes, et al. Automated phase mapping with agilefd and its application to light absorber discovery in the v-mn-nb oxide system. ACS combinatorial science, 19(1):37-46, 2017.
Christian Szegedy, Vincent Vanhoucke, Sergey loffe, Jon Shlens, and Zbigniew Wojna. Rethink- ing the inception architecture for computer vision. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 2818 2818-2826, 2016.
Bart Thomee, David Shamma, Gerald Friedland, Benjamin Elizalde, Karl Ni, Douglas Poland, Damian Borth, and Li-Jia Li. 10cc100m: The new data in multimedia research. Communications of the ACM, 59(2):44-73, 2016.
Dimitris Tsipras, Shibani Santurkar, Logan Engstrom, Andrew Ilyas, and Aleksander Madry. From imagenet to image classification: Contextualizing progress on benchmarks. arXiv preprint MrXiv:2005.11295 2020.
Grant Van Hor, Oisin Mac Aodha, Yang Song, Yin Cui, Chen Sun, Alex Shepard, Hartwig Adam, Pietro Perona, and Serge Belongie. The inaturalist species classification and detection dataset. In Proceedings of the IEEE conference computer vision and pultern recognition, pp. 8769-8778, 2018.
Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Lukasz Kaiser, and Illia Polosukhin. Attention is all you need. In Advances in neural information processing systems, pp. 5998-6008, 2017.
Yu-Xiong Wang, Deva Ramanan, and Martial Hebert. Learning to model the tail. In Advances in Neural Information Processing Systems, pp. 7029-7039, 2017.
Chulhee Yun, Suvrit Sra, and Ali Jadbabaie. Small relu networks are powerful memorizers: tight analysis of memorization capacity. In Advances in Neural Information Processing Systems, pp. 15558-15569, 2019.
Bianca Zadrozny and Charles Elkan. Obtaining calibrated probability estimates from decision trees and naive bayesian classifiers. In Icml, volume 1, pp. 609-616. Citeseer, 2001.
Chiyuan Zhang, Samy Bengio, Moritz Hardt, Benjamin Recht, and Oriol Vinyals. Understanding deep learning requires rethinking generalization. arXiv preprint arXiv:161) (0l:::6/1..350 1.03530, 2016.
Yuchen Zhang. Xi Chen, Dengyong Zhou, and Michael Jordan. Spectral methods meet em: A provably optimal algorithm for crowdsourcing. In Advances in neural information processing systems, pp. 1260-1268. 2014.
Yudian Zheng, Guoliang Li, Yuanbing Li, Caihua Shan, and Reynold Cheng. Truth inference in crowdsourcing: Is the problem solved? Proceedings of the VLDB Endowment, 10(5):541-552, 2017.
11

---

Under review as a conference paper at ICLR 2021
Table 2: Standard deviations corresponding Table
We round two digits.
cifar10
cifar100
eurosat
exp
min
max
ratio
min
max
ratio
min
max
ratio
original
0.02
0.03
0.84
0.02
0.02
0.13
0.01
0.04
2.32
sampling
0.02
0.04
0.83
0.01
0.02
0.07
0.01
0.07
1.71
weighted
0.01
0.01
0.44
0.03
0.06
0.14
0.01
0.02
0.34
label smooth
0.01
0.04
1.06
0.0
0.01
0.03
0.0
0.05
0.48
focal (Lin 2017)
0.02
0.03
0.99
0.03
0.01
0.13
0.01
0.05
1.3
CB (Cui et al. 2019)
0.01
0.04
0.61
0.01
0.04
0.15
0.01
0.02
0.32
cifar10
cifar100
eurosat
step
min
ratio
min
ratio
min
ratio
original
0.01
0.02
4.37
0.02
0.01
0.56
0.01
0.07
5.53
sampling
0.01
0.03
2.82
0.02
0.01
0.38
0.0
0.05
5.7
weighted
0.01
0.05
1.18
0.01
0.04
0.39
0.03
0.03
0.98
label smooth
0.0
0.04
0.48
0.0
0.02
0.19
0.0
0.02
0.26
focal (Lin 2017
0.01
0.05
4.74
0.01
0.02
0.48
0.01
0.04
6.33
CB (Cui et al. 2019)
0.01
0.07
1.75
0.01
0.06
0.49
0.01
0.04
0.59
A APPENDIX
Data Preprocessing. For the cifarl0 and cifar datasets, we use the original dataset size of 32- by-32 pixels. Cropping is performed by first padding with 4 pixels on each side, and thereafter performing random 32-by-32 crop. For the eurosat dataset, the images are subsampled to 32-by- 32 pixels. Since the eurosat dataset does have a dedicated train/test split, we split ourselves, using fixed random tenth of the dataset for testing across all experiments. Random cropping is performed as for cifar. For the inaturalist dataset, the images are resized into 64-by-64 images, and we pad by 8 pixels on each side before extracting random 64-by-64 crop.
Table 3: Hyper-parameters for computer vision.
Parameter
Value
init. learning rate
0.1
learning rate decay per step
0.1
decay after
(50%, 75%
SGD momentum batch size
0.9
128
horizontal flipping
True
cropping
True
weight decay
0.0001
loss
cross-entropy
Table 4: Hyper-parameter used for Transformers "transformer-iwss--deeea of Fairseq (Ott al. 2019).
Vaswani et al. 2017). The architecture is
Parameter
Value
learning rate
0,0005
0.9, 0.98
Calam batch size
0.00000001
32
label smoothing
0.1
dropout probability
0.3
max-tokens
4096
12

---

Under review as a conference paper at ICLR 2021
cifar10
cifar100
eurosat
sampling
0.5
0.4
0.3
0.1
weighted
0.4
0.4
0.54
class
Class
class
0.9
label smooth
0.4
0.4
0.3
class
class
class
0.5
0.4
focal
0.4
0.3
300
0.8
Class
class
class
0.4
CB
0.14
0.2
100
Figure 8: Classwise calibration error for all methods for exponential-inballnced datasets. Class shuffling uses the same seed between methods.
13

---

Under review as a conference paper at ICLR 2021
cifar10
cifar100
eurosat
weighted sampling
0.6
class
claos
claos
0.0
label smooth
0.44
0.3
100
focal
0.2
class
cloos
cloos
0.5
0.30
0.4
0.27
0.5
0.2
0.3
0.59
0.
0.00
Figure 9: Classwise calibration error for all methods for step-ittableee impallaeade datasets. Class shuffling uses the same seed between methods.
14

---

Under review as a conference paper at ICLR 2021
B PROOF
The results of Yun et al. (2019) mean that under Assumptions there exists a four-layer Relu network with (((w)) parameters that can memorize the dataset, perfectly matching the one-hot encoding of the labels. For details, we refer to Yun et al. (2019).
Proof Sketch for Theorem 1 We start with part (i). Assume for contradiction that we in a global minima with loss tn and perfect accuracy. Now. consider & when we scale the final layer by (1 +a) for a > The network output is then (1 + o.)kkj. and & is log [exp((1 + aa, / Σj exp((1 +
a)4y)) log(1 Ejसi exp (11+ - lec.)) )) The fact that have perfect train accuracy means that (lij 4c,) < 0Vj # G. Thus the loss must shrink, as Σf exp ((1 + (1, l2c.) decreases with a and as log is monotone. This contradicts us being in such a global minima. we do not have perfect accuracy, there must be some example & such that lg must be larger than log (since there another class with larger logit). Results Yun et al. (2019) says that we can find network with perfect accuracy and thereafter scale the weight to less than Tog (ii). The confidence for a single exp(hee, example is Σ, exp(ly) = exp( We have noted how l - 0, since = ΣS 4 we have 4 - 0.
This implies that exp(-(i + 1 which means that the confidence approaches 1.
15


Reviews:
[{"review_id": "aVVxVfb3gvn", "reviewer": "AnonReviewer3", "summary": "", "questions": "", "limitations": "", "rating": 4, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "\tThis paper discussed how data properties (e.g., label noise, label imbalance, data size) affects calibration error. The author designed experiments on varying computer vision datasets (i.e., cifar10, cifar100, eurosat and iNaturalist) qualitatively: 1) calibration error for various individual classes under class-imbalance situation; 2) calibration error for different scale of label noise; 3) calibration error under non-uniform noise; 4) calibration error under various scale of dataset size; 5) Calibration error under different combinations of data augmentations. The experimental results show that poor calibration performance accompanies with large noisy label rate, large imbalance ratio and small dataset size. For the reason of small dataset size causing poor calibration error, this paper provided the theoretical proof. \n\t\n\tAdvantages:\n\t\t○ The idea of considering a softmax-cross entropy logit loss to help explain how data size affect the calibration error is interesting.\n\t\n\tMajor concerns:\n\t\t○ Organization should be improved. In particular, the factors that affect the calibration error should be listed and well described in a separate section (e.g., Intro -> Background -> (affected data properties) -> experiments), and the theoretical motivation could also be integrated in such section rather than put it after experiments. \n\t\t○ The novelty and practicability of this paper is limited, since this paper only tells people that low label quality and small data size would arise calibration error, the paper analyzed the factors qualitatively but not quantitatively. In the future research, the researcher still hard to justify how much calibration error the current dataset whould bring or can't tell whether the current the current classifier whould be robust enough to defense the calibration bring by the current set. An example is: \n\t\t[1] \"Robustness of classifiers: from adversarial to random noise.\" Fawz et al. NIPS2016. This paper analyzed the robustness of classifiers quantitatively with considering adversarial and random noise. \n\tMinor comments: \n\t\t○ Table 1, \"exp-inbalance\" -> \"exp-imbalance\"\n\t\t○ Should the captions of Figure 2 and Figure 3 be changed?\n\t\t○ Assumption 1, \"x_i != x_i\" -> \"x_i != x_j\"\n\t\t○ Equation 2, \"-sum_i(A+B)\" -> \"sum_i(A-B)\".\n\t\t○ There are many typos in this paper, should go over the paper again and correct these small mistakes.\n", "extra_scores": {}, "extra_text": {}}, {"review_id": "X9QI5g_7BqY", "reviewer": "AnonReviewer1", "summary": "", "questions": "", "limitations": "", "rating": 4, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "The paper is an empirical study looking at how different dataset properties affect model calibration in the context of vision tasks. All experiments use a specific well-known vision model (ResNet 50). \n\nIn particular, the dataset properties that are investigated are:\n\n- Balanced/Unbalanced classes.\n- Label quality.\n- Dataset size.\n- Augmentations.\n- NLP.\n\nI briefly present the main conclusions below.\n\n- Balance in classes. Often times some classes have way more datapoints than others. The authors look at four datasets (Cifar 10, Cifar 100, Eurosat, iNaturalist). The last one's classes are unbalanced, whereas the first three require some sampling method to (artificially) make them unbalanced (note in this case by design there is no relationship between balance/unbalance and the class properties). Figure 1 shows the results. For Cifar and Eurosat those classes with more examples are better calibrated. The trend is somewhat similar for iNaturalist.\n\nThen, the authors present a number of approaches people have tried in the past to mitigate the consequences of unbalance in data. They repeat the previous experiment (on Cifar 10, Cifar 100, Eurosat) but, this time, using each of those methods while training the model. Table 1 shows the results. The ratio column offers very mixed results depending on the dataset and method. The authors conclude that overall the imbalance in calibration persists in most cases.\n\nQ. How do these results compare to accuracy? One would also expect to do better on classes with more data.\n\n- Label quality. The authors tackle the question of how label noise affects calibration. In order to do that, they artificially inject noise to the \"true\" labels with increasing probability. Figure 2 summarizes the calibration error for a number of datasets and noise level. The pattern is clear: the more noise, the worse the calibration. Importantly, the calibration is measured on a test set that is not perturbed with random noise. Accordingly, results were to be expected: there's a mismatch between training and test distributions, and the further apart they are, the less \"meaningful\" predicted probabilities one should expect. Again, it would be informative to see how the *accuracy* of the model also degrades under this circumstances. Similarly, Figure 3 shows the effect of non-uniform noise across classes. Those classes \"attacked\" with more noise are worse calibrated.\n\n- Dataset size. Another important practical aspect to study is dataset size. The authors subsample uniformly at random a fraction of the data points, and measure ECE. Figure 4 shows how models trained on more data are better calibrated. Again, the accuracy of the model should also be shown for context.\n\n- Augmentations. It is common to use data augmentation to train better models; augmentations make the effective datasize larger. Figure 5 shows how removing augmentation axes leads to worse calibration. The same probably applies to accuracy (that's the reason why people use this!). This result is probably intimately related to the previous point (dataset size).\n\n- NLP. The conclusions regarding dataset size also hold with a Transformer on an NLP dataset.\n\nFinally, Section 4 provides some theoretical explanation. We can summarize this as: the cross-entropy loss wants to have more and more confidence / probability on the right class for a given example, and when the data is small and the model powerful enough, we can basically memorize it to make cross-entropy happy. This, however, leads to overconfidence and poor calibration.\n\nOn one hand, it's recently becoming clear that ECE is not a very robust estimator. Depending on design choices (as number of bins, argmax vs all, adaptive versus fixed bins, etc.) the ranking among models and conclusions can change wildly [1]. On the other, this study fixed a specific model, so one could say that the conclusions are \"shown\" for the (dataset, model) pairs. Still, I believe the conclusions are true in a more general setting, though, and the model is fairly reasonable. However, while the paper is titled \"Dataset Curation Beyond Accuracy\", I do not see how the outcome and conclusions of all these experiments would be different if we were looking at accuracy rather than calibration. The authors should measure, include, and address this, and try to disentangle both aspects, or argue for any correlation / causation relationship among them.\n\n[1] - Measuring Calibration in Deep Learning - https://arxiv.org/abs/1904.01685", "extra_scores": {}, "extra_text": {}}, {"review_id": "LY3yLjXs7-", "reviewer": "AnonReviewer2", "summary": "", "questions": "", "limitations": "", "rating": 4, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "This work is an empirical survey of the calibration problem with convnets. The authors use several existing benchmark datasets and create synthetic class-imbalance for datasets that are initially balanced. They then extend the well-known results on higher prediction error of minority class, to its calibration error. The work investigates several existing methods that alleviate prediction error in imbalanced datasets and examine their effect on calibration error. At last, the effect of dataset size and data augmentation on calibration error is reported. Later on, the effect of random label noise is also examined. The observations, although not surprising, have not been reported before \nThe work is interesting, the writing is clear, and the experiments are comprehensive. Although the observations are very informative, the overall contribution of the paper is not sufficient for the ICLR venue. The work is mostly focused on reporting an existing issue with no major theoretical analysis of the problem and guidelines for alleviating the mentioned problems. The paper is in an interesting direction but needs to become more mature. \n\nQuestions and suggestoins:\n1- The label noise experiments are interesting. In reality, label noise is rarely random and is structured. It would be more helpful if the authors could extend the experiment to incorporate such scenarios.\n2- There seems to be an interesting difference among various reweighting methods in Table 1. It would be interesting if authors compared their calibration error performance to their prediction error performance to find out if there is a trade-off or the two phenomena are in the same direction.\n3- In a lot of experiments, for instance the dataset size, it's expected to have higher calibration error for smaller data. It would be more informative if the general trend of calibration error is compared with the trend in prediction error side by side.", "extra_scores": {}, "extra_text": {}}, {"review_id": "Bf5rDzJjmlq", "reviewer": "AnonReviewer4", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "In this work, authors demonstrate that dataset properties can significantly affect calibration and suggest that calibration should be measured during dataset curation. In the field of applied AI to real-life problem, we face all the time decision-makings on what is the most effective strategy in the pipeline (eg. sampling, noise, labeling) and this paper present some evidence for those decisions.\n\n\nThis type of work is important to systematically highlight areas or processes to follow in model development.\nThe study is not very novel, but important. Since the conclusions are very important and have key implications, I would suggest to apply this to more datasets, and also some of the existing synthetic datasets. Personally, I would like to see if these observations remain solid with more datasets and more variation of datasets.  \n\nI did not found any inconsistencies.", "extra_scores": {}, "extra_text": {}}]

Author rebuttal:

Meta-review: The authors empirically analyse the properties of datasets which lead to poor calibration. In particular, they show that high class imbalance, high degree of label noise, and small dataset size are all likely to lead to poor overall calibration or poor per-class calibration. While there are some interesting insights in this work, the reviewers argued that the contribution is not substantial enough for ICLR. To improve the manuscript the authors should consider accuracy and calibration jointly and extend the results pertaining to label noise which were appreciated by the reviewers. For the former, the same conclusions hold for accuracy, instead of calibration, which raises the question of their relationship -- is there a tradeoff? For the latter, the reviewers pointed to a concrete extension with structured label noise. Finally, the theoretical analysis is a step in the right direction, but the assumption on the width of the network required to fit the training set is too restrictive in practice. Therefore, I will recommend rejection.
Decision: Reject