Original: On Understanding the Hardness of Samples in Neural Networks
Revised: On Understanding the Hardness of Samples in Neural Networks

Original: Anonymous Authors

Original: Abstract
Revised: Abstract

Original: Deep neural network (DNN) has been an important tool for nonlinear function approximation.
Revised: The mechanisms used by the human visual system and artiﬁcial convolutional neural networks (CNN) to understand images are vastly different.

Original: While the effectiveness is widely validated in practice, current DNN models somehow remain black-box when making predictions.
Revised: The two systems have different notions of hardness, meaning the set of images which appear to be ambiguous and hard to classify are different. In this paper, we answer the following question: are there measures we can compute in the trained CNN models that correspond closely to human visual hardness? We employ human selection frequency, the frequency with which human annotators label a given image, as a surrogate for human visual hardness.

Original: In this paper, we aim to provide a novel perspective for understanding deep neural networks.
Revised: This information is recently made available on the ImageNet validation set (16). The CNN model conﬁdence does not correlate well with this human visual hardness score, and it is not surprising given that there are calibration issues in the models. We propose a novel measure known as angular visual hardness (AVH).

Original: Speciﬁcally, we hypothesize that the hardness of a sample largely depends on the angle between its embeddings and the corresponding ground truth classiﬁer.
Revised: It is the normalized angular distance between the image feature embedding and the weights of the target category. We demonstrate that AVH is strongly correlated with human visual hardness across a broad range of CNN architectures.

Original: To validate our hypothesis, we compare the hardness evaluated by human and our proposed characterization, which reveals their intrinsic correlations. Inspired by these empirical observations, we propose to employ novel forms of hashing for hard sample mining at scale.
Revised: We conduct an in-depth scientiﬁc study and test multiple hypotheses to draw this conclusion. We observe that CNN models with the highest validation accuracy also have the best AVH scores. This agrees with the earlier ﬁnding that the stateof-art (SOTA) models are improving classiﬁcation of harder examples.

Original: In addition to improving generalization, we show our proposal has a substantial impact on a broad range of applications such asdetecting the intrinsic long-tail intra-class variations, correcting “bias to ensure fairness in learning, and principled knowledge distillation.
Revised: We also observe that during the training of CNNs, AVH reaches a plateau in early stages even as the training loss keeps improving. We conjecture the different causes for such plateau of easy and hard examples, which suggests the need to design better loss functions Correspondence to: that can target harder examples more effectively and improve SOTA accuracy.

Original: 1. Introduction
Revised: 1. Introduction

Original: In recent years, neural networks have achieved unprecedented successes in a wide variety of tasks.
Revised: Convolutional Neural Networks (CNN) have achieved rapid progress on many computer vision tasks such as image classiﬁcation (8), face recognition, and scene analysis.

Original: Despite such successes, how deep neural networks understand the semantics of an input and make prediction accordingly remains not fully explored. There has been an active research line (Zhang et al., 2018; Zeiler & Fergus, 2014; Mahendran & Vedaldi, 2015; Simonyan et al., 2013; Fong & Vedaldi, 2017; Koh & Liang, 2017; Bau et al., 2017) that focuses on the visualization and diagnosis of deeply learned representations.
Revised: On large benchmark datasets such as ImageNet (3) they have even surpassed human-level accuracy. Despite this, these models are no match to the human visual system when it comes to other measures such as robustness and few-shot learning (1; 21; 15). This is not surprising given that they have entirely different processing mechanisms.

Original: Different from these works, we cast a novel view on understanding neural networks by identifying the sample hardness.
Revised: Due to the blackbox nature of CNNs and our limited understanding of the human brain, it is challenging to map out these differences precisely.

Original: Inspired by the observation in Figure 1 (also in (Liu et al., 1 Anonymous Institution, Anonymous City, Anonymous Region, Anonymous Country. Correspondence to: Anonymous Author <
Revised: In this paper, we focus on one key aspect, viz.

Original: anon.email@domain.com > .
Revised: , how different are the hard examples for these two systems?

Original: Preliminary work.
Revised: By hard examples, we mean the the set of images that appear ambiguous and are error prone.

Original: Under review by the International Conference on Machine Learning (ICML). Do not distribute. [Figure] [Figure]
Revised: Much of current deep learning research focuses on measuring the hardness of an image sample for deep models rather than for a human.

Original: The first dimension - - [Figure] [Figure] [Figure]
Revised: For example, hardness for models can be deﬁned using the loss value (18), relative Euclidean distance (17; 20) and gradient norm (11).

Original: h e [Figure] ond
Revised: On the other hand, there is a rich history in cognitive and neuroscience communities to understand human visual perception (6; 2).

Original: [Figure] i [Figure] n
Revised: Many works focus on mechanisms used by the human brain to translate visual information into mental representations.

Original: [Figure] on Semantic/Label Difference I n
Revised: These representations are subject to many correspondence differences and errors, and thereby are not isomorphic to the real world (13).

Original: [Figure] 2018b)) that the deeply learned features are naturally decoupled to the norm and the angle, we study how the hardness of samples affects the norm and angle of the embeddings separately.
Revised: They can be affected by the ambiguity of different semantics (10) such as occlusion, distortion, motion blur, and inherent similarity among objects. However, such detailed semantic information is typically not present in large-scale image benchmarks used to train the CNN models.

Original: Speciﬁcally, we empirically evaluate the hardness of samples using human annotation accuracy and draw the connections between the intermediate network outputs and human-deﬁned hardness.
Revised: A natural approach to measure human visual hardness is the human selection frequency, i.e. the rate with which human annotators select a speciﬁc image as belonging to a certain category. Unfortunately, most publicly available benchmarks do not have this information.

Original: (Liu et al., 2017b; 2018a) show that the angular information of  the embeddings is crucial to visual semantics.
Revised: But thanks to the recent efforts of (16), we now have this information on the ImageNet test set. We employ this new dataset in this paper and as a result, come up with many novel insights.

Original: It motivates us to hypothesize that  the hardness of a sample is mostly determined by the angle between its feature embedding and the corresponding ground truth classiﬁer.
Revised: In addition, we propose a novel measure that closely aligns with the human selection frequency, and hence, can be employed in other datasets where such information is not available.

Original: To validate our argument, we conduct experiments on thereannotated ImageNet testing set from (Recht et al., 2019). More speciﬁcally, we use the human annotation accuracy as a reﬂection of the ground truth hardness of samples.
Revised: Our Contributions: We conduct an in-depth exploratory study on the ImageNet testset with newly available human selection frequency information. We employ the scientiﬁc method and carefully test multiple hypotheses and present our ﬁndings.

Original:  We ﬁnd that the angular characterization of sample hardness is highly correlated with human annotation accuracy.
Revised: • We observe that the CNN model conﬁdence and human selection frequency are not strongly correlated.

Original: Connection to deep metric learning . Measuring the hardness of samples is also of great importance in the ﬁeld of deep metric learning (Oh Song et al., 2016; Sohn, 2016; Wu et al., 2017), because most objective functions in deep metric learning, , triplet loss (Schroff et al., 2015) and contrastive loss (Hadsell et al., 2006), requires data pair/triplet mining in order to perform well in practice. One of the most widely used data sampling strategies is semi-hard negative sample mining (Schroff et al., 2015) and hard negative sample mining.
Revised: The CNN model tends to be overconﬁdent and this is a well known calibration issue (7).

Original: These negative sample mining techniques highly depend on how one deﬁnes the hardness of samples, which motivates us to study how the hardness of samples is represented in deep neural networks. Connections to fairness in machine learning .
Revised: • We propose a new measure known as angular visual hardness ( AVH ) described below. • We observe that AVH is strongly correlated with human selection frequency across a wide range of CNN models.

Original: Easy and hard samples can implicitly reﬂect the inductive bias of the dataset, and the hardness of samples ispartially affected by the data imbalance problem.
Revised: To the best of our knowledge, this is the ﬁrst model score that correlates strongly with human visual hardness. Hence, it can serve as its proxy on datasets where such information is not available.

Original: For example, the CASIA-WebFace dataset (Yi et al., 2014) mostly contains western celebrities, so the neural network trained on CASIAWebFace is highly biased towards westerners. (Buolamwini & Gebru, 2018) also demonstrates the performance drop of faces of darker people due to the biases from the training dataset.
Revised: • We observe that the state-of-art (SOTA) models have the highest AVH score.

Original: In order to ensure fairness and remove dataset biases, the ability to identify hard samples and improve the model accuracyon these hard samples is of great signiﬁcance.
Revised: This implies that improving SOTA accuracy will entail improving accuracy of hard examples. AVH serves as a good measure to mine such hard examples in any dataset.

Original: Connections to knowledge transfer . The efﬁciency of knowledge transfer (Hinton et al., 2015) is also partially determined by the sequence of input training data.
Revised: • We observed the evolution of AVH score during training of CNN models. It plateaus early in training even as the training (cross-entropy) loss function keeps improving.

Original: (Liu et al., 2017a) theoretically shows thatfeeding easy samples ﬁrst and hard samples later can improve the convergence of the model. (Bengio et al., 2009) also show that the curriculum of feeding training samples matters in terms of both accuracy and convergence.
Revised: This suggests the need to design better loss functions that can improve performance on hard examples.

Original: There are various ways to deﬁne hardness of samples (Wu et al., 2017), but we focus onhow the norm and angle of the  embeddingreﬂect the sample hardness. To be more speciﬁc, we ﬁrst deﬁne the norm and angle of the embedding.
Revised: Angular visual hardness: We propose a new score for a given CNN model based on the normalized angular distance between the image feature embedding and the weights of the target category. The normalization takes into account the angular distances to other categories.

Original: The embedding of the i -th sample x i (belonging to class j ) is extracted from the input of the classiﬁcation layer (, classiﬁers). The classiﬁer for the i -th class is denoted as i .
Revised: We argue that the semantic ambiguity that affects human visual hardness is strongly correlated with this score.

Original: The norm of the embedding is deﬁned as (cid:107) x i (cid:107) , while the angle of x i is deﬁned as the angle between x i and the classiﬁer of the ground truth class W j . According to Figure 1, the angle stands for the  semantic hardness of a sample.
Revised: This is inspired by the intuition in (14) that the angle between image feature embedding and the weights of the target class accounts for the inter-class semantic differences while the (cid:96) 2 norm of the feature embedding accounts for intra-class variation.

Original: The norm corresponds to  the intra-class variation, and speciﬁcally when the angle is ﬁxed, smaller norm indicates harder sample. We will validate these characterizations with some preliminary empirical observations.
Revised: (14) used this insight to try to improve the generalization of the model. On the other hand, we use it to study the correspondence with human visual hardness.

Original: 3. Empirical Observations
Revised: 2. A Discovery of the Bridge: Angular Visual Hardness

Original: Our goal is to answer the following questions empirically: 1)

Original: How does human accuracy correlate with model accuracy?
Revised: In order to quantify Human Visual Hardness and Model

Original: 2) How does human accuracy correlate with (cid:96) 2 norm of the embedding? 3) How does human accuracy correlate with the angle between the feature embedding and the class embedding? What is the evolution of the norm and the angle of the embedding throughout the training process?
Revised: Predictions for convenience purposes in experiments, we use corresponding surrogates which are formally deﬁned as the following throughout the paper.

Original: Datasets and Settings .
Revised: Deﬁnition 1 (Model Conﬁdence) .

Original: We employ two standard image datasets MNIST (LeCun et al., 1998) and ImageNet(Deng
Revised: We deﬁne model conﬁdence on a single sample as the probability score of the true objective class output by the CNN models,

Original: [Figure]
Revised: [Equation]

Original: et al., 2009) in the  following experiments. Since a recent paper (Recht et al., 2019) demonstrates the ﬁndings when building new ImageNet validation set, wealso take advantage of the new dataset.
Revised: We employ the standard image benchmark ImageNet(4) in all following experiments. Particularly, we take advantage of the Human Selection Frequency information for validation images provided by the recent paper (16).

Original: The additional information about human labeling frequency which is a surrogate for  the human conﬁdence level is recorded for each image.
Revised: Recall that such information serves as a proxy for Human Visual Hardness. Besides, in order to verify that the our experimental results hold consistently across models instead of a particular model, we use four popular ImageNet pre-trained models

Original: For experiments on ImageNet, we use ResNet50(He et al., 2016) as the pre-trained model.
Revised: AlexNet (12), VGG19 (19), DenseNet121 (9), ResNet50 (8). We select ResNet50 as the representative model for some experiments.

Original: Baseline Results .
Revised: Studying the precise connection or gap between human visual hardness and model predictions is not feasible because data collection involving human labelling or annotation requires large amount of work. In addition, usually those human data is application or dataset speciﬁc, which makes the scalability of this study even worse. Therefore, all the testing and experiments we design are at best effort given the limited resources.

Original: We ﬁrst benchmark the correlation between human accuracy and model accuracy as our observational baseline.
Revised: That is exactly another motivation for us to bridge the gap between Human and models because models predictions require minimum costs compared to human efforts.

Original: Speciﬁcally, after being passed through the pre-trained model, each of the 10, 000 validation images obtains a softmax score for their labeled class.
Revised: An interesting observation in (16) shows that Human Selection Frequency has strong inﬂuence on the Model Conﬁdence. Speciﬁcally, examples with low Human Selection

Original: We deﬁne the softmax score as themodel accuracy.
Revised: Frequency tends to have relatively low Model Conﬁdence. Naturally we examine if the correlation between Model Conﬁdence and Human Selection Frequency is strong. Speciﬁcally, all ImageNet validation images are evaluated by the pre-trained models. The corresponding output is simply the

Original:  In addition,  each such image is provided with the frequency of being identiﬁed as the labeled class out of 50 workers who manually perform the labeling task. We also deﬁne the ratio of the frequency and the total number of 50 as human accuracy.
Revised: Model Conﬁdence on each image. In addition, because each such image is provided with the frequency of being identiﬁed as the labeled class out of 50 workers who manually perform the labeling task, i.e. Human Selection Frequency.

Original: The left plot in ﬁgure 5 presents a two-dimensional histogram for the correlation visualization.
Revised: The left plot in ﬁgure 1 presents a two-dimensional histogram for the correlation visualization.

Original: The x-axis represents human accuracy, and the y-axis represents model accuracy.
Revised: The x-axis represents Human Selection Frequency, and the y-axis represents Model Conﬁdence.

Original: Each bin exhibits the number of images which lie in the correspondinghuman and model accuracy range.
Revised: Each bin exhibits the number of images which lie in the corresponding range.

Original: We can observe the high density at the right corner, which means the majority of the images have both high human and model accuracy.
Revised: We can observe the high density at the right corner, which means the majority of the images have both high human and model accuracy.

Original: However, there is a considerable amount of density on the range of medium human accuracy but either extremely low or high model accuracy.
Revised: However, there is a considerable amount of density on the range of medium human accuracy but either extremely low 0.0 0.2 0.4 0.6 0.8 1.0 Human Selection Frequency 0.

Original: Overall, human and model accuracy is not in direct proportion.{+M o d e l C o n f i d e n c e 0 100300 400600 700 800 # o f S a m p l e s 0.0 0.2 0.4 0.
 0.8 1.0 Human Selection Frequency 0. || x || 
+}
Revised: {+M o d e l C o n f i d e n c e 0 100300 400600 700 800 # o f S a m p l e s 0.0 0.2 0.4 0.
 0.8 1.0 Human Selection Frequency 0. || x || 
+}

Original: {+2575 100150 175 200 # o f S a m p l e s 0.0 0.2 0.4 0.
+}
Revised: {+2575 100150 175 200 # o f S a m p l e s 0.0 0.2 0.4 0.
+}

Revised: 0.8 1.0 Human Selection Frequency 0.

Revised: AV H ( x ) 0200400600 # o f S a m p l e s or high model accuracy.

Revised: Overall, Model Conﬁdence and

Revised: Human Selection Frequency are not in direct proportion and thereby not strongly correlated.

Original: 3.1. Embedding (cid:96) 2 Norm
Revised: 2.2. Bridging the Gap

Revised: Followed by identifying the gap in last section, we naturally propose a hypothesis: Hypothesis 2. There exists some characteristic in CNN

Revised: Models strongly correlates with Human Selection Frequency to bridge the gap?

Revised: In this section, We ﬁrst provide two predictions and test them accordingly.

Revised: Denote S n as the unit n -sphere, formally,

Revised: S n = { x ∈ R n +1 |(cid:107) x (cid:107) 2 = 1 } .

Revised: Below by A ( · , · ) , we denote the angular distance between two points on S n , i.e. ,

Revised: A ( u , v ) =

Revised: arccos( (cid:104) u , v (cid:105) (cid:107) u (cid:107)(cid:107) v (cid:107) ) .

Revised: Let x be the feature embeddings input for the layer before the last one of the classiﬁer of the pretrained CNN models, eg.

Revised: FC2 for VGG19.

Revised: Let C be the number of classes for a classiﬁcation task.

Revised: Denote

Revised: W = w i | 0 < i ≤ C} as the set of weights for all C classes in the ﬁnal layer of the classiﬁer.

Revised: Deﬁnition (Angular

Revised: Visual

Original: Both Figure 1 and (Liu et al., 2018b) demonstrate the observation that the embedding (cid:96) 2 norm accounts for intra-class variation.
Revised: Hardness (AVH)) . AVH, for any x , is deﬁned as, AVH (x) = A(x,w y ) (cid:80) Ci =1 A ( x,w i ) ,which w y represents the weights of the target class.

Revised: Prediction 1: (cid:107) x (cid:107) has

Revised: a strong correlation with Human Selection Frequency

Revised: (14) conjectures that (cid:107) x (cid:107) 2 accounts for intra-class Human/Model Conﬁdence.

Original: If the norm is larger, the prediction from the model is also more conﬁdent, to some extent.
Revised: Particularly, if the norm is larger, the prediction from the model is also more conﬁdent, to some extent.

Original: In this section, we try to understand the relationship between human accuracy and embedding (cid:96) 2 norm (a measurement of model conﬁdence).
Revised: Therefore, we conduct similar experiments like previous section to demonstrate the correlation between (cid:107) x (cid:107) 2 and Human Selection Frequency.

Original: Initially, we compute the (cid:96) 2norm of the embedding layer for every validation sample.
Revised: Initially, we compute the (cid:107) x (cid:107) 2 for every validation sample for all models.

Original: Then we normalize the value of the norm within each class.
Revised: Then we normalize (cid:107) x (cid:107) 2 within each class.

Original: The right plot in ﬁgure [Figure] uses a two-dimensional histogram to show the relationship between human accuracy and embedding (cid:96) 2 norm or model conﬁdence for all the validation images.
Revised: The middle plot in ﬁgure 1 uses a two-dimensional histogram to show the correlation for all the validation images.

Original: Given that the norm has been normalized with each class, naturally, there is notable density when the norm is 0 or 1 .
Revised: Given that the norm has been normalized with each class, naturally, there is notable density when the norm is 0 or 1 .

Original: Except for that, there is no obvious correlation between human accuracy and norm.
Revised: Except for that, there is no obvious correlation between (cid:107) x (cid:107) 2 and Human

Original:  We further verify if presenting all samples across 1000 different classes affects the visualization of the relationship between human accuracy and norm.
Revised: Selection Frequency. We further verify if presenting all samples across 1000 different classes affects the visualization of the correlation.

Original: According to WordNet (Fellbaum, 2005) hierarchy, we map the originalﬁne-grained classes to 45 higher hierarchical classes.
Revised: According to WordNet (5) hierarchy, we map the original 1000 ﬁne-grained classes to 45 higher hierarchical classes.

Original: Figure 4 exhibits the relationship between human accuracy and norm for four representative higher classes containing90 , 58 , 7 , 1 ﬁne-grained classes respectively.
Revised: Figure 7 exhibits the relationship between Human Selection Frequency and (cid:107) x (cid:107) 2 for three representative higher classes containing 58 , 7 , 1 ﬁne-grained classes respectively.

Original: Noted thatfrom all four plots, there is still not any visible direct proportion between these two properties.
Revised: Noted that there is still not any visible direct proportion between these two variables across all plots.

Original: 3.2. (cid:96) 1 Normalized Logits and Cosine Similarity
Revised: Prediction 2: AVH ( x ) has a strong correlation with Human Selection Frequency

Original: In this section, we explore the correlation between human accuracy and (cid:96) 1 normalized logits.
Revised: We test the correlation between AVH ( x ) and Human Selection Frequency.

Original: We extract the logits of each validation sample before the softmax layer and process them with (cid:96) 1 normalization, formally,
Revised: Correspondingly, after evaluating each validation sample on pre-trained models, we extract feature embeddings x and also the class weights W to compute

Original: [Equation]
Revised: AVH ( x ) .

Original: where W y represents the class embedding for the labeled class.

Original: Noted that we linear scale the range of the normalized logits to [0 , 1] .
Revised: Noted that we linear scale the range of AVH ( x ) to [0 , 1] .

Original: The plot on the left in Figure 5 shows positive correlation between human accuracy correlates with (cid:96) 1 normalized logits for validation images. According to equation 1, embedding (cid:96) 2 norm has been canceled in the ﬁnal form and thereby does not have any inﬂuence to the correlation.
Revised: The plot on the right in Figure 1 shows strong correlation between AVH ( x ) and Human Selection Frequency for validation images. One intuition behind this correlation is that the class weights W might corresponds to human semantic for each category and thereby AVH ( x ) corresponds to human semantic categorization of an image.

Original: Moreover, in order to explore if (cid:96)  2 norm of the class embedding has any effect to  the correlation, we normalize it, formally,
Revised: Embedding (cid:96) Norm (cid:107) x (cid:107) 2 is on the other hand irrelevant. In order to test if the strong correlation holds for all models, we perform the same experiments on AlexNet, VGG19 and

Original: [Equation] This represents purely the normalized cosine similarity between class embedding and feature embedding.
Revised: DenseNet121.

Original: The right plot in Figure 5 showsalmost exactly the same correlation.
Revised: Figure 6 shows the strong correlation of

Original: Our direct intuition from those observations is that (cid:96)normalized logits (or the cosine similarity) corresponds to human semantic categorization of an image. Embedding (cid:96) Norm is on the other hand irrelevant.
Revised: AVH ( x ) and Human Selection Frequency consistently. 3. Dynamics of AVH during Training

Original: Recall the deﬁnition ofhard example described in Section 2, we can derive the above intuition to: Hard examples for human correspond to  the ones with lower cosine similarity between example and labeled class embedding in the model .
Revised: After discovering the strong correlation of human visual hardness and AVH score, a natural question would be: What role does AVH play during the training process? Optimization Algorithms are used to update weights and biases i.e. the internal parameters of a model to improve the training loss. Both the angles between feature embedding and classiﬁers and the L 2 norm of the embedding can inﬂuence the loss.

Original: 3.3. Dynamic Experiments over (cid:96) 2Norm and Cosine Similarity
Revised: While it is well-known that the training loss or accuracy keeps improving but it is not obvious what would be the dynamics of the angles and norms separately during training. we design the experiments to observe the training dynamics of various network measurements like (cid:107) x (cid:107) 2 and AVH(x).

Original: In previous sections, we demonstrate some insights into the correlation between human and model hard examples.
Revised: Experiment Settings For datasets and models, we use exactly the same setting as the experiments in ?? . Nevertheless, observing training dynamics involves training models from scratch on ImageNet training set instead of directly using the pre-trained models. Therefore, we follow the standard training process of AlexNet (12), VGG19 (19), DenseNet121 (9), ResNet50 (8).

Original: In this section, we aim to observe how the norm and  the angle change throughout the training process on MNIST and{+initial learning 20 40 60 80 Epoch30 40 5070 80|| x || 20 40 60 80 Epoch
 || x || 
+}
Revised: For consistency, we train all four models for 90 epochs and decay the {+initial learning 20 40 60 80 Epoch30 40 5070 80|| x || 20 40 60 80 Epoch
 || x || 
+}

Original: ImageNet dataset.{+40 60 80 Epoch
 || x || 20 40 60 80 Epoch 7.75 8. AV H ( x ) 1e  20 40 60 80 Epoch 7. AV H ( x ) 1e  20 40 60 80 Epoch 6.+}
Revised: {+40 60 80 Epoch
 || x || 20 40 60 80 Epoch 7.75 8. AV H ( x ) 1e  20 40 60 80 Epoch 7. AV H ( x ) 1e  20 40 60 80 Epoch 6.+}

Original: {+AV H ( x ) 1e  20 40 60 80 Epoch
+}
Revised: {+AV H ( x ) 1e  20 40 60 80 Epoch
+}

Original: {+M o d e l A cc u r a c y 20 40 60 80 Epoch 020 3050 
+}
Revised: {+M o d e l A cc u r a c y 20 40 60 80 Epoch 020 3050 
+}

Original: {+M o d e l A cc u r a c y 20 40 60 80 Epoch30 
+}
Revised: {+M o d e l A cc u r a c y 20 40 60 80 Epoch30 
+}

Original: Figure 6 presents how the average norm of the feature embedding and the cosine of the angles ( cosθ ) between feature andclass embedding for testing samples vary in 90 epochs of training on ResNet50 (He et al., 2016).
Revised: M o d e l A cc u r a c y rate by a factor of 10 every 30 epochs. The initial learning rate for AlexNet and VGG19 is 0 . 01 and for DensetNet and ResNet50 is 0 . We split all the validation images into 5 bins based on their human selection frequency, respectively [0 . 0 , 0 . 2] , [0 . 2 , 0 . 4] , [0 . 4 , 0 . 6] , [0 . 6 , 0 . 8] , [0 . 8 , 1 . 0] for experiments on dynamics under different human selection frequency.

Revised: Note that for all the ﬁgures in this section, Epoch starts from 1.

Revised: Observation 1: Dynamics of (cid:107) x (cid:107) 2 and model accuracy are similarly increasing Figure 2 presents the dynamics of the average (cid:107) x (cid:107) 2 and the dynamics of the accuracy for testing samples vary in 90 epochs during the training on four architectures.

Original: Note that both examples are testing data  and thereby have never ﬁt  into the model.
Revised: Note that we are using the validation data for dynamics observation and thereby have never ﬁt them into the model.

Original: The average norm increases with a small initial slope but it suddenly climbs after 30 epochs when the ﬁrst learning rate decay happens.
Revised: The average (cid:107) x (cid:107) 2 increases with a small initial slope but it suddenly climbs after 30 epochs when the ﬁrst learning rate decay happens.

Original: On the other hand, the average cosθ increases sharply at the beginning and then suddenly peaks also at 30 epoch.
Revised: The accuracy curve is very similar to that of the average (cid:107) x (cid:107) 2 . The above observations are consistent in all models. Observation 2: AVH(x) hits a plateau very early even when the accuracy or loss is still improving Figure 2 exhibits the change of average AVH(x) for testing samples in 90 epochs of training on four models. The average AVH(x) for AlexNet and VGG19 decreases sharply at the beginning and then starts to bounce back a little bit before converging.

Original: Then it gradually drops until the second-time learning rate decay on epoch 60. After this second peak, the curve remains stable.
Revised: However, the dynamics of the average AVH(x) for DenseNet121 and ResNet50 are different. They both decrease slightly and then quickly hits a plateau in all three learning rate decay stages.

Original: Moreover, we explore the difference between norm and angle change for easy and hard human examples in more details.
Revised: But the common observation is that they all stop improving even when (cid:107) x (cid:107) 2 and model accuracy are increasing. However, AVH(x) is more impor20 40 60 80 Epoch30 40 5070 80|| x || Freq=[0.0 ,0.2] Freq=[0.2 ,0.4] Freq=[0.4 ,0.6] Freq=[0.6 ,0.8] Freq=[0.8 ,1.0]

Original: With the human accuracy information on ImageNet testing dataset provided by (Recht et al., 2019), we split the samples into ﬁve bins corresponding to a different level of human hardness.
Revised: 20 40 60 80 Epoch3050|| x || Freq=[0.0 ,0.2] Freq=[0.2 ,0.4] Freq=[0.4 ,0.6] Freq=[0.6 ,0.8] Freq=[0.8 ,1.0] 20 40 60 80 Epoch 200 1030 40|| x || Freq=[0.0 ,0.2] Freq=[0.2 ,0.4] Freq=[0.4 ,0.6] Freq=[0.6 ,0.8] Freq=[0.8 ,1.0] 20 40 60 80 Epoch 8.0 8. AV H ( x ) 1e

Original: Figure 7 demonstrates the change of the norm and cosθ similar to Figure 6, but average over testing samples in ﬁve bins separately.
Revised: Freq=[0.0 ,0.2] Freq=[0.2 ,0.4] Freq=[0.4 ,0.6] Freq=[0.6 ,0.8] Freq=[0.8 ,1.0] 20 40 60 80 Epoch 7.0 7. AV H ( x ) 1e Freq=[0.0 ,0.2] Freq=[0.2 ,0.4] Freq=[0.4 ,0.6] Freq=[0.6 ,0.8] Freq=[0.8 ,1.0] 20 40 60 80 Epoch 6.5 7. AV H ( x ) 1e

Original: We can observe that for the norm, the gaps between the samples with different level of hardness are not obvious.
Revised: Freq=[0.0 ,0.2] Freq=[0.2 ,0.4] Freq=[0.4 ,0.6] Freq=[0.6 ,0.8] Freq=[0.8 ,1.0] 20 40 60 80 Epoch 0.0 0. M o d e l A cc u r a c y Freq=[0.0 ,0.2] Freq=[0.2 ,0.4] Freq=[0.4 ,0.6] Freq=[0.6 ,0.8] Freq=[0.8 ,1.0] 20 40 60 80 Epoch 0.0 0.1 0. M o d e l A cc u r a c y Freq=[0.0 ,0.2] Freq=[0.2 ,0.4] Freq=[0.4 ,0.6] Freq=[0.6 ,0.8] Freq=[0.8 ,1.0] 20 40 60 80 Epoch 0.1 0.

Original: Besides, the gaps are closed when the training converges. However, for  the angle, the gaps are signiﬁcant and consistent across the training process. With all critical observations above, we propose a hypothesis that human harder testing examples correspond to relatively larger angles throughout the training process.
Revised: M o d e l A cc u r a c y Freq=[0.0 ,0.2] Freq=[0.2 ,0.4] Freq=[0.4 ,0.6] Freq=[0.6 ,0.8] Freq=[0.8 ,1.0] tant than (cid:107) x (cid:107) 2 because during inference it is the key factor deciding which class the input sample is classiﬁed to The dynamics for (cid:107) x (cid:107) 2 should be monotonically decreasing so we raise the question that is the cross-entropy loss the best for CNNs? Observation 3: AVH(x)’s correlation with human selection frequency holds across models and throughout the training process.

Original: However, the model would adjust the norm to be much higher just for optimization purpose, eg. reducing the loss. [Figure] [Figure]
Revised: Figure 3 demonstrates the change of (cid:107) x (cid:107)and AVH(x) similar to Figure 2, but average over testing samples in ﬁve human selection frequency bins separately.

Original: [Figure] [Figure] L 1 L o g i t s [Figure]
Revised: We can observe that for (cid:107) x (cid:107) 2 , the gaps between the samples with different human visual hardness are not obvious in

Original: [Table] m p l e s [Figure]
Revised: ResNet50 and DenseNet121.

Original: We show a special case in Figure 3 to illustrate how the norm and the angle change when one sample switches from one class to another. Speciﬁcally, we change the sample from one class to another using adversarial perturbation. It is essentially performing gradient ascent to the ground truth class.
Revised: Besides and closed near convergence.

Original: In Figure 3, the purple line denotes the trajectory of an adversarial sample switching from one class to another. We can see that the sample will ﬁrst shrink its norm towards origin and then push its angle away from the ground truth class. Such a trajectory indicates that the norm corresponds to the intra-class hardness, and the adversarial sample will ﬁrst approach to the origin in order to become a hard sample for this class. The angle corresponds to the inter-class semantic hardness, and  the sample will change the angle in order to switch its label.
Revised: However, for AVH(x), such gaps are signiﬁcant and consistent across every single model during the whole training process.

Original: This special example fully justiﬁes the importance of both norm and angle in terms of the hardness of samples. 4. Future Work: Adaptive Sampling via LSH The key ingredient of our proposal isto use LocalitySensitive Hashing (Indyk & Motwani, 1998) as an adaptive sampler to efﬁciently sample the hard or easy examples based on designed metrics. LSH is a popular, sub-linear time algorithm for approximate nearest-neighbor search.
Revised: Observation 4: AVH(x) is an indicator of model’s generalization ability From Figure 2 and Figure 3, we observe that better models have better AVH(x) throughout the training process and also across samples under different human selection frequency.

Original: The high-level idea is to place similar items into the same bucket of a hash table with high probability.

Original: An item returned as a candidate from a ( K, L ) -parameterized LSH

Original: [Figure] [Figure] algorithm is sampled with probability 1 − (1 − p K ) L , where p is the collision probability of LSH function. The LSH family deﬁnes the precise form ofp that used to build the hash tables.
Revised: For instance, Alexnet is the worst model and its overall AVH(x) or average AVH(x) on each of ﬁve bins are worse than those of the other three models.

Original: Recent advances in maximum inner product search (MIPS) using asymmetric locality sensitive hashing has made it possible to sample large inner products. Assume that given a collection C of vectors and query vector Q , using ( K, L ) - parameterized LSH algorithm with MIPS hashing (Shrivastava & Li, 2014), we get a candidate set S that every element x i ∈ C is sampled withprobability p i ≤ 1 , where p i is a monotonically increasing function of Q · x i . Thus, we can pay a one-time linear cost of preprocessing C into hash tables, and any further adaptive sampling for query Q only requires few hash lookups. We can also compute the probability of getting x .
Revised: This observation aligned with the earlier observations of (16) that better models also generalize better on samples across different human selection frequencies.

Original: A. Additional Experiments
Revised: A. Additional Experiments

Original: Figure 8 illustrates how the average norm of the feature embedding and angles between feature and class embedding for testing samples vary in 60 iterations during the training process.
Revised: Figure 4 illustrates how the average norm of the feature embedding and angles between feature and class embedding for testing samples vary in 60 iterations during the training process.

Original: The average norm increases with a large initial slope but it ﬂattens slightly after 10 iterations.
Revised: The average norm increases with a large initial slope but it ﬂattens slightly after 10 iterations.

Original: On the other hand, the average angle decreases sharply at the beginning and then becomes almost ﬂat after 10 iterations.
Revised: On the other hand, the average angle decreases sharply at the beginning and then becomes almost ﬂat after 10 iterations.

Original: Moreover, we explore the difference between norm and angle change for easy and hard human examples in more details.
Revised: Moreover, we explore the difference between norm and angle change for easy and hard human examples in more details.

Original: Figure 9 also plots the angle and norm changes for two examples, which are hard and easy for human visualization, in the training phase.
Revised: Figure 5 also plots the angle and norm changes for two examples, which are hard and easy for human visualization, in the training phase.

Original: Note that both examples are testing data and thereby have never ﬁt into the model.
Revised: Note that both examples are testing data and thereby have never ﬁt into the model.

Original: We can see that for the angle, both of them drop largely initially and then the angle for the easy one converges to a much lower value.
Revised: We can see that for the angle, both of them drop largely initially and then the angle for the easy one converges to a much lower value.

Original: For the norm, both of them are increasing drastically at an early stage but that for the harder example keeps climbing even when that for the easy one saturates. 
Revised: For the norm, both of them are increasing drastically at an early stage but that for the harder example keeps climbing even when that for the easy one saturates. [Figure] [Figure] [Figure]

Revised: [Figure]

Revised: [Figure]

Revised: [Figure]

Revised: [Figure]

Revised: [Figure]

Original: {+L 2 N o r m 0 26 812# o f S a m p l e s 0.0 0.2 0.4 0.
+}
Revised: {+L 2 N o r m 0 26 812# o f S a m p l e s 0.0 0.2 0.4 0.
+}

Revised: 0.8 1.0 Human Accuracy 0.

Revised: L 2 N o r m 0.0 0.

Original: {+# o f S a m p l e s 0.0 0.2 0.4 0.
+}
Revised: {+# o f S a m p l e s 0.0 0.2 0.4 0.
+}

Revised: 0.8 1.0 Human Accuracy 0.

Revised: L 2 N o r m 0.00 0.25 0.

Revised: # o f S a m p l e s under different class granularity (according to WordNet hierarchy).

Revised: From left to right, there are 58, 7, 1 classes respectively.

Revised: The human selection frequency is therefore computed based on the new class granularity.