Introducing Sample Robustness
Authors: Monty Maximilian Zühlke
Venue:  (2021)

Abstract: Choosing the right data and model for a pre-defined task is one of the critical competencies in machine learning. Investigating what features of a dataset and its underlying distribution a model decodes may enlighten the mysterious "black box" and guide us to a deeper and more profound understanding of the ongoing processes. Furthermore, it will help to improve the quality of models which directly depend on data or learn from it through training. In this work, we introduce the dataset-dependent concept of sample robustness, which is based on a point-wise Lipschitz constant of the label map. For a particular sample, it measures how small of a perturbation is required to cause a label-change relative to the magnitude of the label map. We introduce theory to motivate the concept and to analyse the effects of having similar robustness distributions for the training- and test data. Afterwards, we conduct various experiments using different datasets and (non-)deterministic models. In some cases, we can boost performance by choosing specifically tailored training(sub)sets and hyperparameters depending on the robustness distribution of the test(sub)sets.

Under review as a conference paper at ICLR 2021
INTRODUCING SAMPLE ROBUSTNESS
Anonymous authors
Paper under double-blind review
ABSTRACT
Choosing the right data and model for a pre-defined task is one of the critical competencies in machine learning. Investigating what features of dataset and its underlying distribution a model decodes may enlighten the mysterious "black box" and guide to a deeper and more profound understanding of the ongoing processes. Furthermore, it will help to improve the quality of models which di- rectly depend on data learn from it through training. this work, we introduce the dataset-dependent concept of sample robustness, which is based on point- wise Lipschitz constant of the label map. For a particular sample, it measures how small of perturbation is required to cause a label-change relative to the magnitude of the label map. We introduce theory to motivate the concept and analyse the effects of having similar robustness distributions for the training- and test data. Afterwards, we conduct various experiments using different datasets and (non-)der doon-)ddeemmiiiticc models. In some cases, we can boost performance by choos- ing specifically tailored (aining(sub)sett and hyperparameters depending on the robustness distribution of the test(sub)sets.
INTRODUCTION:
In the age of automated machine learning, we shift our focus evermore towards regarding meta- hyperparameters such as model-type or training- and validation budget as variables of loss function in the most abstract sense. For training sets, however, the mere number of samples often determines how well suited it is perceived for a particular task. The motivation of this work is introduce concept allowing for the use of datasets variables of such a generalised loss function well.
Imagine, for example, patients who share almost identical medical records, but react differently to some prescribed treatment. They may pose a challenge to machine learning models similar to what is known as natural adversarial examples in vision tasks, see Hendrycks et al. (201)). The relationship between medical features and appropriate personal treatments may be sensitive towards small input variations, i.e. not robust towards perturbations. In this work, we are to the best of knowledge the first introduce and analyse model-agnostic measure for the robustness of data. We show that knowledge about the robustness distribution of a specific test(sub)set can allow for choosing more appropriate training(sub)set in terms of performance optimisation. Finally, discover that the optimal choice of hyperparameters may also depend on the robustness distributions of both training- and test data.
Let us motivate the concept of sample robustness first on high level. When collecting and process- ing dataset for a pre-defined task, we identify certain features and expressive samples such that model may be able to abstract and generalise from these finite points to the whole space of possible inputs. Assume we have a certain settangle-shapee and a dataset labelled according to two distinct ground truth maps y² E {x, *} and =" E {c, (comp. Figure 1). Here, one can imagine classifying images of horses and cats (assuming ground truth y²) and classifying images of animal-human pairs (assuming ground truth 2²). Evidently, the distance between differently labelled samples depends on the ground truth map labelling them.
distrriittiittion
circle-shaped feature space
For every sample in dataset, the intrinsic information of closeness to differently labelled sam- ple can be considered a feature itself. For regression tasks and label distributions which are not necessarily categorical one may also include the distance of the corresponding labels as additional information. By taking the quotient of these two and maximising it over the dataset, i.c. calculating point-wise Lipschitz constant of the label map,
measure how sensitive sample is to label-

---

Under review as a conference paper at ICLR 2021
Figure I: Images from the CIFAR-10 dataset labelled "horse" and "cat", respectively.
changing perturbations. Precisely this is the non-formal basis of the concept we propose, namely sample robustness. Our work aims to analyse the connection between both model performance and aligning the robustness distributions of training- and test sets.
1.1 OUTLINE
After citing and discussing related work concerned with decision boundaries. model robustness and Lipschitz calculus in section 2, we introduce the mathematical framework and the measure of sample robustness in section 3. We also motivate the concept theoretically and show some natural relations to K-Nearest Neighbour models. Section 4 is completely devoted to the evaluation using different datasets (CIFAR- 10, Online News Popularity Dataset) and models (Convolutional Neural Networks, K-Nearest Neighbour, Random Forest). Section 5 finally concludes the findings and discusses other research approaches. Letters A F refer to sections in the appendix.
2 RELATED WORK:
Analysing the data-distribution before training yields way to investigate (and boost) model per- formance from an earlier stage as is done by unsupervised pre-training (Erhan et al. 2010)). Many algorithms stemming from the unsupervised setting (Chigirev & Bialek 2003 Cayton 2005 are de- voted to extract information about the so-called data- manifold (Fefferman et al.) [013 Bernstein & Kuleshov 2014). Decoding the latent features which determine the data data-distribution (Bengio al. 2013) provides valuable insight and helps to understand the decision boundaries which model learns throughout the training phase. Furthermore, understanding the data-manifold may provide view into the "black box" transforming inputs to outputs (Fawzi et al.) 2016), Liu et al., [017]; Biggio et al. 2013). In this work, we want to use the intrinsic information distance between samples in feature space and relate it to the distance of the corresponding labels to introduce new dataset-dependent feature.
The robustness of sample be regarded its susceptibility to label-changing perturbations. Here, one immediately reminded adversarial examples (Szegedy et al.) (2013) in the context proposed concept, however, is that we only the
of model robustness. The difference pre-defined labels instead model predictions as additional input. The term robustness itself is one of the most prominent throughout the recent literature many different contexts, from robust attacks to robust models/defences (Evtimov et al.) 2017: Beggel et al.) 2019 Madry et al. 2017] Weng et al. 2018 Tsuzuku al. 2018). State-ol-the-ar machine learning models are susceptible to noise, especially so when crafted purposefully (Fawzi et al. 2016). It leaves these powerful ma- chines vulnerable to attacks either at training- (Zhu et al. (2199) or at test stage (Biggio al. 2013), independent of the architecture used (Papernot et al. 2016b). We follow the idea that models are ex- tensions of the label map from the (metric) subspace defined by a dataset the whole feature space. Hence, they will inherit critical properties from the data. In this work we analyse the robustness distribution of datasets and the thereon dependent performances of models, but plan on investigating the connection model-robustness in the future.
Lipschitz calculus yields a mathematically well-understood approach to describe and measure model robustness in WWng et al. (2018) or Tsuzuku et al. (2018). Framing machine learning theory in terms of metric spaces (and also building robustness concepts thereon) has been done before in Wang et al. (2016). however, not explicitly connecting it to Lipschitz calculus. In this work,
2

---

Under review as a conference paper at ICLR 2021
build the concept of sample robustness based point-wise Lipschitz constant of the label map for metric- and Banach spaces such that it applies to a wide range of different feature- and label spaces including an ample variety of metrics.
Finally, investigating the dependence of seeedddhherrrrmetersss on data has been done previously Nakkiran al. 2020). The authors also showed that more data could sometimes decrease model performance. In this work, we will see similar results regarding these two aspects with the differ- ence that we could identify such data even before training using the proposed measure of sample robustness.
3 SAMPLE ROBUSTNESS
Now we will introduce the primary concept of this work, namely sample robustness. measures how sensitive samples are label-changing perturbations relative the magnitude of the label map.
3.1 DEFINITION: (FRAMEWORK)
Let FS be a feature space, i.e. a metric space with metric == drs, let TS be target- or label space, i.e. real Banach space with norm ||. == || l|Ts. let x, C FS be finite (data)sets with #x, #t 2 2 and let xUt + TS be map of labels with #y(x) #y(t) 2 2.
3.2 DEFINITION: (REACH)
Let FS be sample with label y)r), ||y( llyloc, and (x))) (# ((()) # (())- The reach of x is defined as the distance of æ to (x):
Fx x dist (z. [x()))
min d(x, 2ETx(0)
In other words: [x)x) is the set of samples F in the dataset that are labelled differently from x. One may notice that ã E (x) < x E (T)). The reach of x E x is exactly the minimal distance of the point representing the image of "cat" to differently labelled (coloured) point in Figure. 1. is "close" to ă, one would expect their labels y(x) and (() to be "close" as well. Taking the quotient then gives measure of this error (comp. 3.3) and normalising the latter with respect the magnitude of y (using ||y|| == max [[)())]) yields the main concept of sample robustness 3.4.
Fex
3.3 DEFINITION: (POINT- WISE LIPSCHITZ CONSTANT)
For E FS with label y(x), [[)( < ||a|oo, and (x(x) as 3.2 one defines a point-wise Lipschitz constant of y
(lu(x))
- -())l d(x, 2)
Qx(x) max selv(a)
3.4 DEFINITION AND PROPOSITION: (SAMPLE ROBUSTNESS)
Let FS with label y(z), [x)(())| I llvlloc, and (z) as in 3.2. The robustness of the sample in with respect to d is defined as:
| E 10,
Rx(x) = 2x(x) + 1
It is independent of rescaling the label map y (comp. A.2). Coming back the example in the introduction, we can see now that the almost identical medical records of patients reacting differently to the same treatment considered less robust samples in the above sense.
3

---

Under review as a conference paper at ICLR 2021
3.5 THEORETIC MOTIVATION AND BACKGROUND
Assume we have datasets x and t where both are labelled using the same label map with max [[v(x)] max [1(()]] For any = U it holds that: sEx
(x) - max
(2x (x), 2+(=)) + Rx (xx R+)))
In other words: the closer Rx(2) is to R₂ (2), the closer both values are to the robustness of in the union x U t. It follows:
Rx(2) R+(2) Rx(2) Rx (2)) Re(2).
where at least one side is an equality. For convenience, we write X ~R t : Rx Re(2) V2 U t. Assume now that F' is extension of the label map y from x to x can downsize the set x to X C in order to align both robustness distributions; however, there will likely be trade-off between this alignment and the distance of to y as maps on t, because there are less points to extend from (therefore allowing for a higher variance).
For given t one
For such an extension holds that Flx V1 thus:
[1(())
y(x)
d(z, x) F(()) F(+)
(x(x)
max SET
max =ETx(x)
V
d(I, R)
Assuming ~R then enables the following conclusion for EXXXX t:
(1(2)--(E)]1
(F((z) - y(()) + Eall
(*) ex)x(2) s 2x(2)=
max =ETx(=)
max =ET
d(z,2)
d(z,))
where E, := ((z))F((2) Notably, the rights side depends at most on one point outside x. Therefore it includes at most one compared to the naive approach including both and 6s:
I1v(z) v())l SEl (()) d(2,2)
(F(()) - F(3)) + (c= - (=)]l d(2, 2)
Qx
SET (s)
To summarize: by assuming ~R t one can find small 72 such that 2x u (=) @x(=) + and trade Es E TS for 92 E R. But whereas the first depends on the extension F, the latter only depends on the data.
Let now L(F - y) be the Lipschitz constant the map F - y on x U Using (+) and the reverse triangle inequality one derive the following (comp. A.4):
[1(()) - y(E)I
[F(2)- - F(E)ll
L(F y) 2
max #EX UEETET (
max
d(z,2)
max
d(z,2)
[F((z) F(E)) + Eall
[F(z) - F(5)
max
max (=)
+
max SET (=)))
ZEX
d(2,E)
d(2
Hence, Eg and determine a lower bound on -(F-w) and decreasing it a priori may allow for finding an extension that minimizes both | ylloo and L(F- y) at the same time. By aligning the robustness distributions of x and this bound will not only depend on the extension F, but on the data (trading the possibly uncontrollable for the controllable). Finally, the true motive for such an approach stems from functional analysis: the space of Lipschitz functions from x U t to TS, ie. Lip(x U t, S). is a Banach space with respect to the norm || |laum := -I- Il + L(-) (comp. Cobzay et al. (2019)). ). So by regarding extensions F of y that minimise | ylleum restrict the size of the hypothesis space from arbitrary maps to Lipschitz maps. The completeness of Lip(x U TS) is of importance it prevents sequences of extensions (F.) with F., Ullsum
priori
to exit this smaller space?
'One may think of machine learning model F' trained on x making predictions on t. Training machine learning model produces exactly such sequence F.,
4

---

Under review as a conference paper at ICLR 2021
3.6 SAMPLE ROBUSTNESS AND KNN
Let F(z) = Σ way wiy(=') (z*) be K-nearest-neiihbour model with reference set x, where 2f is the i-th nearest neighbour of 2 in x with weight w,. The formula (*) from 3.5 translates to:
[x(()) v(3))
K way v(2) + czll
max R
d(z,
max EETx(2)
d(2,2)
Hence, by assuming x ~ t we have imposed constraint on the (deterministic) model. More precisely, it is forced to base its prediction of 2 on those K-samples 2' close to for which the weighted linear combination of their labels suffices the above formula a priori. If among the most robust samples, we know that small change in feature space will only cause a small change label space. Therefore we expect that a higher produces a higher accuracy as samples close to will be more reliable predictors. Conversely, this is not the case for less robust samples can be seen by comparing prediction and true label:
F(z) - v(=) = - (u(2)) y()))
If there exists a =" close to such that the quotient [|(()))~(>]I| is large, then by increasing K we d(=",2)
are more likely to find a 2² close or equal to 2 causing the difference to grow. Another interesting fact is that less robust samples can by construction be considered natural adversarial examples for a KNN model (Hendrycks et al. (019). Assume for simplicity a classification task with K=1: x and show similar robustness distributions and
among the less robust samples, then there likely exists a sample to E t such that d(z, ta) is small and y(x) # y(ta) (the true labels). However, r and ta being close may cause the prediction yarg min d((a. 2)) to be equal to y(r).
2Ex
EVALUATION
4.1 EVALUATION MODEL
We now determine the robustness distribution of the training- (x) and test data (t) for both CIFAR- 10 and the Online News Popularity Dataset (short: ONP) to cover the classification and regression setup. In each case, we identify subsets of x and t stemming from the extremes of the respective robustness distribution hoping to amplify any possible effects. Then we use deterministic mod- els (K-Nearest Neighbou==KNN) and the non-deterministic models (Convolutional Neural Net- works=:CNNs, Random Forest=:FF) to analyse performance in terms the different subsets of x and t. For this purpose, we measure accuracy and loss that indicate how much and how well model can generalise. "Best" performances are displayed in boldface. The algorithms are noted in B as they are partially based on additional results in A. We also extended the analysis to include the MNIST dataset in F.
The CIFAR-10 dataset (Krizhevsky) 2012)
consists of 60,000 322-pixel RGB-images evenly split into the classes airplane, automobile, bird, cat, deer, dog, frog, horse, ship, truck. Figure 2 provides visual examples. The ONP dataset consists of metadata of over 39,000 articles published by Mash- able [Fernandes] 2015) split into 58 predictive attributes (e.g. "number of words", "data channel", "polarity of positive/negative words".... ....), 2 non-predictive ("URL", "timedelta") and I goal field (" ("shares"). Although the latter is traditionally regarded as regression dataset, one can associate a csssfooclasiificatioon task with it, where we classify popular (2 1400 shares, comp. |ernandes 2 and unpopular articles based on the prediction of shares. For almost all ONP-subsets, the binary labels are equally distributed (rate is 1:1).
For 10 we used KNN classifier and two different CNN architectures, one "small" CNN and the more complex ResNet-56 model (see C.1 for details). For ONP, we used a KNN regressor as well as an RF regressor (see C.2 for details). Neither for CIFAR-10 nor for ONP we used cross-validation due to the high computational effort of computing sample robustness values for each newly formed training set and test set t. One could determine Rxve (2) for all samples beforehand and then apply cross- cross-validatte validation to allow for feasibility, but these robustness values would no longer be independent of each other.
5

---

Under review as a conference paper at ICLR 2021
For the small CNN we first estimated its overfitting threshold for the different training(sub)sets, for the ResNet-56 we measured after how many epochs validation loss did not decrease further (using callbacks). In both cases, used t as the validation set (instead of held-out data) to emphasise the impact of the different robustness distributions in an artificial best-performance setup. The RF regressor we trained using different numbers of trees E 21, 4, 5, 6, 7, 8, 9) and both Mean Abso- lute Error (MAE) and Mean Squared Error (MSE) as loss criteria. The KNN classifier and regressor 15} and both uniform weights ("uni") and weights defined by Euclidean distance || 2l (" ("ddist). The non-deterministic models we trained 25 times and discarded the ten highest and lowest loss- and accuracy values. Finally, we averaged the remaining five to avoid focus individual performances. The details for each model are in C.3.
we constructed for K E 21,
4.2 CLASSIFICATION CENTRIC ANALYSIS (CIFAR-10)
Consider the feature space FS := [0, 1| with Euclidean metric || 1/122 and the CIFAR- 10 dataset of 50,000 training images x and 10,000 test images labelled as k E {airplane, automobile, bird, cat, deer, dog, frog, horse, ship, truck). We regard the label maps as functions y x + R. where (x) is 1 the label of æ is and 0 otherwise. The target space IR is equipped with its Euclidean norm making it Banach space. Using Algorithm 2 (comp. B) determined the robustness of all samples in both x and t, respectively. Afterwards we collected subsets of least- and most robust samples, the distribution of which is shown in Table 1. The subscript defines the subset-size in thousands (number) and whether they belong to the least- (L) or most (M) robust samples of x or t. As an example, XL.40 stands for the 40,000 least robust samples of the training x. The graphs in D.1 compare the label-wise robustness distributions for x and t.
Table 1: Sample distribution per label of all data
Class
x
X1
X1,40
X1,25
Xy 1891
fg_E
tM
f222
fM2
"airplane"
5000
4352
(67%)
3555 (11%)
31057 (62%)
1000
622 (62%)
378
303 (30%)
136 (16%)
"auto"
5000
3171
(63%)
4898 (98%)
982 (2095)
4018
1800
207 (215%)
793
24 (255)
358 (369)
"hind"
5000
4540
3104
(68%)
"cn""
5000
3795 (76%)
4358 (87%)
2153 (43%)
2847
1000
435 (445))
135 (149)
224 (14%)
"dkee""
5000
4704 (94%)
2904 (58%)
3844 (77%)
1156
1800
787 (70%)
213
432 (435))
54 (5%)
"dog"
5000
3701 (74%)
4403 (85%)
2029 (415))
2971
1800
398 (401))
602
123 (129)
281 (28%)
"frug"
5000
(71
5000
3780 (76%))
(41 (65)
2052
1000
387 (705))
(79)
245 (255)
"ship" 5000
4323 (86%%
3927 (79%))
2921 (58%)
2079
1000
567 (57%)
433
207 (2155)
136 (1
"Irack"
5000
3101 (62%%)
4884 (9855)
985 (20%))
4015
1000
213 (215)
789
(39)
356 (36%)
30000
40000
23000
2000
2000
The relative iabel-distiibutions of both x and t are similar judging by the 50%, 20% and 80% quantiles (black percentage values are almost equal, blue and green values add up to about 100%). In Figure. 2 some of the most- and least robust samples in x are displayed together with their robustness value. Here, the values of the least robust samples of "airplane" and "ship" coincide as one is the closest differently labelled image with respect to the other (and vice versa). The same holds for the least robust images of "bird" and "deer". In this classification setup, one can see that close "visual distance" causes low sample robustness the difference of labels is either O. More examples are in D.2.
0.9515
0.9456
0.9461
0,9379
0.7334
0.7334
0.7526
0.7526
Figure 2: Most- (left) and least robust samples in labelled "airplane", "ship", "bird", "deer".
4.2.1 CNNS
The results (LOSS) \ ACC) for the CNNs are displayed in Table 2 4. We noted the number of epochs it took to reach the lowest average loss on t (comp. C.3) and the corresponding performances for all training- and test(sub)sets. Table displays the performance matrix of an alternative session for the small CNN where loss was re- re-weighted to account for class imbalance in the subsets of t. D.3 displays the accuracy and loss curves of a small CNN prototype model trained on x for all test(sub)sets.

---

Under review as a conference paper at ICLR 2021
Table 2: Small CNN performance matrix and number of epochs until overfitting commences.
XL
XM440
XL225
XM225
Lpocts
8.5
80
10
73
75
0,7822
0.6858
0.7766
0.7766
0.8123
0,7973
0.7336
$1.5
0,6970
0.7766
0.6819
0.7772
0.0631
0.7517
0.7570
0.8884
0.7000
0.6609
0.7871
0.6916 \ 0.7764
0.6417
0.7900
0.8808
0.7226
0,7089
0.76676
0.658
0.7799
0.6358
0.7953
0,7578
0,7643
0.6553
0,7839
0.9327
0.6677
M21
0.6684
0.7852
0.7122
0.7708
0.7940
0.942
0.7766
0.7793
Table 3: Small CNN performance matrix with we-hhihhted LOSS (same epochs as in Table.2).
X1,10
XM440
X1,25
X22
Lppccts
80
75
75
0.6807
10,833
0.6912 0
0.6972
0,7763
0.07T
0,741
0,7979
0,73449
$1.5
0.6074
0.7.774
0.6956
0.7626
0.7651
0.77563
0.8571
0.77045
0.7436
0.7580
0,7658
0.7761
0.7900
099466
0.7293
0.8030
0.7660
$1.2
0.7559
0.7927
0.7544
\ 0.7950
0.8335
0.7623
0.8102
0.7816
0.8432
0.6902
M2
0.7943 0.7786
0.8634
\ 0.7688
0.7608
0.7958
1.1018
007777
0.8475
(.77781
Table 4: ResNet-56 performance matrix where loss was lowest on t for the amount of epochs noted.
X1,25
XM25
Lpociss
(13)
(03)
13+1
6.7568
0.8535
0.8466 \ (03316
0.8419 X 0.8308
L05677
0.76663
[,0567 1 0.7656
0.7735
0.8493
0.8404
(08332
0.8811
0088886
1.099
\ 0.7821
1,1431
0.7335)
MM5
0.7397
0.7938
$1.2
0.7236
0.8570
0.7769
0.8453
0.8691
0.8168
0.9177
\ 0.8034
1,1785 0.7193
M2
0.7542
0.866
0.8842
0.8209
0.7971
0.8439
1.1698
\ 0.7390
0.9576
0.8010
Observations (CNNs): performance on t and the number of epochs necessary to reach the least loss thereon decreased both by downsizing x. Training on XL40 (i.e. 80% of the original training set) causes a decrease in performance on tss and tM2, but increases performance on tLs and tL2. It shows the effect of training and testing on subsets which are more aligned in terms of their robustness distribution discussed in 3.5. Similar results hold for XM40- When removing half the samples from x, we can see the trade-off between creating more similar robustness distributions and the overall ability of the model to generalise from its training data. Table 3 reveals furthermore that training on the more robust half of x causes less of a bias: loss on the test(sub)sets is more evenly distributed compared to that of models trained XL25- This effect is likely caused by the model learning to connect small differences in feature space with large differences in label space. Hence, it upscales predictions of more robust samples that further away from the less robust data it was trained on. For the ResNet-56, we can see the same relation between training and performing on subsets of different robustness for the small CNN. However, there no (partial) improvement over the baseline. We attribute this to the different learning structure of a residual neural network for which the trade-off mentioned above may behave differently.
4.2.2 KNN
For KNN, the highest accuracy in every case was observed for K=1 Table shows the performance matrix for all data(sub)sets (note that for K = 1 both weights coincide). D.4 displays the accuracy curves for K E (I,..., 15} and both weights using x, XL25 and XM25 as reference sets.
Table 5: INN accuracy matrix.
x
40
Xy440
X1,25
XM22
0.3539
0.3454
0.3863
0.315
0.3806
tL5
0.3700
0.3702
0.3656
1345
0.3378
0.9166
0.3114
112
0.4095
0.4095
0.3950
0.4095
0.3055
03375
03155
0.3875
0.3410
0.4320
Observations (KNN): the highest accuracy on the most- and least robust test(sub)sets is achieved using the reference sets which are the most similar in terms of their robustness distribution, comp.
We attribute this to the following fact: a large "closest" distance to differently labelled samples does not ensure that there exist close samples of the same label after all. This effect is amplified for image data, where changing the hackground does not necessarily change the label of displayed object, but will increase Euclidean distance significantly.
7

---

Under review as a conference paper at ICLR 2021
3.5. Whereas accuracy on tL2 did not decrease by removing more robust samples, it increased on
M2 when removing less robust samples. This effect and the increased accuracy on by using XL40 attribute to the removal of natural adversarial examples as discussed in 3.6.
4.3 REGRESSION CENTRIC ANALYSIS (ONLINE NEWS POPULARITY DATASET)
The Online News Popularity dataset (Fernandes 2015) was split randomly into training set x and test set consisting of 32,000 and 7,644 samples, respectively. Each sample
is an element in FS := [0, 1 where we use the Euclidean metric || ll2 again. The label map y x + R (where equipped with Euclidean norm as well) assigns to each sample normalised number of shares in [0, 1]. This time we used algorithm (comp. B) to calculate the robustness for each element in x and t (again independently. We then defined different subsets both x and t using the same notation as for the CIFAR-I0 set. The sets and tM consist of the least- and most robust half of t: the sets t and tm consist of the 500 least- and most robust samples of tL and tM. respectively. Figure displays the relative distribution of shares for all data(sub)sets. The black dashed line divides samples into those below 10,000 shares on the left side and those above 10,000 shares on the right (cumulated at 1 10). The values in the middle display the relative amount of samples with shares above this threshold. For most of the data(sub))sets about 5% of the samples have more than 10,000 shares. For both XM2 and tm the relative amount is about 9.4%. Conversely, while 7.6% of samples in XL2 have more than 10.000 shares, this holds for only 0.34% of t (i.c. less than half the relative amount). Indeed, this implicates that the listi-distri of the R 6% least robust samples in x is quite different from that in t. E.1 compares the overall robustness distribution of x and t.
Distribution
Distribution Shares
09693
0535
Xuss GUUSU
6.06
0.00
Kaze: 0-03-99
Nu 00.094
0.06
Kap
Mia 0.016
0.00
0.00
Figure 3: Relative distribution of shares for all training- and test(sub)sets.
4.3.1 KNN
Table 6 shows the performance matrix (LOSS ACC) for uniform ("uni")- and distance ("dist") weights, as well as the particular choice of K where ACC was highest on tt. Table shows the performances and the specific K (in brackets) for which ACC was the highest on each test(sub)set, respectively, E.2 displays the performance graphs
Table 6: "uni" (left) / "dist" (right) weights where ACC was highest on t.
00.4556
0.3900
4..1377
0.3939
-0.915
0.3907
00.1328
0.5899
-0.1903
0.5943
00,114]
0.3917
-0,0869
0.5900
-009660
0.6049
-0.1290
0.5912
tr.
-0.0943
0.59L1
-0.0899
0.6023
-0.4527
0.593]
0033863
0.5900
0.3869
40.2331
0.3871
-0.3315
40.2962
-0,2558
0.5903
0,5900
-0.003
00.1500
.0.1255
0.6320
.0,2217
0.6490
.00544
0.6500
km
-0.1222
0.6280
-0.2224
0.6400
-0,0511
0.6620
as for the KNN classifier on CIFAR-10 10 we can see an improvement on the most robust subsets of t by choosing the corresponding subset XM24- Conversely, this time a similar result holds for the less robust data as well. For both weights, ACC on tm is highest and drops off significantly for tj. This may be explained by the different distributions of the least robust samples
Observations (KNN):
2000
0.0625
500
0.0654
8

---

Under review as a conference paper at ICLR 2021
Table 7: "uni" (left) / "dist" (right) weights where ACC was highest for each test(sub)set.
XL.24
XM424
x
XL24
Xb424
0.1456
(3)
-0.1577
0..959 (4)
0.3945 \ 0.3907 (4)
3.1128
0.559 L3
-0.1303
0,5942 (4)
0.4141 \ 0.591 (4)
000867
0.5947
-0.0940
0.609
-0.4290 \ 0.5942 (4)
-0.0540
\ 0.5945 (4)
+0.0899
0.6023
-0.4527
0.55931 (4)
0,5900
0.5926
-0.3664 \ 0L5882 (3)
tM
-0.3315
0.5887 (3)
0.5939
+0.258
0.5503
0.0045
0.6100 (4)
0.0016
0.6140 (4)
-0.336 \ 0.6020 (4)
0.0040
0.6100 (4)
0.0016
0.6120 (4)
-01500 0.6000 (4)
400278
0.6800 (14)
40.1309
0,0720 (13)
-0.0165
0,6820 (12)
1m
-0,0299
(13)
40.1314 \ 0.0740
-0.0185 0.6840 (12)
in x and t (comp. XL.2 and 6 in Figure 3). The optimal K for each test(sub)set varies between 3 and 4, except for im where it is nearly 3-4 times as high. Moreover (and in contrast to KNN on CIFAR- 10), this causes an increase in accuracy of about 2% -5% in accordance with our theoretic explanation in 3.6.
4.3.2 RANDOM FOREST
Table 8 shows the performances matrix (LOSS) \ ACC) and the number of trees where ACC t was the highest. Table 9 shows the performances and numbers of trees (in brackets) for which ACC
was the highest on each tssb(su respectively. E.3 displays the performance graphs
Table 8: MAE (left) MSE (right) as loss-criteria where ACC was highest on
X1.24
XM24
x
X1,24
XM24
Trees
00.866
0,5951
+12984
\ 0.3907
-0.1760
0.3920
t
-0,1148
0,5953
00.796
00,9909
0,1305
0,3925
-..1776
-0.188
0.5974
fr.
-0,0590
0.5959
0.5968
-0.4611
0.5932
.0,0004
0.5848
.0.4743
0.3864
TM
-0.3814
0.5905
-0,9002
0.5873
-0.3704
0,3884
0.01477
0.6256
000144
\ 0.6232
0.0236
0.6232
0.0194
0.6296
00,0013
0.6276
0,0138
0.6256
-0.3125
0.6564
<0,4946
0.6376
-0.3488
0.6408
1m
-0.3278
0.6616
-0.7771
0.6456
-0.3306
0.6452
Table 9; MAE (left) / MSE (right) as loss-criteria where ACC each test(sub)set is highest.
X1-24
XM24
X1.24
X3124
01887
05951 (51
4,2884
59076600 (4)
4.1760
000900 (3)
-0.1148
0.385 (7)
-0.2796
0.5009
00,300
0.5923 (4)
.020998
0.5973 (3)
-n 15.3
o 59649900 (4)
.0.0006 : 0.5974 (3)
tr.
-0.038
0.6006 (9)
-0.1617
0.5963 (4)
-0.0516
0.9968 (4)
0.4461
0.5922
-0.0000
0.5867 (3)
-0.3172
0.5899 (4)
-0.5619
0.5925 (4)
-0,9052
0.5873
-0.2197
0,5895 (7)
-00198
0.0199
0.6244
0
0.6058 (5)
02000
0.6620 (7)
-0.2708
0.6512 (9)
.0.2054
0.6620 (8)
6m
-0.3217
0.6692 (8)
-0.6496
0.6570 (9)
-0.2361
0.6673
Observations (RF); using 75% of the original training data correlates with a a slight decrease in performance (independent of the metric) and smaller optimal number of trees for XM24 than for XL24- When training on XL24, LOSS on tm and tm increases significantly. This is likely caused by the model learning to connect small changes in feature space with large changes in label space. Hence, it tends to overshoot predictions for more robust data, which is also in accordance with our observations in Table 3. As for KNN, we can see two things: (i) large difference in ACC t and Im and (ii) the optimal hyperparameter being significantly higher for tm- The first we attribute again the different distributions of X1,2 and t;; for the second we expect similar explanation as for KNN (comp. 3.6).
5 CONCLUSION AND FUTURE WORK
We introduced the concept of sample robustness for measuring how sensitive elements of a dataset towards label-changing perturbations. We provided a theoretical motivation and analysed the robustness distribution of different datasets, as well as the connection to model performance. In concordance with our theoretical analysis, we found that it possible to boost performance on spe- cific test(sub)sets by choosing training(sub)sets exhibiting similar robustness distributions. Empiri- cal results, however, indicate that there is model-depent dependdett trade-off between discarding samples to align these distributions and the general ability of model to generalise from its training- or ref- erence data. Finally, we found that optimal hyperparameters may also depend on the robustness of both the training- and test set. Possible future research directions are: (i) expanding experiments (us- ing cross- cross-valide validation, more datasets and models, different metrics, etc.); (ii) analyse optimal model- hyperparameters as functions h(XAB, tAB) in terms of training- (XAB) and test(sub)sets (TAB) of different parts of the robustness spectrum; (fii) explore the relation between training on more- or less e.g. susceptibility to adversarial examples (Szegedy al. 2013).
robust data and

---

Under review as a conference paper at ICLR 2021
REFERENCES
Laura Beggel, Michael Pfeiffer, and Bernd Bischl. Robust anomaly detection in images using ad- versarial autoencoders, 01 2019.
Y. Bengio, Aaron Courville, and Pascal Vincent. Representation learning: A review and new per- spectives. IEEE transactions on pattem analysis and machine intelligence, 35:1798- 1828, 08 2013. doi: 10. 10.1109/TPAM..2013550.
Alexander Bernstein and Alexander Kuleshov. Low-dimensional data representation in data analy- sis. pp. 47-58, 10 2014. ISBN 978-3-319-11655.6 doi: 10.10079978-33319-11666--
Battista Biggio, Igino Corona, Davide Maiorca, Blaine Nelson, Nedim Smndc,, Pavel Laskov, Gior- gio Giacinto, and Fabio Roli. Evasion attacks against machine learning at test time. ArXiv, abs/1108.06131, 2013.
Lawrence Cayton. Algorithms for manifold learning. 07 2005.
Raghavendra Chalapathy and Sanjay Chawla. Deep learning for anomaly detection: A survey. CoRR, abs/1900.0340)) 2019. httphtt:://axttt org/ abs/ /atss//90111 03407
Denis Chigirev and William Bialek. Optimal manifold representation of data: An information theo- retic approach. 01 2003.
S. Cobzaş, R. Miculescu, and A. Nicolae. Lipschitz Functions. Lecture Notes in Mathematics. Springer International Publishing, 2019. ISBN 9783030164898. URL https://books. google.des
Dumitru Erhan, Aaron C. Courville, Yoshua Bengio, and Pascal Vincent. Why does unsupervised pre-training help deep learning? J. Mach. Learn. Res., 11:625-660, 2010.
Ivan Eytimov, Kevin Eykholt, Earlence Fernandes, Tadayoshi Kohno, Bo Li, Atul Prakash, Amir Rahmati, and Dawn Xiaodong Song. Robust physi attacks deep learning models. 2017.
Alhussein Fawzi, Seyed-Mohsen Doossavi-eezfooli, and Pascal Frossard. Robustness of classifiers: from adversarial to random noise. In NIPS, 2016.
Charles Fefferman, Sanjoy Mitter, and Hariharan Narayanan. Testing the manifold hypothesis. Journal of the American Mathematical Sociery, 29, 10 2013. doi: 10.1090/jams/85..
Kelwin Fernandes. A proactive intelligent decision support system for predicting the popularity of 3-1-7-3--19922285---4
online news. 08 2015. doi:
K. He, X. Zhang, S. Ren, and J. Sun. Deep residual learning for image recognition. In 2016 IEEE Conference on Computer Vision and Puttern Recognition (CVPR), pp. 770-778, 2016.
Dan Hendrycks, Kevin Zhao, Steven Basart, Jacob Steinhardt, and Dawn Song. Natural adversarial examples, 07 2019.
Alex Krizhevsky. Learning multiple layers of features from tiny images. University of Toronto, 05 2012.
Yann Lecun, Leon Bottou, Y. Bengio, and Patrick Haffner. Gradient-based learning applied to document recognition. Proceedings of the IEEE, 86:2278 2324, 12 1998. doi: 10.1 726791.
Yanpei Liu, Xinyun Chen, Chang Liu, and Dawn Xiaodong Song. Delving into transferable adver- sarial examples and black-box attacks. ArXiv, abs/161 1.00770, 2017.
Aleksander Madry, Aleksandar Makelov, Ludwig Schmidt, Dimitris Tsipras, and Adrian Vladu. Towards deep learning models resistant adversarial attacks. ArXiv, abs/1706. 060883 2017.
Preetum Nakkiran, Gal Kaplun, Yamini Bansal, Tristan Yang, Boaz Barak, and llya Sutskever. Deep double descent: Where bigger models and more data hurt. In International Conference on Leam- ing Representations, 2020. / http:///wwnneeeeeeeeeeeeeeeeraaa
10

---

Under review as a conference paper at ICLR 2021
Nicolas Papernot, Patrick McDaniel, Somesh Jha, Matt Fredrikson, Z. Berkay Celik, and Ananthram Swami. The limitations of deep learning in adversarial settings. pp. 372-387, 03 2016a. doi: 10.1109/EEmOSPP201...66
Nicolas Papernot, Patrick D. McDaniel, and Ian Goodfellow. Transferability in machine learning: from phenomena to black-box attacks using adversarial samples. ArXiv, abs//6605.077 2016b.
Christian Szegedy, Wojciech Zaremba, Ilya Sutskever, Joan Bruna, Dumitru Erhan, Ian J. Goodfel- low, and Rob Fergus. Intriguing properties of neural networks. CoRR, abs/1312.6199 2013.
Yusuke Tsuzuku, Issci Sato, and Masashi Sugiyama. Lipschitz-margin training: Scalable certifica- tion of perturbation invariance for deep neural networks. ArXiv, abs/1 802.000344 2018.
Beilun Wang, Ji Gao, and Yanjun Qi. A theoretical framework for robustness of (deep) classifiers against adversarial examples, 2016.
Tsui- Wei Weng, Huan Zhang. Pin- Yu Chen, Jinfeng Yi, Dong Su, Yupeng Gao, Cho-Jui Hsieh, and Luca Daniel. Evaluating the robustness neural networks: An extreme value theory approach. ArXiv, abs//801. 10578, 2018
Chen Zhu, W. Huang, Ali Shafahi, Hengduo Li, Gavin Taylor, Christoph Studer, and Tom Goldstein. Transferable clean-label poisoning attacks on deep neural nets, 05 2019.

---

Under review as a conference paper at ICLR 2021
A ADDITIONAL THEORY AND PROOFS:
A.1 PROPOSITION:
Let L(y) be the Lipschitz constant of the label map y x - TS. The following equality holds:
2x (2)) L(y)
Proof. One has L(y) 2 max
(x(x) by construction. Furthermore, as x is compact:
nex
[1(x) y()) [u(())) y(āo)ll
max [(y)
x+2 d(r,3) 3))
d(ro, Fo)
for some 2o + To such that y( + y(a)). Thus:
(2( - y(āo))l
I (x(x)) s max 2x(x) #EX
d( mo)
A.2 PROPOSITION:
(x) is independent of rescaling the label map y.
Proof. Suppose we have label map == ay for E R
{0), then:
1
Rx Rx(x) =
((())
11
x(Tx(a))
d(x,
la| 11y|
Rx(x1) (x)
[g(2)) ( y())
|a| max (
+ |a] ||v||oo
d(x, .)
A.3 PROPOSITION:
Let x E x with reach r(x). (z)(2) e {0, e} for all E x, where lies on the unit sphere in TS (i.e. y is binary label map), then:
[x(2)
Rx(x)
rx +
Proof. It holds that:
[1 y(3))
+ -1 = max
1
+1 -1
Rx(x)=
max
(2))((2)
d(x, t)
(ETx(a)
d(x, σ)
1
+1)
+1
[xxx((z)
min d(x, 5) (x)x(*)
(x)(x))
rx (x))
12

---

Under review as a conference paper at ICLR 2021
A.4 LEMMA:
With the notions of 3.5 it holds:
lu(2) - v(3)1 d(2, 2)
F((2) F(2)
L(F 9) max max sES (=)
u
d(z,8)
Proof. Using the reverse triangle inequality twice:
[1x(2) - y(3)]
[F(22) F(3)
max L
max
d(z,2)
u
d(z, 8)
[1v(2) (())
[F(z) F()
max (())
d(z,
d(z, 2)
[u(=) y(3) F(z) + F(2)
max sET's (x)))
d(z, 2)
To see the first inequality, let X, == (xxx((()) V=( = - (=( = [
d(a.5) and
d(,,5)
note that V [V] and W -W Then, by using the uniform norm X₂ defined as
|Vllo,
max [V(3), seX,
see that the first inequality is indeed consequence of the reverse triangle inequality:
|Vllo, |
I|V
Wll
Taking the maximum over = yields the statement.
B ALGORITHM (SAMPLE ROBUSTNESS)
Let x C FS be a dataset in some feature space with metric and label map y taking values in some target space TS with norm || ||. For sample E FS with label y(z) such that [|v(=)]| S max [|) one can calculate its robustness using the algorithms below. Note that the FEx
second algorithm a special case of the first.
Algorithm Calculate Rx(x) (General y)
Require: #y(x) 2 2, |y/loc =1
4 y(z)
Vj + v(5;)) for any 2j Ex s.t. Vi + y
Q 1 Vill
for x, E x do
Y1 v(I,)
if + yi then
1/ vill
max Q,
d(xr
end if end for
print (Q + 1)
13

---

Under review as a conference paper at ICLR 2021
Algorithm 2 Calculate (x) (Binary y)
Require: y(x) {0, e) with ||e||
< y(x)
r max
for x; do
yi < v(z))
if # ye then
r r minfr, d(r,
end if end for print r
C MODELS
C.I CIFAR-10
SMALL CNN
The small CNN architecture was taken initially from the Keras homepage and consists of two convolutional blocks, each built using two convolutional layers with kernel size (3.3) and RELU activation function, followed by MaxPooling with pool size (2,2) and Dropout ratio 0.25. After this, there two dense layers (512/RELU and 10/Softmax) with Dropout ratio of 0.5 in between. was compiled using the R.M.propo optimizer with learning rate 10-4 and decay of 10-6. We used the categorical cross-entropy our loss function and trained the model with batch size of 32.
RESNET-56
The ResNet-56 architecture (He al., 2016) also taken directly from the Keras homepag We used n=9, batch sizes of 32 and the pre-defined learning rates of 10-3 and 10-4
KNN
For the KNN classifier we used the scikit-learn pre-defined algorithm with both uniform weights ("uni") and weights defined by the Euclidean metric ||2 ("dist").
C.2
ONLINE NEWS POPULARITY DATA SET (ONP)
KNN
For the KNN regressor we used the scikit-learn pre-defined algorithm with both uniform weights ("uni"" and weights defined by the Euclidean metric || ||2 ("dist"). The loss function is defined as
with
E. yt uf
LOSS 1
(-oc, 1].
En MRAN())
where the optimum. Accuracy is based on the pseudo-classificatte task (see 4.1).
'I was recently removed.
http:://ccsssnhhhcceaapacaaaaaao KNe Neeii
.eeaar..niiiie ghbors.
http:///ww....cceeeeeeeeee/eeeee nei ghbora. https://acciiit-1earr
KNe KNe KleiighborsHeeresrrr..
14

---

Under review as a conference paper at ICLR 2021
RANDOM FOREST
To construct random forest regressor we also used the scikit-learn pre-defined algorithm We measured the same loss as for KNN. Again, accuracy is based on the pseudo classification task (see 4.1).
C.3 PROCEDURE DETAILS FOR THE NON-DETERMINISITC MODELS
SMALL CNN
For the small CNN, we trained multiple models for an increasing amount of epochs until could roughly pinpoint the overfitting-threshe Cover on the test set t for every training(sub)set. Then, in second session, we trained 25 independent models (always starting with different weights) for Cover epochs and collected their performances on all of the test(sub)sets t, tLS. tss tL2. t22 in five accuracy- and five loss-lists, respectively. From each list, we discarded the ten highest and the ten lowest values of the 25 and then averaged the remaining five in order to avoid focus on individual model performance. After this we repeated the procedure for 6 == Cover 5k; is E Z, epochs until we found local loss-minimum on t away from the lowest and highest number of epochs in each case. However, it is still possible to fall victim to the stochastic behaviour of neural networks, and in some cases, we repeated second sessions to account for this fact. The optimal number of epochs may vary in #5 vicinity. Also, since couble-descens has been discovered (Nakkiran et al. 2020 one may always question the optimal overfitting threshold.
RESNET-56
The ResNet-56 models were trained 15 times on each of the different training(sub)sets for 80 epochs using a learning rate of 10-3, Here, we used callbacks to save the best weights and identify the particular number of epochs for which loss on was the lowest. We then picked an individual epoch-threshold from (5k outliers). Finally, we conducted second sessions similar to those for the small CNN, where we (i) trained the models for the number epochs indicated by these upper thresholds, (ii) saved the best weights, (iii) rebuilt the models with these weights and (iv) topped everything off with an additional epoch of training using a lower learning rate of 10
N} such that at most of the 15 values lay above (to account for
The epoch-history throughout training the 15 models per training(sub)set is shown below. The number the brackets is the rounded average of the values after we discarded the highest- and lowest three entries.
[13, 17, 20, 12, 12, 15, 15, 18, 13, 19, 12, 23, 23, 17, 8]
(15)
X1.40 [9, 17, 14, 15, 12, 18, 13, 16, 11, 12, 12, 7, 13, 12,9] (13)
X [14, 11, 21, 11,9, 10, 77, 15, 14, 10, 8, 14, 8, 13] (12)
X125: [12, 7, 9, 14, 77,9, 14, 2,,
(11)
X225: [6, 15, 9, 13, 13, 7, 10, 15. 11, 13, 17, 10, 10, (11)
RANDOM FOREST
For every number of trees, we simply used the same second session procedure as for the small CNN, tailored for the ONP training- and test(sub)sets.
bet.e//e//.........emmmebeelelee ennemble.
RandomPorest candom/orrestieereessor html
15

---

Under review as a conference paper at ICLR 2021
D CIFAR-10: 10: ADDITIONAL MATERIAL
D.1 LABEL-WISE ROBUSTNESS DISTRIBUTION
Clecp witee st strubution RDd Ex
'Sear"
Avo2ogg
Rid Q- REO where 1o 2001 digteyyed
Class- vilue Dist Wottion of N6d &
1a "pppeee""
'Sear"
000
Qu-t Rbd whert 2001 dagleyyd
Claso stribution RGd For
'123'
0.00
20
20
60
300
2d0 Rost i2 1001 THE Bist i 10td where 10.0 dopfersd
Distiiuuton R&d
thog"
0.00
0.00
25
39
200
Riel 12 10 mec Rbx 1000 W C.0 100) a duptayad
Figure 4: Relative class-wise sample robustness distribution for and t.
16

---

Under review as a conference paper at ICLR 2021
D.2 LEAST- AND MOST ROBUST SAMPLES IN THE CIFAR- 10 TRAININGSET
Below is the juxtaposition of the least- and most robust samples in x for each class together with their respective sample robustmess and the average model output probability (in the brackets) of each image representing its respective class. Here, the average is determined using the small CNN and the same second session procedure deseribed in C.3. Notably, the background of each animal object displayed in the image impacts robustness as differently coloured surroundings will significantly increase distance w.r.t. ||2- Comparing the robustness and average model probability for the "bird"- and "frog" images, there seems to be no definitive relationship between those values.
0.9515 (0.99168)
0.9463 (0.9057)
0.9461 (0.6126)
0.9461 (0.8088
0.9379 (0.7343)
0,9457 (0.9394)
0.9428 (0.3835)
D 0.9444
0.9456 (0.9770)
0.9449 (0.9966)
8-46)
0.7526 (0.8775)
0.7526 (0.6538)
0.8193 (0.2691)
0.7334 (0.7865)
0.8497 (0.0219)
(0.6791)
(0.1900)
(0.0389)
(0:4009)
(0.3291)
Figure Most- (upper row) and least robust samples (lower row) the CIFAR-10 training set.
D.3 PROTOTYPE MODEL LOSS AND ACCURACY OVER 85 EPOCHS
We trained single small CNN (see C.1) on x for 85 epochs (which approximately showed the average performance in Table 2) and monitored its performance throughout the process. The graphs below show the loss and accuracy values for all test(sub)sets. From the beginning. the model did perform better on the more robust half of the test set ts5 than on the less robust half tLs. Loss on tL2 is almost always the least of all five, though this does necessarily cause high accuracy. Indeed, the subset on which the model has the highest accuracy changes every few epochs from t tM2 and vice versa throughout the whole training process.
44
Figure 6: Epoch-wise learning behaviour of a small CNN trained on the CIFAR-10 training set x approximately expressing the loss in Table 2.
17

---

Under review as a conference paper at ICLR 2021
0.4
0.80
6.1
Figure 7: Epoch-wise learning behaviour of a small CNN trained on the CIFAR-10 training set x approximately expressing the accuracy in Table 2.
D.4 KNN
3.6%
343-
339
K55
513
340
X35
322
Figure 8: KNN for CIFAR-10, split for each test(sub)set. The colour of the graph corresponds to the reference training set.
ONP: ADDITIONAL MATERIAL
E. RELATIVE ROBUSTNESS DISTRIBUTION
The minimum- and maximum sample robustness for and are:
min R(x) 2626 ZEx
R(x) 0.7616, min R(x) 0.3792,
R(x) 0.8195
SE*
Figure displays the relative sample robustness distributions. Overall, they exhibit very similar behaviour (the peak for is slightly higher).
18

---

Under review as a conference paper at ICLR 2021
0110
6.00
2.4
322
902
20
Figure 9: Relative robustness distributions for x and t. The green dashed line marks the 65% threshold. For 1272 samples are below this robustness level (=c 4%); for t, 267 (=2 3.5%).
E.2 KNN
LHOOW
To 1.00
-24
E:PC
6 1100551
Ln (0055)
cirt
-2.-
11400)
(400)
0.50
090
6990
699
056
659
030
6:0
0355
631
630
0.79
095
142
306
3.5
317
1211 1040
Figure 10: KNN LOSS and ACC for ONP, split for each test(sub)set. The colour the graph corresponds to the reference set.
19

---

Under review as a conference paper at ICLR 2021
E.3 RANDOM FOREST
-12
-15-
-06
-011
1907
-15
Yout 1907)
XIMAD hast
3350
5430
5910
3500
XIRAD
3375
$3.00
KINAO
1550
COC
Figure 11: RF LOSS and ACC for ONP, split for each test(sub)set. The colour of the graph corre- sponds to the reference training set.
F MNIST
The MNIST data set (Lecun et al.) 1998) consists of 60,000 training- (x) and 10,000 test- (t) 282. pixel greyscale images of handwritten digits (we used the same mathematical notions as for the CIFAR-I0 set). Table 10 shows the label-distributions of different data(sub)sets (again, the notation the same for the CIFAR- 10 set). The relative label-distributions of both and t are similar judg- ing by the 50%. 20% and 80% quantiles (black percentage values are almost equal, blue and green values add up about 100% except for class "1"). Overall, images displaying the digits "1". "4" "7" and "9" were very prominent in X1.48 X1.30 trs, tr., whereas images in XM48. X tM5, tM2 mainly consist of "0"s. "2"s and "6"s.
20

---

Under review as a conference paper at ICLR 2021
Table 10: Sample distribution per label of all data(sub)sets,
Dight
x
XL48
Xb149
XL.00
Xb490
Fg.5
EM5
f1.2
f2:22
"0"
5923
26.22 (6316)
5853 (9976)
0.52
5251
960
541 (100)
336
(016)
582 (5716)
"1"
6742
67.99
(100%)
2750 (115)
6702 (99%)
1135
1125 (998)
896 (79%)
(09R)
(14%6)
14.(118)
"5"
6131
(8218)
5828 (95%)
2219 (36%)
3912
1010
393 (39%)
617
34 (395)
161 (18%)
"4"
5842
5353 (915))
4310 (356)
3819 (6558)
2023
982
(56 (0756)
326
256 (289))
91 (955)
"5"
5421
5154
(35963
"y"
0265
5862 (949))
4257 (08963
2506 (7556)
1899
1028
743 (7216)
285
289 (2896)
(5%)
"8"
5851
4832 (8316)
5577 (9516)
2185 (3756)
3006
974
387 (6016)
587
22 (218)
155 (1015)
"
59-19
5779 (9756)
2934 (19%)
5183 (87%)
736
859 (8685)
471 (47%)
(3%)
60000
48000
45000
30000
10000
5000
2000
2000
The authors of Papernot et al. (016a) noticed that when crafting source-target pairs of adversarial examples from the MNIST test set, "classes "0", "2" and "8" are hard to start with, while classes "I", "7" and "9" are easy start with". Interestingly, comparing the compositions of the 2000 least- and most robust samples we can see that the three most prominent classes in tL2 were "I", "7" and "9" (and close thereafter "4"), while the two most prominent classes XM2 were "0" and "2" (followed by "6" and "3" before "8").
Figure 12 displays the most and least robust samples of each class together with their sample ro- bustness and the average probability of shallow CNN with average accuracy of about 98% (25 models were trained independently and accuracy values were collected similarly to the second ses- sion for the small CNN CIFAR-10). The boldness of the written digit has beneficial impact on its robustness as will increase Euclidean distance significantly. We can also see that the least robust image of "3" presents itself as mislabelled "9". Indeed, this makes sense as distance in feature space to another image of a "9" in x is relatively small.
1
2
3
H
S
Lo
7
e
on
0.9086 (1.0000) 0
0.8780 (0.9987)
0.9031 (1.0000)
0.8951 (1.0000)
0.9060 (1,0000)
0.8979 (1.0000)
O. 0.9025 (1.0000)
0.8973 (1.0000)
0.9031 (0.9984)
0. 0.9009 (0:9707)
9
4
5
C
7
P
9
0.7984 (0.7363)
0.7877
0.7389 (0.9940)
0.7561 (0.7781)
0.7058 (0.7590)
0.7635 (0.0212)
0.7389 (0.9996)
(0:9114)
(0.7235)
(0.0053)
(0.9764)
Figure 12: Most- (upper row) and least robust (lower row) samples of each label in the MNIST training set ("0" -7"" from left to right).
One may be tempted regard sample robustness tool of anomaly detection (Chalapathy & Chawla 2019; [Bggel et al.) 2019) as it rightfully identifies the mislabelled image of a "9". To underline that is utterly intrinsic and metric-dependent concept, consider the set of "handwritten digits" in Figure 13 and their respective robustness values (note that the image of the "4" is not missing but plain white). The relatively high value of the "3" being "4" may be caused by the missing normalization of size and position (as was done for the MNIST set).
4
VII
2
NL (9)
0.9028 (0)
0.9139 (1)
0.8923 (2)
0.8071 (3) (.7812
0.9441 (5)
0.8984 (6)
0.9139 (7)
0.9152
Figure 13: Robust samples of "handwritten digits" labelled as the number in the brackets.
21


Reviews:
[{"review_id": "zblfXUpg262", "reviewer": "AnonReviewer3", "summary": "", "questions": "", "limitations": "", "rating": 4, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "Overview: \n\nThe authors introduce sample robustness, a pointwise measure of the sensitivity of the label map to perturbations in the feature space. They do so by taking the pointwise Lipschitz constant of the label map and normalizing it by the label norm. The authors show that training on samples that are / are not sensitive helps on a test set that is / is not sensitive. \n\nPositives:\n\nThe authors do extensive experimentation to evaluate the effect of training with various data subsets constructed by taking the most or least sensitive samples, and testing on other subsets of sensitive / not sensitive samples.\n\nThe results generally show that performance on non-sensitive subsets improves when training with non-sensitive data.\n\nNegatives:\n\nI'm not terribly convinced by the experimental results. It seems like the conclusion of the paper is that if the goal is to get good performance on the standard test set, you should just use the original data without taking into account sample robustness.\n\nThere are gains for specific subsets, but the paper doesn't really do a good job of convincing me why I should care about this. It seems like the end goal should be performance on whatever test set I originally collected, and not a subset where you remove particular hard or easy examples. This concern is exacerbated by the fact that the train and test data selection procedures are matched (i.e. you select points based on sample robustness and evaluate on similar selection criteria). \n\nFinally, I'm not sure if this notion of sample robustness is conceptually novel compared to methods like adversarial pruning (Yang 2019). The authors could have done more in linking the effects of sample robustness to the margin or generalization properties of the resulting classifier. For an example of such discussion see the recent line of work on r-consistency and nonparametric methods for adversarial robustness.\n\nFinally, the writing style is generally convoluted and requires substantial effort to parse. The abstract sentence starting with \"In this work\" and first sentence of the intro are both examples of this. The problem is too pervasive for me to give exhaustive pointers, but the paper requires substantial copyediting before it is accepted or published.\n\nMinor / trivial: \n\nAbstract - \"In this work ...\" has way too many commas.\nPlease use proper left quotes for Latex.", "extra_scores": {}, "extra_text": {}}, {"review_id": "y7-Tb_X6YI2", "reviewer": "AnonReviewer1", "summary": "", "questions": "", "limitations": "", "rating": 4, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "Summary:   \n\nThis work introduces the concept of sample robustness – based on computing the pointwise Lipschitz constant of a data point – and use it to empirically analyze the effects of training on least and most robust training subsets on the performance for different models. This is done for both classification and regression setups. It is shown that the model performance can be sometimes improved by choosing particular training subsets and hyperparameters depending on the robustness distribution of the test (sub)sets.\n\n-------------------------------------------------------------------\nPros:\n\n-\tThe problem studied is interesting and is very relevant from a practical point of view.\n-\tThe paper is structured well, and the problem has been motivated well in the introduction.\n-\tThe related work and experiment results sections are quite thorough, and the empirical results are interesting.\n\n------------------------------------------------------------------\nCons:\n\n-\tMy main concern is that the definition of the sample robustness (in Section 3) seems overtly simplistic and ad-hoc. I understand the underlying reasoning behind the choice, but one could also consider many other definitions. It would have been more satisfactory had some supporting theoretical results been provided for the proposed notion.\n-\tThe grammar and quality of writing can be improved at many places.\n-\tIn Table 3, it seems that training on the complete training set gives the best loss/accuracy when tested on the complete test set (compared to when we train on subsets of the data). But then, what is the benefit here of training on a (least/most robust) subset of the training data? No explanation is provided in this regard.\n-\tIn Table 4 as well, downsizing x drastically increases the loss on the complete test t. However, in Table 6 (KNN), choosing the 40,000 most robust training samples gives the best accuracy on t. There is no explanation provided as to why this is the case.\n\n----------------------------------------------------------------------\nFurther remarks:\n\n-\tIn Section 4.2: 50.000 --> 50,000 (same for 10.000).\n-\tIt will be helpful for the reader if Algorithm 2 were in the main text.\n-\tIn the experiments, it is not clear to me why the least and most robust test subsets are considered. In practice, we do not have access to the test data so we cannot process it.  \n\n--------------------------------------------------\nPost rebuttal:\nI have read the authors response and appreciate the effort made to improve the paper. But I think the results still need additional work, especially from the theoretical front. So my original score is unchanged.", "extra_scores": {}, "extra_text": {}}, {"review_id": "o5r4cnxfk0z", "reviewer": "AnonReviewer2", "summary": "", "questions": "", "limitations": "", "rating": 5, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "This paper proposes a novel concept of sample robustness, which is used to find tailored training(sub)sets and hyperparameters depending on the robustness distribution of the test(sub)sets for boosting model performance. The motivation is straight forward and the proposed concept of sample robustness is inspiring. However, the works in this paper do not match with the announced contribution. Hence, I vote to reject.\n\nReasons for scores:\n1，this paper announces that the novel concept of sample robustness can be used to choose training subsets and boost the model performance. In addition, some simple theoretical analysis about sample robustness are also given. But, this paper just gives some simple results of theoretical analysis and have no any explanation about those analytical results. More significantly, those analysis have little relationship with the goal of this paper. So this paper lacks some key theoretical analysis about sample robustness for its advantages.\n\n2，The main works of this paper is based on empirical studies. Because of the lack of necessary theoretical analysis, more datasets are needed for verifying the behavior of model based on the concept of sample robustness. However, the datasets in this paper are too few to guarantee the good behavior of proposed technique in orther datasets.\n\n3，this paper does not analyze the pros and cons of proposed technique based on the concept of sample robustness.\n\n4，Some irregular use of mathematical symbols and grammatical errors.\n\nOverall, the works in this paper do not support the announced contribution\n", "extra_scores": {}, "extra_text": {}}, {"review_id": "8uA9pstvjAj", "reviewer": "AnonReviewer4", "summary": "", "questions": "", "limitations": "", "rating": 2, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "This work investigates how to choose the right training set and hyper-parameter for a test set. The authors claim that they introduce a concept of sample robustness based on the Lipschitz constant of the label map. The authors then empirically evaluate the robustness distribution of two datasets and investigate the model performance when using training subsets from different parts of the robustness distribution.\n\nPros:\n- The idea of robustness distribution for a dataset seems interesting. It might be used to compare different datasets.\n- The authors provide extensive experiments using different models. It is nice to see the details for each model and the robust samples for each dataset.\n\nCons:\n- The novelty of this work is somewhat limited. The notion of sample robustness that the authors propose is just a minor modification of Lipschitz constant. Moreover, there is no theoretical or empirical justification regarding why this definition of sample robustness is useful.\n\n- Despite that I really appreciate the efforts that the authors have put on the experimental section, I think that the contribution to the community is somewhat weak. This paper just uses the so called sample robustness to select a subset of the training sample and then reports the performance of model trained with that subset on some test data. But the authors do not provide any insights on how it works. In fact, from the results, it is still not clear how sample robustness can help choose the right training set and improve the model performance. \n\n- There are some typos. For example, in the nine line of abstract, “be regarded as an intrinsic” should be “be regarded as being intrinsic”; in the related work, when citing multiple work, it is better use the command “\\citep{work1, work2}”.", "extra_scores": {}, "extra_text": {}}]

Author rebuttal:

Meta-review: The paper proposes sample robustness (a data-dependent measure) which is essentially a point-wise Lipschitz constant of the label map. The measure is used to choose a subset of training data for training and it measures how small of a perturbation is required to cause a label change w.r.t. label map.
initially, the paper lacked theoretical motivation and backing and the empirical studies were limited to be convincing enough. The authors added additional theoretical explanations. There were some mathematical mistakes that were fixed in the revision. 
However, that is not enough to justify the proposal fully. Therefore, I suggest the authors improve the theoretical explanation. The paper would also benefit from more empirical analysis and discussion. As is, the paper has limited significance to the community since the conclusion is not convincing enough.


The paper writing quality although improved from the original version, still has room for improvement. 

The proposed measure is simple, which can be a plus. but that means that we are also missing on some relationships and interactions between samples impact on training. Therefore, The paper will benefit from clearly discussing pros and cons of the proposed method. Moreover, discussing how this definition works in choosing the best subset of samples will improve the paper. 

i thank the authors for their effort and improving the paper in response to the reviews. However, given that myself and reviewers find the modifications enough for addressing all the concerns, I vote for rejecting the paper. Please improve the paper and resubmit to a future venue.
Decision: Reject