Adversarial Filters of Dataset Biases
Authors: Ronan Le Bras
Swabha Swayamdipta
Chandra Bhagavatula
Rowan Zellers
Matthew Peters
Ashish Sabharwal
Yejin Choi
Venue:  (2020)

Abstract: Large-scale benchmark datasets have been among the major driving forces in AI, supporting training of models and measuring their progress. The key assumption is that these benchmarks are realistic approximations of the target tasks in the real world. However, while machine performance on these benchmarks advances rapidly --- often surpassing human performance --- it still struggles on the target tasks in the wild. This raises an important question: whether the surreal high performance on existing benchmarks are inflated due to spurious biases in them, and if so, how we can effectively revise these benchmarks to better simulate more realistic problem distributions in the real world.  
In this paper, we posit that while the real world problems consist of a great deal of long-tail problems, existing benchmarks are overly populated with a great deal of similar (thus non-tail) problems, which in turn, leads to a major overestimation of true AI performance. To address this challenge, we present a novel framework of Adversarial Filters to investigate model-based reduction of dataset biases. We discuss that the optimum bias reduction via AFOptimum is intractable, thus propose AFLite, an iterative greedy algorithm that adversarially filters out data points to identify a reduced dataset with more realistic problem distributions and considerably less spurious biases. 
AFLite is lightweight and can in principle be applied to any task and dataset. We apply it to popular benchmarks that are practically solved --- ImageNet and Natural Language Inference (SNLI, MNLI, QNLI) --- and present filtered counterparts as new challenge datasets where the model performance drops considerably (e.g., from 84% to 24% for ImageNet and from 92% to 62% for SNLI), while human performance remains high. An extensive suite of analysis demonstrates that AFLite effectively reduces measurable dataset biases in both the synthetic and real datasets. Finally, we introduce new measures of dataset biases based on K-nearest-neighbors to help guide future research on dataset developments and bias reduction. 

Under review as a conference paper at ICLR 2020
ADVERSARIAL FILTERS OF DATASET BIASES
Anonymous authors Paper under double-blind review
ABSTRACT
Large-scale benchmark datasets have been among the major driving forces in AI, supporting training of models and measuring their progress. The key assumption is that these benchmarks are realistic approximations of the target tasks in the real world. However, while machine performance on these benchmarks advances rapidly often surpassing human performance it still struggles on the target tasks in the wild. This raises an important question: whether the surreal high performance on existing benchmarks are inflated due spurious biases in them, and if so, how we can effectively revise these benchmarks to better simulate more realistic problem distributions in the real world.
In this paper, we posit that while the real world problems consist of great deal of long-tail problems, existing benchmarks are overly populated with a great deal similar (thus non-tail) problems, which in turn, leads to major overestimation of true Al performance. To address this challenge, we present a novel framework Adversarial Filters to investigate model-basee reduction of dataset biases. We dis- cuss that the optimum bias reduction via AFOPTIMUM is intractable, thus propose AFLITE, an iterative greedy algorithm that adversarially filters out data points identify a reduced dataset with more realistic problem distributions and consider- ably less spurious biases.
AFLITE is lightweight and can in principle be applied to any task and dataset. We apply to popular benchmarks that are practically solved - ImageNet and Natural Language Inference (SNLI, MNLI, QNLI) - and present filtered counter- parts as new challenge datasets where the model performance drops considerably (c.g., from 84% 24% for ImageNet and from 92% to 62% for SNLI), while human performance remains high. An extensive suite of analysis demonstrates that AFLITE effectively reduces measurable dataset biases in both the synthetic and real datasets. Finally, we introduce new measures of dataset biases based on K-nearest-neighbors to help guide future research dataset developments and bias reduction.
INTRODUCTION
Large-scale neural networks have achieved superhuman performance across many popular Al bench- marks, for tasks as diverse as image recognition (ImageNet; Russakovsky al. (0155). natural lan- guage inference (SNLI: Bowman et al. (2015)), and question answering (SQuAD; Rajpurkar et al. (2016). Yet these deep models struggle when taken out of these dataset environments and evaluated on adversarial data or problems in the wild (Eykholt et al., 2018 Jia & Liang, 2017). This raises a key question: Does high model performance on today's 's benchmark datasets indicate the underlying task is solved, or do those datasets overestimate the true capabilities of current Al systems?
Answering this question is key because benchmarks serve important roles the community. Not only do they direct progress on core tasks, they also make it easier to tackle the lofty target tasks such as image recognition in the wild through a more practically-scopet dataset such as ImageNet. How- ever, the closed-wo world assumption of most existing datasets is subject to significant bias (Torralba) & Efros 2011). Much of the data that is easy to obtain and label isn 't necessarily representative of the task seek to measure. Thus, if left unchecked, artifacts from data collection (Fouhey et al. 2018) or human labeling (Gururangan et al., 2018 Poliak et al. [0188) Tsuchiyal 2018) Geva et al. 2019) can significantly inflate model performance. Though there exist task- and dataset-specificc ap-

---

Under review as a conference paper at ICLR 2020
manara by APLls
07
24
37
ducc theaniy ArLa
thibudos bydudaday
-1
10
03
61
-1
0. 1.0
Figure I: Random ImageNet images for two labels - Monarch Butterfly and Chickadee that were either selected (left) adversarial by AFLITE algorithm, excluded (right). The heatmap shows pairwise cosine similarity between EfficientNet-B7 features (Tan & Le 2019). The AFLITE images show significantly greater diversity such the cocoon of butterfly, or the non-canonical chickadee poses that is in turn reflected by the cosine similarity. This diversity suggests that the AFLITE examples more directly measure progress on the true task of image classification, versus fitting to dataset bias.
proaches for addressing these biases (Goyal et al., 2017 Geirhos al. 2018), the complex artifacts that emerge from large-scale dataset creation are challenging to exhaustively identify and remove.
In this paper, we present AFLITE computationally efficient dataset reduction algorithm, aimed at systematically reducing spurious artifacts in dataset. AFLITE is general and can be applied to any task and dataset. Our approach leverages high capacity model learn dataset specific biases small subset, then uses to identify and filter artifact-pront instances in the remainder of the dataset to yield final dataset that is possibly closer to the intended task.
We first evaluate the effectiveness of our method synthetic data and show that AFLITE lowers the performance of models relying on annotation artifacts while preserving the performance of models whose representation captures the underlying tasks. In addition, while AFLITE aims to retain the more challenging, confusing instances, our experiments show that can successfully remove biased instances that adversarial the correct representation of the data.
Finally, we apply the method to several benchmark datasets across various tasks and domains. In language understanding,
apply AFLITE the SNLI (Bowman al. 2015) and MNLI (Williams) et al. 2018) datasets for natural language inference, and to QNLI (Wang et al., 2018) for question answering. We show a 30% absolute gap in performance in the current state-of-art methods before and after AFLITE. In computer vision, AFLITE reduces the performance of image classification neural methods on ImageNet (Russakovsky al., 2015), showing a 49% absolute gap.
DATASET REDUCTION FOR REPRESENTATION-BILE MINIMIZATION
In this section, we introduce AFLITE, a general approach for reducing the scope of bias in datasets. Large datasets the risk of prioritizing performance the data-rich head of the distribution, where examples are plentiful, and discounting the tail. Our goal is to minimize the ability a model exploit biases in the head of the distribution, while preserving the inherent complexity of the tail.
Let & represent feature representation, defined over dataset D (X, Y). With AFLITE, we seek a subset S с D of size |S| 2 that is maximally resilient to the features uncovered by Φ. For any identically-distree -istriibt train-test split of D, the features extracted by & should not generalize the held-out set. Our approach allows for any choice of feature representation.
Formalization More formally, M denote family of classification models (e.g., logistic re- gression, SVM, particular neural architecture) that can be trained subsets of D (X, Y)
2

---

Under review as a conference paper at ICLR 2020
using features (X).. We define the representation bias of 4 in S w.r. M. denoted R(D, M,M), as the best possible sample classification accuracy achievable by models in M when predicting the true labels Y using features (.)). For given target reduced dataset size of at least n. the goal to find a subset 5 C D, |S| 2 n that minimizes this representation bias in S w.r.t. M:
min 1812R
SMM)
(1)
Eq. (1) corresponds to the optimum bias reduction, referred to as AFOPTIMUM. R( S,M can be formulated as the expected classification accuracy resulting from the following process. Let 2 + [0, 1] be probability distribution over subsets T = "X".Y", of D. The process is to randomly choose a subset T with probability (T)). train a bias estimator Mr E M on D T. and evaluate its classification accuracy far, (#(X"), Y") T. Note that the resulting classification aceuracy on itself is a random variable, since the training set
is random. We define the expected value of this classification accuracy to be the representation bias:
R R M. & Er~g (*("), )
(2)
While this expression formalizes the intended objective function, it involves large summation over subsets T C S just to compute the representation bias present in single set S. It does not suggest a practical way to compute the minimization in Eq. 4 without further considering each of the exponentially many subsets S C D individually thus an optimal solution for Equation (2) intractable. To get around this difficulty, we reformulate the representation bias in as a sum factored over the [S] individual instances i S. This will allow to efficiently decide whether or not to include in the targeted, reduced subset we are constructing.
The idea is aggregate the contribution of each i towards the representation bias expression across all random choices of the training set D T. We call this the predictability score p(i) for i: on the label y, be predicted using features ((x;) when model from M is
average, how reliably trained on a randomly chosen training set D \ T not containing i. The higher the value of p(i), the easier it is to correctly classify the instance (Iii, yys using model family M. This is the signal we will use to decide whether to include in the reduced subset S we are constructing.
With some abuse of notation, for i E D, let ((1) - Er>: (T)) denote the marginal probability of choosing a subset T that contains The ratio 2(T) is then the probability of T conditioned on containing i. Let far (#(z),, y:) be the classification accuracy of Mr on i. The reformulation of
9(8)
representation bias in terms of predictability scores of individual instances works as follows:
Er-g [ (D(XT),
Σ Q(T) TCS
Σ SET
fuy (9(x)).))
IT]
fMy (+(x)) v6)
(T))
[T'
TCS iET
fMy (#))), v6)
Σ Σ
((T).
[T]
iES TCS
((T)
fay (z(s).. vs)
Σ g(i)
qii)
[
IE5
TCB
T5i
S'y (1)(2)) vi)
Σ
g(i)
Eres. T3
Σ (())
SES
where p(i) is the predictability score of i defined as:
SM (x::) v.)
p()) e q(1) Eres.
(3)
[T]
3

---

Under review as a conference paper at ICLR 2020
While the method works for any probability distribution q with con-zero support on all samples, for
simplicity of exposition, we restrict 9 to be the uniform distribution over all subsets T C S of fixed
size. This makes both |T] and (i) fixed constants; in particular, g() = ( / (1) (1))
[2]
This
[8]
reduces the predictability score expression of Eq. ( to the simplified variant p(i):
1 p(t) _ |S|
Trc (*(z)) v4)l
(4)
Putting the pieces together, we have a factored reformulation of the representation bias in Eq. 2
R(),
Σ p(i) IES
Armed with this factored representation, we return the task of identifying an S |S| 2 n that minimizes the representation bias. We use the simplified predictability scores (henceforth simply referred to as predictability scores) heuristic metric decide which i D include in S. We consider three approaches that iteratively filter out the most predictable instances from D to arrive at S. In all cases, we use fixed training set size [S T = < n. Further, since a larger filtered set generally desirable, we terminate the filtering process early (i.e., while |||| > n) if the predictability score for every i falls below a pre-specified early stopping threshold T E [0,1].
The three approaches are as follows. (A) A simple greedy approach starts with the full set S = D, identifies an E S with the highest predictability score, removes it from S, and repeats up to DD|-n times. (B) A greedy slicing approach identifies the instances with the k highest predictability scores, removes all of them from S, and repeats the process up to [ times.
(C) A slice sampling approach where, instead of greedily choosing the top & instances, it randomly samples l instances with probabilities proportional their predictability scores In this paper, we use the greedy slicing approach in our experiments and refer to it as AFI.TE. While the optimum bias reduction via AFOPTIMUM is intractable, its weight version, AFLITE, is applicable in practice.
The slice sampling approach can be efficiently implemented using what is known as the Gumbel method or Gumbel trick (Gumbel& & Lieblein 1954; Maddison et al.] 2014), which uses random perturbations to turn sampling into a simpler problem of optimization. This has recently found suc- cess in several probabilistic inference applications (Kim et al., 2016 Jang et al. 2016 Maddison ee al., 2016 Balog et al. 2017: Kool et al. 2019). Starting with the Dog-preiictability scores log p(t) for various 7, the idea is to perturb them by adding an independent random noise y drawn from the standard Gumbel distribution. Interestingly, the maximizer i" of 7/ +log p(() turns out be exact sample drawn from the (unnormalized) distribution defined by p. Note that i" is a random variable since the %/ are drawn at random. This result can be generalized (Vieira 2014) for slice sampling: the & highest values of Guumbel-perturber replacement, items from the probability distribution defined by p. The Gumbel method is typically applied to exponentially large combinatorial spaces, where it is challenging to scale up. In our set- ting. however, the overhead minimal since the cost of drawing a random % is negligible compared to computing p(1).
fog-predictability scores correspond to sampling, without
Implementation Algorithm I provides an implementation of AFLITE, The algorithm takes as input a dataset D= (X,Y), representation ((X)
interested in minimizing the bias in, a model family M (e.g.. linear classifiers), a target dataset size n, size m of the support of the expectation in Eq. 4.. training set size / for the classifiers, size k of each slice, and early-stopping filtering threshold r. Importantly, for efficiency, ((X) is provided to AFLITE in the form of pre- computed embeddings for all of X. To obtain (xX) practice, we train a first model on a small fraction of the data based on the learning curve in low-data regime, and do not reuse this data for the rest of our experiments. Moreover, this fraction corresponds to the training size for AFLITE and it remains unchanged across iterations. We follow the iterative filtering approach, starting with D and iteratively removing some instances with the highest predictability scores using the
All of these approaches can be improved further by considering not only the predictability score of the top-k instances but also (via retraining without these instances) how their removal would influence the pre- dictability scores of other instances in the next step. We found our computationally lighter approaches to work well even without the additional overhead of such look-ahead.
4

---

Under review as a conference paper at ICLR 2020
Algorithm 1: AFLITE
Input: dataset (X, Y). pre-computed representation (xX),, model family M, target dataset size n1, number of random partitions m, training set size < n, slice size S n, carly-stopping threshold
Output: reduced dataset 5
5=
D
while [S] do
// Filtering phase
3
forall E do Initialize multi-set of out-of-sample predictions [ii
for iteration j I..m do
Randomly partition
(T), S 23) s.t. IS Til =t
7
Train classifier L E M on ((((x)))) (x,y) E E (T) (C is typically linear classifier)
forall i = (zz, y) E Tj do
Add the prediction (x)( F(i)
10
forall = (x,,)) y) S
Compute the predictability score p(1) K E (()) s.t. y = /
12
Select up to instaacess S" in S with the highest predictability scores subject to p(1) 2
S 5= 5 \ S
14
[S'] ls then
15
break
16
return S
greedy slicing strategy. Slice size k and number of partitions m are determined by the available computation budget.
At each filtering phase, we train models (linear classifiers in our implementation) on m different random partitions the data, and collect their predictions on their corresponding test set. For each instance i, we compute its predictability score as the ratio of the number of times its label is predicted correctly, over the total number of predictions for it. We rank the instances according to their predictability score and use the greedy slicing strategy of removing the top-k instances whose score not less than the early-stopping threshold т. We repeat this process until fewer than instances pass the r threshold in filtering phase fewer than n instances remain.
EXPERIMENTAL ANALYSIS
We evaluate AFLITE across various domains (synthetic, natural language processing, computer vi- sion), different tasks given domain (language inference and question answering NLP), differ- ent datasets for a given task (SNLI and MNLI in natural language inference), and different repre- sentations for given dataset (pre-computed embeddings from ESIM+GLoVe, BERT, RoBERTa for the SNLI dataset).
3.1 SYNTHETIC EXPERIMENTS
We demonstrate the utility of AFLITE in synthetic data setting. Our dataset consists of two- dimensional data, arranged in concentric circles, at four different levels of separation, as shown in the Figure? 2 As is evident, linear function might not adequate for separating the two classes; requires more complex non-linear model such as an SVM with an RBF kernel.
We add class-specifis artificially constructed features (artifacts) sampled from two different Gaus- sian distributions. These features are only added to 75% of the data in each class, while for the rest of the data, we insert random (noise) features. These artifacts make the task solvable through a linear function. Furthermore, for the first dataset, with the largest separation, of some examples with artifacts, making the data slightly adversarial even to the RBF. Both models clearly leverage the artifacts, and demonstrate improved performance artifacts.
flipped the labels
baseline without
We standard implementations from scikit-leam: https::/ kit 1ic-leerrn hatt-llarn.org//tabbl
5

---

Under review as a conference paper at ICLR 2020
Model
D
Dy2k
D(Los
D(#PaRt)
D D(OROMERT)
ELIM+ELMo (Peters
2018
88.7
86.0
61.5
54.2
51.9
BERT (Devlin
2019)
91.3
87.6
74.7
61.8
57.0
RoBERTa Liu et al
2019b
92.6
88.3
78.9
71.4
62.6
Max-PPMI baseline
54.5
52.0
41.1
41.9
BERT-HypOniy
71.5
70.1
52.3
46.4
48.4
RoBERTa Hyyolll
72.0
70.4
53.6
49.5
48.5
Human performance Training set size
88.1
88.1
82.3
80.3
77.8
550k
92k
138k
109k
92k
Table 1: Dev accuracy (%) on the original SNLI dataset and the datasets obtained through vari- representation-biat. minimization. The -HypOnly baselines correspond to models trained on the instances restricted to their hypotheses.
Once we apply AFLITE, as expected, the number of examples with artifacts is reduced considerably, making the task hard once again for the linear model, but still solvable for the non-linear one. The filtered dataset is shown in the bottom half of Fig. [ and the captions indicate the performance of linear and an SVM model. For the first dataset, we see that AFLITE removes most of those examples with flipped labels.
1B ane LNICE (EN0A00)
541 an CONSS (H1EON
521 anc 00117 (-6608)
127 ATC 0.00E 141000
12T MEC COOZS CH1
327 MCC 67703 -4600)
137 MEC 67000 (41020)
Figure 2: Four sample datasets with artifacts as input to AFLITE (top). Blue and orange indicate two different classes. Only the original dimensions shown, not the artifacts. For the leftmost dataset with the highest separation, we flip some labels at random, so even an RBF kernel cannot achieve perfect performance. AFLITE makes the data more challenging for the models (bottom).
3.2 NLP EXPERIMENTS
We evaluate AFLITE on two NLP tasks, namely NLI and question answer sentence selection. We use two popular NLI large-scale datasets SNLI (Bowman 2018). For the answer sentence selection task, we use QNLI which is transformed version of the SQuAD question answering dataset (Rajpurkar al. 2016) converted to binary classification where systems determine whether a sentence contains the answer to question.
2013) and MNLI (Wang al.
SNLI
Each instance in the SNLI dataset consists of a premise-hypotheeeeee pair that belongs to one out of three possible categories (entailment, contradiction, neutral) based the relationship between the premise and the hypothesis.
For SNLI, experiment with three different feature representations derived from strong baseline models: ΦEERT and $ PRABERT which are based on BERT (Devlin et al. 2019) and RoBERTa (Liu al. 2019b), large-scale pretrained masked language models, plus DESM- which uses the

---

Under review as a conference paper at ICLR 2020
MNLI
QNLI
Model
D
D PROBENIO)
D
D(PRoBEKE)
BERT (Devlin et al.
2019)
86.6
55.8
92.0
63.5
RoBERTa (Liu al.
2019b
90.3
66.2
93.7
77.7
BERT-Partiallnpa
59.7
43.2
62.6
56.6
RoBERTa-PPrrialnnu
60.3
44.4
63.9
59.4
Table 2: Dev accuracy (%) on the original MNLI-matched and QNLI datasets and the datasets obtained through 4 Reaernn- to models trained on partial, incomplete input, namely the Hypotheses for MNLI instances and the Answers for QNLI instances.
minimization. The Partiallmput baselines correspond
ESIM model (Chen al.] 2016) with GLoVe word embeddings (Pennington et al., 2014). In all cases, feature representation & is trained on random sample of 10% of the original training in- stances, and feature representations are extracted from the final layer before the output layer. These features pre-computed for all remaining instances while we discard the instances (10% of train- ing) used for training the embeddings in the subsequent steps of our algorithm. Additionally, to measure the ability of a weaker adversary to filter biases only learned by a stronger model, we eval- uate the filtered datasets (for SNLI) with three different models: (i) ESIM with ELMo embeddings (Peters al. 2018), y (ii) BaRT-large, and (iii) RoBERTa-large models.
Table M shows the results for SNLI. In all cases, applying AFLITE substantially reduces overall model accuracy, with typical drops of 15-35% depending the models used for learning the feature representations and those used for evaluation of the filtered dataset. In general, performance is lowest when using the strongest model (RoBERTa) for learning feature representations. Results also highlight the ability of weaker adversaries to produce datasets that are still challenging for much stronger models with a drop of 13.7% for RoBERTa using #EXM+GGLVe as feature representation. We also include model that uses Point-wise Mutual Information (PMI) between words in a given instance and the target label as a feature. The baseline captures the extent which datasets exhibit word-association artifacts. While this baseline is relatively weaker than other models, we still show that its performance reduce from 54.5% on D to 41.9% on the (( dataset.
It might seem unsurprising that reducing the size of the training set results in lower performance. To control for the confounding factor of the dataset size, we create another filtered dataset Dø2k, sampled randomly from D such that its size is approximately equal to the size of D (( dataset. All models achieve nearly the same performance as their performance on the full dataset- when trained on just miffftfth the original dataset size. This result further points to the fact that current benchmark datasets contain significant redundancy within its instances.
Finally, demonstrate the value of the iterative, ensemble-based AFLITE algorithm, compare with a baseline where using a single model, we filter out the most predictable examples in a single iteration - a non-iterative, single-model version of AFLITE, A RoBERTa-large model trained on this subset (of the same size as D(ORoBERT.)) achieves dev accuracy of 72.1% Compared to the performance of RoBERTa on D(OROBERTa) (62.6%, see Table [T. makes this baseline sensible yet less effective approach. In particular, this illustrates the need for an iterative procedure involving models trained on multiple partitions of the remaining data in each iteration.
We also report the k-nearest neighbors distances between examples in the train and heldout data in Table We consider distances for examples within each class, as well as examples across classes. The distances are computed using cosine similarity between pooled features from BERT-base model (features for the [CLS] token, indicating a sentence-pair feature) trained on the original SNLI dataset. Distances are measured between samples from the heldout data, and their nearest neighbors in the training data, before and after filtering. Distances generally increase after filtering, indicating that AFLITE promotes selecting diverse set of examples from the dataset. The only exception to the rule is the neutral class, where distances to other classes decrease - this is not surprising since the neutral class is known to be associated with the least number of artifacts (Gururangan al. 2018).
7

---

Under review as a conference paper at ICLR 2020
Before AFLITE
After AFLITE
Top1
Top5
Top10
Top50
Top1
Top5
Top10
Top50
Entailment
0.29
1.32
2.45
9.5
0.32
1.42
2.64
9.98
Neutral
0,40
1.89
3.68
16.83
0.42
1.97
3.80
16.66
Contradiction
0,49
2.41
4.77
23.09
0.52
2.49
4.84
22.10
Entailment others
0.32
1.48
2.87
12.98
0.34
1.53
2.92
12.31
Neutral vs others
0,43
2.05
3.99
18.42
0,41
3.72
16,41
Contradiction vs others
0.49
2.38
4.70
22.61
0.53
2.51
4.87
22.38
Table 3: KNN-distances by class, before and after applying (RoBERTa-filteree AFLITE SNLL
HANS
NL.-DDagnosties
Adversarial-NL.
Model
All
Lex.
Subseq.
Constit.
All
Logic
Knowl.
Rdl
Rd2
Rd3
RoBERTa
70.7
84.4
35.4
13.4
59.3
52.8
48.9
58.5
48.3
50.1
RoBERTa-AFII
74.5
96.3
56.6
57.4
62.0
53.2
57.7
65.1
49.1
52.8
SNLI accuracy (%) on three out-of-distribution evaluation tasks, comparing RoBERTa- large models pre-trained on the original SNLI data, and on AFLiTE-filtred data. On the HANS dataset, both models are evaluated on All, as well as on the non-entailment cases of the three syn- tactic heuristics (Lexical overlap, Subsequence, and Constituent). The NLI-Diagnostics dataset is broken down into the full dataset (All), as well as the instances requiring logical reasoning (Logic) and the ones requiring world and commonsense knowledge (Knowledge). For Adversarial NI.I, we finetuned both models on the in-distribution training data for each round (Rdl, Rd2, and Rd3).
Table 4:
MNLI and QNLI Following the same procedure described above, we apply AFLITE on the MNLI and QNLI datasets. Since RoBERTa resulted in the largest drops in performance across the board in SNLI, we only experiment with RoBERTa as adversary for MNLI and QNLI. While RoBERTa achieves over 90% on both original datasets, its performance drops to 66. 2% for MNLI and to 77.7% for QNLI on the reduced datasets. Similarly, partial input baseline performance also decreases substantially on both dataset compared to their performance on the original dataset. Table 2 shows these results. We show that AFLITE consistently result in reduced accuracy the filtered datasets across multiple NLP benchmark datasets, even after controlling for the size the training set.
3.2. OUT-OR- NLI
We measure the performance of AFLITE on three other benchmarks for NLI evaluation, which pro- vide out-of-distribution examples to challenge reliance dataset biases in the original SNLI data (Glockner al.) 2018 Naik et al., 2018). Such benchmarks approximate the performance of NLI models in the wild. NLI Diagnosties (Wang et al. 2018 is set of hand-crafted examples designed to demonstrate model performance on several fine grained semantic categories, such as logical rea- soning and commonsense knowledge. HANS (McCoy al. 2019) contains evaluation examples designed to avoid common structural heuristics (such as word overlap) which could be used by models to correctly predict NLI inputs, without true inferential reasoning. Adversarial NLI (Nie) et al. 2019) consists of premises collected from Wikipedia and other news corpora, and human gen- erated hypotheses, arranged at different tiers of the challenge they present a model, using a human and model in-the-looy procedure. Given that these benchmarks are collected independently of the original SNLI task, the biases from SNLI are less likely to carry over; however these benchmarks might contain their own biases (Liu et al. 2019a).
AFLITE assigns predictability score to all samples in dataset, resulting in an ordering of the data. Filtering out examples from the head of the data distribution based on this order yields more accurate benchmarks for measuring true model performance. On the other hand, transferability to out-of-distribution data would involve a greater balance between examples from the head and the tail ends of the data distribution. Hence we evaluate AFLITE for generalization using a larger filtered subset, amounting only third of the full training data. We present these results all the above benchmarks in Table 4 On the two diagnostic datasets (HANS and NLI-Diagnostics), perform zero-shot evaluation of the models. Adversarial NLI allows to test for transfer
8

---

Under review as a conference paper at ICLR 2020
100% Train, Original Val
20% Train, Original Val
AFLITE
Model
Top-1
Top-5
Top-1
Top-5
Top-1
Top-5
EfficientNet-B0
76.3
93.2
58.5
81.2
18.1
48.1
EfficientNet-BB B2
79.8
94.9
60.9
82.8
20.7
53.1
EfficientNet-B B4
82.6
96.3
64.4
85.8
23.3
58.8
EfficientNet-a BB
84.4
97.1
73.8
90.8
24.5
60.6
ResNet-33
78.4
94.4
51.8
74.3
11.1
30.2
ResNet-50
79.2
94.7
53.2
75.5
12.2
30,2
ResNet- 101
80.1
95.4
55.6
77.5
12.3
32.1
ResNet-152
80.6
95.5
56.5
78.2
13.2
33.8
Table 5: Experimental results on ImageNet. We compare between three settings: the original dataset's train-test splits, using 20% of the training set but evaluating on the validation set, and using the AFLITE produced training and validation sets. AFLITE produces a training dataset that is also 20% of the training size, making it fair comparison in terms of dataset examples. The results show a significant drop in Top-1 and Top-5 accuracy: the Top-1 accuracy goes down by roughly 40 percentage points per model in this new training and evaluation setting.
capabilities, by finetuning these models each of the three training datasets (Rdl, Rd2 and Rd3). each the benchmarks above, the model trained on the AFLITE data consistently outperforms the model trained the full SNLI data. challenging examples in the HANS benchmark, which targets models purely relying on lexical and syntactic cues. Similarly, our model performs better on the instances in NL.-Diagnosties that require logical reasoning and commonsense knowledge, as opposed to instances that can be solved through lexical entailment alone.
3.3 IMAGENET EXPERIMENTS
We evaluate AFLITE image classification through ImageNet (ILSVRC2012) classification. On ImageNet, we use the state-of-the-an EfficientNet-B7 model as our core feature extractor (Tan & Le,20119 The EfficientNet model learned from scratch fixed 20% sample of the ImageNet training set, using AutoAugment data augmentation (Cubuk et al. 2019). We then use the 2560- dimensional features extracted by EfficientNet-B7 then underlying representation for AFLITE to use to filter the remaining dataset.
In Table we evaluate the robustness of the filtered dataset by considering ImageNet accuracy across the EfficientNet and ResNet model families (He et al. 2016). When lowering the size of the training set down 20% of the original, we find a large drop in performance. The Efficient- Net models seem to suffer less from 84% to 73% on EfficientNet-B7 versus 80.6% to 56.5% on ResNet-152. However, the biggest performance drop comes from training and evaluating on AFLITE-filteed dataset: the top performer is still EfficientNet-B77. but its accuracy drops to 24.5% top-1. This is despite controlling for dataset size, as well as discrepancy between the training and validation sets.
Overall, these results suggest that image classification even within a subset of the closed world of ImageNet is far from solved. These results echo other findings that suggest that common biases that naturally occur web-scale image data, such as towards canonical poses (Alcom et al. 2019) or towards texture rather than shape (Geirhos et al.) 2018), are problems for ImageNet-traimed classifiers. Indeed, the randomly-selected ImageNet images in Figure | suggest that the AFLITE algorithm learns identify subsets of the data that particularly challenging.
4 RELATED WORK
Our proposed framework for artifact reduction is related to the adversarial filtering (AF) algorithm in Zellers et al. 20188). yet distinct two key ways: our approach (i) much more broadly applicable (by not requiring over generation of data instances), and (ii) considerably more lightweight (by not requiring re- re-training a model at each iteration of AF). Variants of this AF approach have recently
been used create other datasets such HellaSwag (Zellers et al. 2019) and ANLI (Bhagavatula

---

Under review as a conference paper at ICLR 2020
et al. 2019) by iteratively perturbing dataset instances until a target model cannot fit the resulting dataset. While effective, these approaches run into three main pitfalls. First, dataset curators need to explicitly devise a strategy of collecting generating perturbations of given instance. Second, the approach runs the risk of distributional bias where a discriminator can learn to distinguish between machine generated instances and human-generated ones. Finally it requires re-training model each iteration, which is computationally expensive especially when using a large model such as BERT (Devlin al,, 2019) as the adversary. In contrast, AFLITE focuses on addressing dataset biases from existing datasets instead of adversarially perturbing instances. AFLITE was earlier proposed by [Sakaguchi et al.) (2019) create the Winogrande dataset. This paper presents more thorough experiments, theoretical justification and results from generalizing the proposed approach to multiple popular NLP and Vision datasets.
AFLITE is also inspired by Gururangan et al. (2018), who study lexical biased prevalent in the SNLI dataset (Bowman et al., 2015) and use point-wise mutual information (PMI) between a word and an inference class determine the words that are highly indicative of the target label. Instead of lexical features, we adopt a deeper representation of the instances using their pre-computed dense feature representations. We determine whether the dense feature representations are highly indicative of the target label. If so, discard the corresponding instances and proceed iteratively.
ensemble of linear classifiers trained random subsets of the data to
Li & Vasconcelos 2019) recently proposed REPAIR, a method to remove representation bias by dataset resampling. While resampling is common technique for balancing datasets, the motivation in REPAIR is to learn a probability distribution over the dataset that favors instances that are hard for given representation. This approach targets how train better, less-biased models opposed to creating datasets with fewer artifacts. In addition, the implementation of REPAIR relies on in- training classification loss as opposed to out-of-sample generalization accuracy. RESOUND (Li et al. 2018) is another method that quantifies the representation biases of datasets. It uses the representation biases to assemble a new K-class dataset with smaller biases by sampling an existing C-class dataset (C K).
Arjovsky et al. (2019) argue that unstable, spurious correlations in the data would generalize poorly to novel test environments. Thus, they propose Invariant Risk Minimization as an objective that promotes learning representations of the data which are stable across environments. Instead of learning optimal classifiers, our aim is to remove instances that exhibit artifacts in dataset.
5 CONCLUSION
We presented AFLITE novel iterative greedy algorithm that adversarially filters out data points to arrive at a reduced dataset with more realistic problem distributions and considerably fewer spu- rious biases. We apply AFLITE to four widely-used datasets, including SNLI and ImageNet, where reported performance is extremely high and show that state-of the-thhartt art performance on the result- ing filtered dataset drops by 30 points for SNLI and drops from 84.4% 24.5% Top-1 accuracy for ImageNet. In extensive analysis we show that AFLITE is effective on real as well as synthetic datasets. We hope that dataset creators will employ AFLITE to identify unobservable artifacts be- fore releasing new challenge datasets for the research community in order to have more reliable estimate of model performance on future AI benchmarks.
REFERENCES
Michael A Alcorn, Qi Zhitao Gong, Chengfei Wang, Long Mai, Wei-Shinn Ku, and Anh Nguyen. Strike (with) a pose: Neural networks are easily fooled by strange poses of familiar objects. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 4845- 4854, 2019.
Martín Arjovsky. Léon Bottou, Ishaan Guirajani, and David Lopez-Paz. Invariant risk minimization. ArXiv, abs//197.00293,, 2019.
Matej Balog, Nilesh Tripuraneni, Zoubin Ghahramani, and Adrian Weller. Lost relatives of the gumbel trick. In /CML,, 2017.
10

---

Under review as a conference paper at ICLR 2020
Chandra Bhagavatula, Ronan Le Bras, Chaitanya Malaviya. Keisuke Sakaguchi, Ari Holtzman, Han- nah Rashkin, Doug Downey, Scott Wen tau Yih, and Yejin Choi. Abductive commonsense rea- soning. ArXiv, hew///98899799 2019.
Samuel Bowman, Gabor Angeli, Christopher Potts, and Christopher D. Manning. A large anno- tated corpus for learning natural language inference. In EMNLP, 2015.
Qian Chen, Xiao-Dan Zhu, Zuen-Hua Ling, Si Wei, Hui Jiang, and Diana Inkpen. Enhanced Istm for natural language inference. ACL, 2016.
Ekin D Cubuk, Barret Zoph, Dandelion Mane, Vijay Vasudevan, and Quoc Le. Autoaugment: Learning augmentation strategies from data. In Proceedings of the IEEE Conference Computer Vision and Pattern Recognition, pp. 113-123, 2019.
Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. BERT: Pre-training of deep bidirectional transformers for language understanding. In NAACI-HIT, 2019.
Kevin Eykholt, Ivan Evtimov, Earlence Fernandes, Bo Li, Amir Rahmati, Chaowei Xiao, Atul Prakash, Tadayoshi Kohno, and Dawn Xiaodong Song. Robust physical-world attacks on deep learning models. CVPR, 2018.
David Fouhey, Wei-cheng Kuo, Alexei A Efros, and Jitendra Malik. From lifestyle vlogs to everyday interactions. In Proceedings of the IEEE Conference Computer Vision and Pattern Recognition, pp. 4991-5000, 2018.
Robert Geirhos, Patricia Rubisch, Claudio Michaelis, Matthias Bethge, Felix A Wichmann, and Wieland Brendel. Imagenet-trained cnns biased towards texture; increasing shape bias im- proves accuracy and robustness. ICLR, 2018.
Mor Geva, Yoav Goldberg, and Jonathan Berant. Are we modeling the task or the annotator? An investigation of annotator bias natural language understanding datasets. In EMNLP, 2019.
Max Glockner, Vered Shwartz, and Yoav Goldberg. Breaking NLI systems with sentences that require simple lexical inferences. In Proceedings of the 56th Annual Meeting of the Asso- ciation for Computational Linguistics (Volume Short Papers), Melbourne, Australia, July 2018. Association for Computational Linguistics. doi: 10. 1.1.8666////118-11000 URL https:
7/www.co www.wb..we......... org/ wwtpo/////nttg///.ooeoeoeeee//ee 2103
Yash Goyal, Tejas Khot, Douglas Summers-Stay, Dhruv Batra, and Devi Parikh. Making the v in vqa matter: Elevating the role of image understanding in visual question answering. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 6904-6913, 2017.
Emil Julius Gumbel and Julius Lieblein. Statistical theory of extreme values and some practical applications: series of lectures, volume 33. US Government Printing Office Washington, 1954.
Suchin Gururangan, Swabha Swayamdipta, Omer Levy, Roy Schwartz, Samuel Bowman, and Noah A. Smith. Annotation artifacts in natural language inference data. In NAACL-HET, pp- 107 New Orleans, Louisiana, June 2018.
Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recog- nition. In Proceedings of the IEEE conference computer vision and pattern recognition, pp- 770-778, 2016.
Eric Jang, Shixiang Gu, and Ben Poole. Categorical reparameterization with gumbel-softmax. In ICLR, 2016.
Robin Jia and Percy Liang. Adversarial examples for evaluating reading comprehension systems. In EMNLP, 2017.
Carolyn Kim, Ashish Sabharwal, and Stefano Ermon. Exact sampling with integer linear programs and random perturbations. In AAAI, 2016.
Wouter Kool, Herke van Hoof, and Max Welling. Stochastic beams and where to find them: The gumbel-top-k trick for sampling sequences without replacement. In ICMI., 2019.

---

Under review as a conference paper at ICLR 2020
Yi Ci Li and Nuno Vasconcelos. REPAIR: Removing representation bias by dataset resampling. In CVPR, 2019.
Yingwei Li, Yi Li, and Nuno Vasconcelos. RESOUND: Towards action recognition without repre- sentation bias. In ECCV, pp. 513-528, 2018.
Nelson F. Liu, Roy Schwartz, and Noah A. Smith. Inoculation by fine-tuning: A method for analyz- ing challenge datasets. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Téchnologies, Volume I (Long and Short Papers), Minneapolis, Minnesota, June 2019a. Association for Computational Lin- guistics. doi: 10.18653/1/NNI--LL 1225. URL https:/ /www. wtt.:/ew.w..g.ttoottoooooo 1199-1225
Yinhan Liu, Myle Out, Naman Goyal, Jingfei Du, Mandar Joshi, Danqi Chen, Omer Levy, Mike Lewis, Luke S. Zettlemoyer, and Veselin Stoyanov. RoBERTa: A robustly optimized BERT pretraining approach. ArXiv, abs/ 1907. 1692. 2019b.
Chris J. Maddison, Daniel Tarlow, and Tom Minka. A sampling. NIPS, 2014.
Chris Maddison, Andriy Mnih, and Yee Whye Teh. The concrete distribution: A continuous relaxation of discrete random variables. In ICLR, 2016.
Tom McCoy, Ellie Pavlick, and Tal Linzen. Right for the wrong reasons: Diagnosing syntactic heuristics in natural language inference. In Proceedings of the 57th Annual Meeting of the As- sociation for Computational Linguistics. Association for Computational Linguistics, 2019. doi:
10.18653/v1/P19. 1334. (tL.https:// //ttpp:://www... acl.lcllele. www.wweb.oo. htg.://ebbbgg/tttooooo///.SS/
Aakanksha Naik, Abhilasha Ravichander, Norman Sadeh, Carolyn Rose, and Graham Neubig. Stress test evaluation for natural language inference. In Proceedings of the 27th International Conference on Computational Linguistics. Association for Computational Linguisties, 2018.
URL https:://///it www.:/wwwww.........//././///... acl.ell webb wwg. www.aa/ceee..oreaatttoooo/////// http//antoooogycc-11111
Yixin Nie, Adina Williams, Emily Dinan, Mohit Bansal, Jason Weston, and Douwe Kiela. Adver- sarial NLI: A new benchmark for natural language understanding. 2019.
Jeffrey Pennington, Richard Socher, and Christopher D. Manning. Glove: Global vectors for word
representation. In EMNLP, 2014.
Matthew E. Peters, Mark Neumann, Mohit lyyer, Matt Gardner, Christopher Clark, Kenton Lee, and Luke S. Zettlemoyer. Deep contextualized word representations, In NAACL, 2018.
Adam Poliak, Jason Naradowsky, Aparajita Haldar, Rachel Rudinger, and Benjamin Van Durme. Hypothesis only baselines natural language inference. Proceedings of the Seventh Joint Conference Lexical and Computational Semantics, pp. 180-191, New Orleans, Louisiana, June 2018. Association for Computational Linguistics.
Pranav Rajpurkar, Jian Zhang, Konstantin Lopyrev, and Percy Liang. Squad: 100, 000+ questions for machine comprehension of text. In EMNLP, 2016.
Olga Russakovsky. Jia Deng, Hao Su, Jonathan Krause, Sanjeev Satheesh, Sean Ma, Zhiheng Huang, Andrej Karpathy, Aditya Khosla, Michael Bernstein, Alexander C. Berg, and Li Fei-Fei. ImageNet Large Scale Visual Recognition Challenge. International Journal of Computer Vision (IJCV), 11533)211-252, 2015. doi: 10. 10.10/7/s1 123--15-08866-
Keisuke Sakaguchi, Ronan Le Bras, Chandra Bhagavatula, and Yejin Choi. Winogrande: adver- sarial winograd schema challenge at scale. arXiv preprint arXiv: /907. 10641, 2019.
Mingxing Tan and Quoe Le. Efficientnet: Rethinking model scaling for convolutional neural net- works. In International Conference on Machine Learning, pp. 1105-6114, 14, 2019.
Antonio Torralba and Alexei A. Efros. Unbiased look at dataset bias. CVPR, pp. 1521-1528, 2011.
Masatoshi Tsuchiya Performance impact caused by hidden bias of training data for recognizing textual entailment. Proceedings of the 11th Language Resources and Evaluation Conference, Miyazaki, Japan, May 2018. European I Language Resource Association.
12

---

Under review as a conference paper at ICLR 2020
Tim Vieira http:///wwwirraaaiiiooo/o//////3 www---maa------a-aaeee tnine-ttii-add-----e-- seee--andwwegheedrreeeeeee sampling.
Gumbel-max trick and weighted reservoir sampling, 2014. Blog post.
Alex Wang, Amanpreet Singh, Julian Michael, Felix Hill, Omer Levy, and Samuel Bowman. Glue: A multi-task benchmark and analysis platform for natural language understanding. In ICLR, 2018.
Adina Williams, Nikita Nangia, and Samuel Bowman. A broad-coverage challenge corpus for sen- tence understanding through inference. In NAACL-IILT, pp. 11 1112-1122, 2018.
Rowan Zellers, Yonatan Bisk, Roy Schwartz, and Yejin Choi. SWAG: A large-scale adversarial dataset for grounded commonsense inference. In EMNLP, 2018.
Rowan Zellers, Ari Holtzman, Yonatan Bisk, Ali Farhadi, and Yejin Choi. HellaSwag: Can
machine really finish your sentence? In ACL, 2019.
13


Reviews:
[{"review_id": "S1gBu5PLtB", "reviewer": "AnonReviewer3", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "This paper proposes to learn a subset of a given dataset that acts as an adversary, that hurts the model performance when used as a training dataset. The central claim of the paper is that existing datasets on which models are trained are potentially biased, and are not reflective of real world scenarios. By discarding samples that add to this bias, the idea is to make the model perform better in the wild. The authors propose a method to do so, and then refine it so that the resulting solution is tractable. They implement the method on several datasets and show that by finding these adversarial samples, they indeed hurt model performance. \n\nCOMMENTS:\n- Overall the method seems to be something like what is done in k-fold CV, except here we want to find a subset that is the worst at predicting model performance. To this end, I find the introduction of terms like \"representation bias\" and \"predictability scores\" unnecessary. Why not model the entire problem in terms of classification error? \n\n- Page 3 : the last 2 paragraphs are repeated from above. \n\nE: i read the author responses and they addressed my concern about model performance in the wild. I have updated my score to reflect this. \n\n- eqn (3) and the set of equations above: for the math to work, you need q(i) to have non-zero support on all samples. To that end, the sentence that says it works for \"any\" q() is incorrect. \n\n- The experiments back your claim that your method makes the data more challenging to train on. But that does not address the central idea, that the resultant models do better in the wild. If the aim is to make the models robust to real world, you have provided no evidence that your method does so. \n\n- Table 1: the D_92k column is good comparison to have. Thanks. \n\n", "extra_scores": {"experience_assessment": "I have read many papers in this area.", "review_assessment:_checking_correctness_of_experiments": "I assessed the sensibility of the experiments.", "review_assessment:_thoroughness_in_paper_reading": "I read the paper at least twice and used my best judgement in assessing the paper.", "review_assessment:_checking_correctness_of_derivations_and_theory": "I carefully checked the derivations and theory."}, "extra_text": {}}, {"review_id": "HkgJVxk6tS", "reviewer": "AnonReviewer1", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "Summary: This paper hypothesizes that even though we are able to achieve very impressive performance on benchmark datasets as of now (e.g. image net), it might be due to the fact that benchmarks themselves have biases. They introduce an algorithm that selects more representative data points from the dataset that allow to get a better estimate of the performance in the wild. The algorithm ends up selecting more difficult/confusing instances.\n\nThis paper is easy to read and follow (apart from some hickup with a copy of three paragraphs), but in my opinion of limited use/impact.\n\nComments:\n1) There is a repetition of the \" while this expression formalizes..\" paragraph and the next paragraph and the paragraph \"As opposed to ..\" is out of place. Please fix\n2) I am not sure \n- What applications the authors suggest. They seem to say that benchmark authors should run their algorithm and make benchmarks harder. To me it seems that benchmarks become harder because you remove most important instances from the training data (so Table 4 is not surprising - you remove the most representative instances so the model can't learn)\n- how practically feasible it is.  Even if in previous point I am wrong, the algo requires retraining the models on subsets (m iterations). How large is this m?\n3) Other potential considerations:\n-  When you change the training size, the model potentially needs to be re-tuned (regularization etc) (although it might be not that severe since the size of the training data is preserved at t)\n- How do u chose the values of hyperparams (t, m,k eta), how is performance of your algorithm depends on it\n4) I don't see any good baselines to compare with - what if i just chose instances that get the highest prediction score on a model and remove these. How would that do? For NLP (SNLI) task i think this would be a more reasonable baseline than just randomly dropping the instances,\n5) I wonder if you actually retrain the features after creating filtered dataset, new representation would be able to recover the performance. \n\nI read authors rebuttal and new experiments that show that the models trained on filtered data generalize better are proving the point, thanks. Changing to weak accept\n", "extra_scores": {"experience_assessment": "I do not know much about this area.", "review_assessment:_checking_correctness_of_experiments": "I carefully checked the experiments.", "review_assessment:_thoroughness_in_paper_reading": "I read the paper at least twice and used my best judgement in assessing the paper.", "review_assessment:_checking_correctness_of_derivations_and_theory": "I assessed the sensibility of the derivations and theory."}, "extra_text": {}}, {"review_id": "H1xnLtZkcS", "reviewer": "AnonReviewer2", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "the paper proposes an algorithm that adversarially filters out examples to reduce dataset-specific spurious bias. the key intuition is that the datasets are curated in a way that easy to obtain samples have higher probability to be admitted to the dataset. however, not all real world samples are easy to obtain. in other words, real world samples may follow a completely different distribution than curated samples with easy-to-obtain ones.\n\nthe proposed approach discounts the data-rich head of the datasets and emphasizes the data-low tail. they quantify data-rich / data-low by the best possible out-of-sample classification accuracy achievable by models when predicting. \n\nthen adjust the dataset via the expected out-of-sample classification accuracy. the idea of the paper is interesting and the experiments show a substantial reduction in the performance of existing algorithms. this make the paper a promising proposal.\n", "extra_scores": {"experience_assessment": "I do not know much about this area.", "review_assessment:_thoroughness_in_paper_reading": "I read the paper at least twice and used my best judgement in assessing the paper.", "review_assessment:_checking_correctness_of_experiments": "I assessed the sensibility of the experiments.", "review_assessment:_checking_correctness_of_derivations_and_theory": "I assessed the sensibility of the derivations and theory."}, "extra_text": {}}]

Author rebuttal:

Meta-review: This paper proposes to address the issue of biases and artifacts in benchmark datasets through the use of adversarial filtering. That is, removing training and test examples that a baseline model or ensemble gets wright. 

The paper is borderline, and could have flipped to an accept if the target acceptance rate for the conference were a bit higher. All three reviewers ultimately voted weakly in favor of it, especially after the addition of the new out-of-domain generalization results. However, reviewers found it confusing in places, and R2 wasn't fully convinced that this should be applied in the settings the authors suggest. This paper raises some interesting and controversial points, but after some private discussion, there wasn't a clear consensus that publishing it as is would do more good than harm.
Decision: Reject