Original: A DVERSARIAL F ILTERS OF D ATASET B IASES
Revised: A DVERSARIAL F ILTERS OF D ATASET B IASES

Original: Anonymous authors
Revised: Anonymous authors

Original: Paper under double-blind review
Revised: Paper under double-blind review

Original: A BSTRACT
Revised: A BSTRACT

Original: Large-scale benchmark datasets have been among the major driving forces in AI, supporting training of models and measuring their progress.
Revised: Large-scale benchmark datasets have been among the major driving forces in AI, supporting training of models and measuring their progress.

Original: The key assumption is that these benchmarks are realistic approximations of the target tasks in the real world.
Revised: The key assumption is that these benchmarks are realistic approximations of the target tasks in the real world.

Original: However, while machine performance on these benchmarks advances rapidly — often surpassing human performance — it still struggles on the target tasks in the wild.
Revised: However, while machine performance on these benchmarks advances rapidly — often surpassing human performance — it still struggles on the target tasks in the wild.

Original: This raises an important question: whether the surreal high performance on existing benchmarks are inﬂated due to spurious biases in them, and if so, how we can effectively revise these benchmarks to better simulate more realistic problem distributions in the real world.
Revised: This raises an important question: whether the surreal high performance on existing benchmarks are inﬂated due to spurious biases in them, and if so, how we can effectively revise these benchmarks to better simulate more realistic problem distributions in the real world.

Original: In this paper, we posit that while the real world problems consist of a great deal of long-tail problems, existing benchmarks are overly populated with a great deal of similar (thus non -tail) problems, which in turn, leads to a major overestimation of true AI performance.
Revised: In this paper, we posit that while the real world problems consist of a great deal of long-tail problems, existing benchmarks are overly populated with a great deal of similar (thus non -tail) problems, which in turn, leads to a major overestimation of true AI performance.

Original: To address this challenge, we present a novel framework of
Revised: To address this challenge, we present a novel framework of

Original: Adversarial Filters to investigate model-based reduction of dataset biases.
Revised: Adversarial Filters to investigate model-based reduction of dataset biases.

Original: We discuss that the optimum bias reduction via A F O PTIMUM is intractable, thus propose
Revised: We discuss that the optimum bias reduction via A F O PTIMUM is intractable, thus propose

Original: A F L ITE , an iterative greedy algorithm that adversarially ﬁlters out data points to identify a reduced dataset with more realistic problem distributions and considerably less spurious biases.
Revised: A F L ITE , an iterative greedy algorithm that adversarially ﬁlters out data points to identify a reduced dataset with more realistic problem distributions and considerably less spurious biases.

Original: A F L ITE is lightweight and can in principle be applied to any task and dataset.
Revised: A F L ITE is lightweight and can in principle be applied to any task and dataset.

Original: We apply it to popular benchmarks that are practically solved — ImageNet and
Revised: We apply it to popular benchmarks that are practically solved — ImageNet and

Original: Natural Language Inference (SNLI, MNLI, QNLI) — and present ﬁltered counterparts as new challenge datasets where the model performance drops considerably (e.g., from 84% to 24% for ImageNet and from 92% to 62% for SNLI), while human performance remains high.
Revised: Natural Language Inference (SNLI, MNLI, QNLI) — and present ﬁltered counterparts as new challenge datasets where the model performance drops considerably (e.g., from 84% to 24% for ImageNet and from 92% to 62% for SNLI), while human performance remains high.

Original: An extensive suite of analysis demonstrates that A F L ITE effectively reduces measurable dataset biases in both the synthetic and real datasets.
Revised: An extensive suite of analysis demonstrates that A F L ITE effectively reduces measurable dataset biases in both the synthetic and real datasets.

Original: Finally, we introduce new measures of dataset biases based on
Revised: Finally, we introduce new measures of dataset biases based on

Original: K-nearest-neighbors to help guide future research on dataset developments and bias reduction
Revised: K-nearest-neighbors to help guide future research on dataset developments and bias reduction

Original:  I NTRODUCTION
Revised:  I NTRODUCTION

Original: Large-scale neural networks have achieved superhuman performance across many popular AI benchmarks, for tasks as diverse as image recognition (ImageNet; Russakovsky et al.
Revised: Large-scale neural networks have achieved superhuman performance across many popular AI benchmarks, for tasks as diverse as image recognition (ImageNet; Russakovsky et al.

Original: , natural language inference (SNLI; Bowman et al. (2015)), and question answering (SQuAD; Rajpurkar et al.
Revised: , natural language inference (SNLI; Bowman et al. (2015)), and question answering (SQuAD; Rajpurkar et al.

Original: Yet these deep models struggle when taken out of these dataset environments and evaluated on adversarial data or problems in the wild (Eykholt et al., 2018; Jia & Liang, 2017).
Revised: Yet these deep models struggle when taken out of these dataset environments and evaluated on adversarial data or problems in the wild (Eykholt et al., 2018; Jia & Liang, 2017).

Original: This raises a key question: Does high model performance on today’s benchmark datasets indicate the underlying task is solved, or do those datasets overestimate the true capabilities of current AI systems ?
Revised: This raises a key question: Does high model performance on today’s benchmark datasets indicate the underlying task is solved, or do those datasets overestimate the true capabilities of current AI systems ?

Original: Answering this question is key because benchmarks serve important roles in the community.
Revised: Answering this question is key because benchmarks serve important roles in the community.

Original: Not only do they direct progress on core tasks, they also make it easier to tackle the lofty target tasks such as image recognition in the wild through a more practically-scoped dataset such as ImageNet.
Revised: Not only do they direct progress on core tasks, they also make it easier to tackle the lofty target tasks such as image recognition in the wild through a more practically-scoped dataset such as ImageNet.

Original: However, the closed-world assumption of most existing datasets is subject to signiﬁcant bias (Torralba & Efros, 2011).
Revised: However, the closed-world assumption of most existing datasets is subject to signiﬁcant bias (Torralba & Efros, 2011).

Original: Much of the data that is easy to obtain and label isn’t necessarily representative of the task we seek to measure.
Revised: Much of the data that is easy to obtain and label isn’t necessarily representative of the task we seek to measure.

Original: Thus, if left unchecked, artifacts from data collection (Fouhey et al., 2018) or human labeling (Gururangan et al., 2018; Poliak et al., 2018; Tsuchiya, 2018;
Revised: Thus, if left unchecked, artifacts from data collection (Fouhey et al., 2018) or human labeling (Gururangan et al., 2018; Poliak et al., 2018; Tsuchiya, 2018;

Original: Geva et al., 2019) can signiﬁcantly inﬂate model performance.
Revised: Geva et al., 2019) can signiﬁcantly inﬂate model performance.

Original: Though there exist task- and dataset-speciﬁc ap-
Revised: Though there exist task- and dataset-speciﬁc ap-

Original: [Figure]
Revised: [Figure]

Original: monarch excluded by AFLite 1.
Revised: monarch excluded by AFLite 1.

Original: [Figure]
Revised: [Figure]

Original: [Figure] [Figure]
Revised: [Figure] [Figure]

Original: [Figure] [Figure]
Revised: [Figure] [Figure]

Original: [Figure] [Figure]
Revised: [Figure] [Figure]

Original: [Figure] [Figure]
Revised: [Figure] [Figure]

Original: [Figure] [Figure]
Revised: [Figure] [Figure]

Original: [Figure]
Revised: [Figure]

Original: proaches for addressing these biases (Goyal et al., 2017; Geirhos et al., 2018), the complex artifacts that emerge from large-scale dataset creation are challenging to exhaustively identify and remove.
Revised: proaches for addressing these biases (Goyal et al., 2017; Geirhos et al., 2018), the complex artifacts that emerge from large-scale dataset creation are challenging to exhaustively identify and remove.

Original: In this paper, we present A F L ITE – a computationally efﬁcient dataset reduction algorithm, aimed at systematically reducing spurioius artifacts in a dataset.
Revised: In this paper, we present A F L ITE – a computationally efﬁcient dataset reduction algorithm, aimed at systematically reducing spurious artifacts in a dataset.

Original: A F L ITE is general and can be applied to any task and dataset.
Revised: A F L ITE is general and can be applied to any task and dataset.

Original: Our approach leverages a high capacity model to learn dataset speciﬁc biases on a small subset, then uses it to identify and ﬁlter artifact-prone instances in the remainder of the dataset to yield a ﬁnal dataset that is possibly closer to the intended task.
Revised: Our approach leverages a high capacity model to learn dataset speciﬁc biases on a small subset, then uses it to identify and ﬁlter artifact-prone instances in the remainder of the dataset to yield a ﬁnal dataset that is possibly closer to the intended task.

Original: We ﬁrst evaluate the effectiveness of our method on synthetic data and show that A F L ITE lowers the performance of models relying on annotation artifacts while preserving the performance of models whose representation captures the underlying tasks.
Revised: We ﬁrst evaluate the effectiveness of our method on synthetic data and show that A F L ITE lowers the performance of models relying on annotation artifacts while preserving the performance of models whose representation captures the underlying tasks.

Original: In addition, while A F L ITE aims to retain the more challenging, confusing instances, our experiments show that it can successfully remove biased instances that are adversarial to the correct representation of the data.
Revised: In addition, while A F L ITE aims to retain the more challenging, confusing instances, our experiments show that it can successfully remove biased instances that are adversarial to the correct representation of the data.

Original: Finally, we apply the method to several benchmark datasets across various tasks and domains.
Revised: Finally, we apply the method to several benchmark datasets across various tasks and domains.

Original: In language understanding, we apply A F L ITE to the SNLI (Bowman et al., 2015) and MNLI (Williams et al., 2018) datasets for natural language inference, and to QNLI (Wang et al., 2018) for question answering.
Revised: In language understanding, we apply A F L ITE to the SNLI (Bowman et al., 2015) and MNLI (Williams et al., 2018) datasets for natural language inference, and to QNLI (Wang et al., 2018) for question answering.

Original: We show a 30% absolute gap in performance in the current state-of-art methods before and after A F L ITE .
Revised: We show a 30% absolute gap in performance in the current state-of-art methods before and after A F L ITE .

Original: In computer vision, A F L ITE reduces theperformance of image classiﬁcation neural methods on ImageNet (Russakovsky et al., 2015),showing a 49% absolutegap.
Revised: In computer vision, A F L ITE reduces the neural methods on ImageNet (Russakovsky et al., 2015), 49% absolute

Original:  D ATASET REDUCTION FOR REPRESENTATION - BIAS MINIMIZATION
Revised:  D ATASET REDUCTION FOR REPRESENTATION - BIAS MINIMIZATION

Original: In this section, we introduce A F L ITE , a general approach for reducing the scope of bias in datasets.
Revised: In this section, we introduce A F L ITE , a general approach for reducing the scope of bias in datasets.

Original: Large datasets run the risk of prioritizing performance on the data-rich head of the disribution, where examples are plentiful, and discounting the tail .
Revised: Large datasets run the risk of prioritizing performance on the data-rich head of the distribution, where examples are plentiful, and discounting the tail .

Original: Our goal is to minimize the ability of a model to exploit biases in the head of the distribution, while preserving the inherent complexity of the tail .
Revised: Our goal is to minimize the ability of a model to exploit biases in the head of the distribution, while preserving the inherent complexity of the tail .

Original: Let Φ represent a feature representation, deﬁned over a dataset D =
Revised: Let Φ represent a feature representation, deﬁned over a dataset D =

Original: ( X, Y ) .
Revised: ( X, Y ) .

Original: With A F L ITE , we seek a subset S ⊂ D of size | S | ≥ n that is maximally resilient to the features uncovered by Φ .
Revised: With A F L ITE , we seek a subset S ⊂ D of size | S | ≥ n that is maximally resilient to the features uncovered by Φ .

Original: For any identically-distributed train-test split of D , the features extracted by Φ should not generalize to the held-out set.
Revised: For any identically-distributed train-test split of D , the features extracted by Φ should not generalize to the held-out set.

Original: Our approach allows for any choice of feature representation.
Revised: Our approach allows for any choice of feature representation.

Original: Formalization More formally, let M denote a family of classiﬁcation models (e.g., logistic regression, SVM, or a particular neural architecture) that can be trained on subsets S of D =
Revised: Formalization More formally, let M denote a family of classiﬁcation models (e.g., logistic regression, SVM, or a particular neural architecture) that can be trained on subsets S of D =

Original: ( X, Y ) using features Φ( X ) .
Revised: ( X, Y ) using features Φ( X ) .

Original: We deﬁne the representation bias of Φ in S w.r.t M , denoted R (Φ , S, M ) , as the best possible out-of-sample classiﬁcation accuracy achievable by models in M when predicting the true labels Y using features Φ( X ) .
Revised: We deﬁne the representation bias of Φ in S w.r.t M , denoted R (Φ , S, M ) , as the best possible out-of-sample classiﬁcation accuracy achievable by models in M when predicting the true labels Y using features Φ( X ) .

Original: For a given target reduced dataset size of at least n , the goal is to ﬁnd a subset S ⊂ D , | S | ≥ n that minimizes this representation bias in S w.r.t.
Revised: For a given target reduced dataset size of at least n , the goal is to ﬁnd a subset S ⊂ D , | S | ≥ n that minimizes this representation bias in S w.r.t.

Original: M :
Revised: M :

Revised: [Equation]

Revised: S

Original: [Equation]
Revised: [Equation]

Revised: D, | S |≥ n R (Φ

Revised: [Equation]

Revised: S, M )

Revised: [Equation]

Original: Eq.
Revised: Eq.

Original: (1) corresponds to the optimum bias reduction, referred to as A F O PTIMUM .
Revised: (1) corresponds to the optimum bias reduction, referred to as A F O PTIMUM .

Original: R (Φ , S, M ) can be formulated as the expected classiﬁcation accuracy resulting from the following process.
Revised: R (Φ , S, M ) can be formulated as the expected classiﬁcation accuracy resulting from the following process.

Original: Let q : 2 S → [0 , 1] be a probability distribution over subsets T =
Revised: Let q : 2 S → [0 , 1] be a probability distribution over subsets T =

Original: ( X T , Y T ) of D .
Revised: ( X T , Y T ) of D .

Original: The process is to randomly choose a subset T with probability q ( T ) , train a bias estimator M T ∈ M on D \ T , and evaluate its classiﬁcation accuracy f M T (Φ( X T ) , Y T ) on T .
Revised: The process is to randomly choose a subset T with probability q ( T ) , train a bias estimator M T ∈ M on D \ T , and evaluate its classiﬁcation accuracy f M T (Φ( X T ) , Y T ) on T .

Original: Note that the resulting classiﬁcation accuracy on T itself is a random variable, since the training set D \ T is random.
Revised: Note that the resulting classiﬁcation accuracy on T itself is a random variable, since the training set D \ T is random.

Original: We deﬁne the expected value of this classiﬁcation accuracy to be the representation bias:
Revised: We deﬁne the expected value of this classiﬁcation accuracy to be the representation bias:

Original: [Equation]
Revised: [Equation]

Original: While this expression formalizes the intended objective function, it involves a large summation over subsets T ⊂ S just to compute the representation bias present in a single set S .
Revised: While this expression formalizes the intended objective function, it involves a large summation over subsets T ⊂ S just to compute the representation bias present in a single set S .

Original: It does not suggest a practical way to compute the minimization in Eq.
Revised: It does not suggest a practical way to compute the minimization in Eq.

Original: (1) without further considering each of the exponentially many subsets S ⊂ D individually – thus an optimal solution for Equation (2) is intractable.
Revised: (1) without further considering each of the exponentially many subsets S ⊂ D individually – thus an optimal solution for Equation (2) is intractable.

Original: To get around this difﬁculty, we reformulate the representation bias in S as a sum factored over the | S | individual instances i ∈ S .
Revised: To get around this difﬁculty, we reformulate the representation bias in S as a sum factored over the | S | individual instances i ∈ S .

Original: This will allow us to efﬁciently decide whether or not to include i in the targeted, reduced subset we are constructing.
Revised: This will allow us to efﬁciently decide whether or not to include i in the targeted, reduced subset we are constructing.

Original: The idea is to aggregate the contribution of each i towards the representation bias expression across all random choices of the training set D \ T .
Revised: The idea is to aggregate the contribution of each i towards the representation bias expression across all random choices of the training set D \ T .

Original: We call this the predictability score p ( i ) for i : on average, how reliably can the label y
Revised: We call this the predictability score p ( i ) for i : on average, how reliably can the label y

Original: i be predicted using features Φ( x i ) when a model from M is trained on a randomly chosen training set D \ T not containing i .
Revised: i be predicted using features Φ( x i ) when a model from M is trained on a randomly chosen training set D \ T not containing i .

Original: The higher the value of p ( i ) , the easier it is to correctly classify the instance ( x i , y i ) using model family M .
Revised: The higher the value of p ( i ) , the easier it is to correctly classify the instance ( x i , y i ) using model family M .

Original: This is the signal we will use to decide whether to include i in the reduced subset S we are constructing.
Revised: This is the signal we will use to decide whether to include i in the reduced subset S we are constructing.

Original: With some abuse of notation, for i ∈ D , let q ( i ) (cid:44) (cid:80) T (cid:51) i q ( T ) denote the marginal probability of choosing a subset T that contains i .The ratio q ( T ) q ( i ) is then the probability of T conditioned on it containing i . Let f M T (Φ( x i ) , y i ) be the classiﬁcation accuracy of M T on i . The reformulation of representation bias in terms of predictability scores of individual instances works as follows:
Revised: With some abuse of notation, for i ∈ D , let q ( i ) (cid:44) (cid:80) T (cid:51) i q ( T ) denote the marginal probability of choosing a subset T that contains i .

Original: As opposed to having to speciﬁcally identify the possible sources of biases, we enable unsupervised data bias reduction by relying on state-of-the-art methods to uncover undesirable annotation artifacts.

Original: Nevertheless, instead of being adversarial to a particular model or architecture, we assume that the input features are ﬁxed for a given dataset, therefore being adversarial to a given representation .

Original: While these features are not semantically indicative of the target label at the instance level, the motivation behind the representation-bias minimization is to limit how indicative these features are collectively , at the dataset level.

Original: Finally, this deﬁnition of bias is tied to a family of models that can exploit the feature representation Φ .

Original: While the next subsection provides a general deﬁnition, we focus on simple, linear classiﬁers that readily exploit the input features by estimating the direct impact of each feature on the target label.

Original: While this expression formalizes the intended objective function, it involves a large summation over subsets T ⊂ S just to compute the representation bias present in a single set S .

Original: It does not suggest a practical way to compute the minimization in Eq.

Original: (1) without further considering each of the exponentially many subsets S ⊂ D individually.

Original: To get around this difﬁculty, we reformulate the representation bias in S as a sum factored over the | S | individual instances i ∈ S .

Original: This will allow us to efﬁciently decide whether or not to include i in the targeted, reduced subset we are constructing.

Original: The idea is to aggregate the contribution of each i towards the representation bias expression across all random choices of the training set D \ T .

Original: We call this the predictability score p ( i ) for i : on average, how reliably can the label y

Original: i be predicted using features Φ( x i ) when a model from M is trained on a randomly chosen training set D \ T not containing i .

Original: The higher the value of p ( i ) , the easier it is to correctly classify the instance ( x i , y i ) using model family M .

Original: This is the signal we will use to decide whether to include i in the reduced subset S we are constructing.

Original: With some abuse of notation, for i ∈ D , let q ( i ) (cid:44) (cid:80) T (cid:51) i q ( T ) denote the marginal probability of choosing a subset T that contains i .

Original: The ratio q ( T ) q ( i ) is then the probability of T conditioned on it containing i .
Revised: The ratio q ( T ) q ( i ) is then the probability of T conditioned on it containing i .

Original: Let f M T (Φ( x i ) , y i ) be the classiﬁcation accuracy of M T on i .
Revised: Let f M T (Φ( x i ) , y i ) be the classiﬁcation accuracy of M T on i .

Original: The reformulation of representation bias in terms of predictability scores of individual instances works as follows:
Revised: The reformulation of representation bias in terms of predictability scores of individual instances works as follows:

Original: [Equation]
Revised: [Equation]

Original: where p ( i ) is the predictability score of i deﬁned as:
Revised: where p ( i ) is the predictability score of i deﬁned as:

Original: [Equation]
Revised: [Equation]

Original: While the method works for any probability distribution q , for simplicity of exposition, we restrict q to be the uniform distribution over all subsets T ⊂ S of a ﬁxed size.
Revised: While the method works for any probability distribution q with non-zero support on all samples, for simplicity of exposition, we restrict q to be the uniform distribution over all subsets T ⊂ S of a ﬁxed size.

Original: This makes both | T | and q ( i ) ﬁxed constants; in particular, q ( i ) =
Revised: This makes both | T | and q ( i ) ﬁxed constants; in particular, q ( i ) =

Original: (cid:0) | S |− 1 | T |− 1 (cid:1) / (cid:0) | S | | T | (cid:1) =
Revised: (cid:0) | S |− 1 | T |− 1 (cid:1) / (cid:0) | S | | T | (cid:1) =

Original: | T | | S | .
Revised: | T | | S | .

Original: This reduces the predictability score expression of Eq.
Revised: This reduces the predictability score expression of Eq.

Original: (3) to the simpliﬁed variant ˜ p ( i ) :
Revised: (3) to the simpliﬁed variant ˜ p ( i ) :

Original: [Equation]
Revised: [Equation]

Original: Putting the pieces together, we have a factored reformulation of the representation bias in Eq.
Revised: Putting the pieces together, we have a factored reformulation of the representation bias in Eq.

Original: [Equation]
Revised: [Equation]

Original: Armed with this factored representation, we return to the task of identifying an S ⊂ D , | S | ≥ n that minimizes the representation bias.
Revised: Armed with this factored representation, we return to the task of identifying an S ⊂ D , | S | ≥ n that minimizes the representation bias.

Original: We use the simpliﬁed predictability scores (henceforth simply referred to as predictability scores) as a heuristic metric to decide which i ∈ D to include in S .
Revised: We use the simpliﬁed predictability scores (henceforth simply referred to as predictability scores) as a heuristic metric to decide which i ∈ D to include in S .

Original: We consider three approaches that iteratively ﬁlter out the most predictable instances from D to arrive at
Revised: We consider three approaches that iteratively ﬁlter out the most predictable instances from D to arrive at

Original: S .
Revised: S .

Original: In all cases, we use a ﬁxed training set size | S \ T | =
Revised: In all cases, we use a ﬁxed training set size | S \ T | =

Original: t < n .
Revised: t < n .

Original: Further, since a larger ﬁltered set is generally desirable, we terminate the ﬁltering process early (i.e., while | S | > n ) if the predictability score for every i falls below a pre-speciﬁed early stopping threshold τ ∈ [0 , 1] .
Revised: Further, since a larger ﬁltered set is generally desirable, we terminate the ﬁltering process early (i.e., while | S | > n ) if the predictability score for every i falls below a pre-speciﬁed early stopping threshold τ ∈ [0 , 1] .

Original: The three approaches are as follows.
Revised: The three approaches are as follows.

Original: (A)
Revised: (A)

Original: A simple greedy approach starts with the full set S = D , identiﬁes an i ∈ S with the highest predictability score, removes it from S , and repeats up to | D |− n times.
Revised: A simple greedy approach starts with the full set S = D , identiﬁes an i ∈ S with the highest predictability score, removes it from S , and repeats up to | D |− n times.

Original: (B) A greedy slicing approach identiﬁes the instances with the k highest predictability scores, removes all of them from S , and repeats the process up to (cid:98) | D |− n k (cid:99) times.
Revised: (B) A greedy slicing approach identiﬁes the instances with the k highest predictability scores, removes all of them from S , and repeats the process up to (cid:98) | D |− n k (cid:99) times.

Original: (C)
Revised: (C)

Original: A slice sampling approach where, instead of greedily choosing the top k instances, it randomly samples k instances with probabilities proportional to their predictability scores.
Revised: A slice sampling approach where, instead of greedily choosing the top k instances, it randomly samples k instances with probabilities proportional to their predictability scores.

Original: 1 In this paper, we use the greedy slicing approach in our experiments and refer to it as A F L ITE .
Revised: 1 In this paper, we use the greedy slicing approach in our experiments and refer to it as A F L ITE .

Original: While the optimum bias reduction via
Revised: While the optimum bias reduction via

Original: A F O PTIMUM is intractable, its light-weight version, A F L ITE , is applicable in practice.
Revised: A F O PTIMUM is intractable, its light-weight version, A F L ITE , is applicable in practice.

Original: The slice sampling approach can be efﬁciently implemented using what is known as the Gumbel method or Gumbel trick (Gumbel & Lieblein, 1954; Maddison et al., 2014), which uses random perturbations to turn sampling into a simpler problem of optimization.
Revised: The slice sampling approach can be efﬁciently implemented using what is known as the Gumbel method or Gumbel trick (Gumbel & Lieblein, 1954; Maddison et al., 2014), which uses random perturbations to turn sampling into a simpler problem of optimization.

Original: This has recently found success in several probabilistic inference applications (Kim et al., 2016;
Revised: This has recently found success in several probabilistic inference applications (Kim et al., 2016;

Original: Jang et al., 2016; Maddison et al., 2016; Balog et al., 2017; Kool et al., 2019).
Revised: Jang et al., 2016; Maddison et al., 2016; Balog et al., 2017; Kool et al., 2019).

Original: Starting with the log-predictability scores log ˜ p
Revised: Starting with the log-predictability scores log ˜ p

Original: ( i ) for various i , the idea is to perturb them by adding an independent random noise
Revised: ( i ) for various i , the idea is to perturb them by adding an independent random noise

Original: γ i drawn from the standard Gumbel distribution.
Revised: γ i drawn from the standard Gumbel distribution.

Original: Interestingly, the maximizer i ∗ of γ i +log ˜ p
Revised: Interestingly, the maximizer i ∗ of γ i +log ˜ p

Original: ( i ) turns out to be an exact sample drawn from the (unnormalized) distribution deﬁned by ˜ p .
Revised: ( i ) turns out to be an exact sample drawn from the (unnormalized) distribution deﬁned by ˜ p .

Original: Note that i ∗ is a random variable since the γ i are drawn at random.
Revised: Note that i ∗ is a random variable since the γ i are drawn at random.

Original: This result can be generalized (Vieira, 2014) for slice sampling: the k highest values of Gumbel-perturbed log-predictability scores correspond to sampling, without replacement, k items from the probability distribution deﬁned by ˜ p .
Revised: This result can be generalized (Vieira, 2014) for slice sampling: the k highest values of Gumbel-perturbed log-predictability scores correspond to sampling, without replacement, k items from the probability distribution deﬁned by ˜ p .

Original: The Gumbel method is typically applied to exponentially large combinatorial spaces, where it is challenging to scale up.
Revised: The Gumbel method is typically applied to exponentially large combinatorial spaces, where it is challenging to scale up.

Original: In our setting, however, the overhead is minimal since the cost of drawing a random γ i is negligible compared to computing ˜ p
Revised: In our setting, however, the overhead is minimal since the cost of drawing a random γ i is negligible compared to computing ˜ p

Original: ( i ) .
Revised: ( i ) .

Original: Implementation Algorithm 1 provides an implementation of A F L ITE .
Revised: Implementation Algorithm 1 provides an implementation of A F L ITE .

Original: The algorithm takes as input a dataset D =
Revised: The algorithm takes as input a dataset D =

Original: ( X, Y ) , a representation Φ( X ) we are interested in minimizing the bias in, a model family M (e.g., linear classiﬁers), a target dataset size n , size m of the support of the expectation in Eq.
Revised: ( X, Y ) , a representation Φ( X ) we are interested in minimizing the bias in, a model family M (e.g., linear classiﬁers), a target dataset size n , size m of the support of the expectation in Eq.

Original: (4), training set size t for the classiﬁers, size k of each slice, and an earlystopping ﬁltering threshold τ .
Revised: (4), training set size t for the classiﬁers, size k of each slice, and an early-stopping ﬁltering threshold τ .

Original: Importantly, for efﬁciency, Φ( X ) is provided to A F L ITE in the form of pre-computed embeddings for all of X .
Revised: Importantly, for efﬁciency, Φ( X ) is provided to A F L ITE in the form of precomputed embeddings for all of X .

Revised: To obtain Φ( X ) in practice, we train a ﬁrst model on a small fraction of the data based on the learning curve in low-data regime, and do not reuse this data for the rest of our experiments.

Revised: Moreover, this fraction corresponds to the training size t for A F L ITE and it remains unchanged across iterations.

Original: We follow the iterative ﬁltering approach, starting with
Revised: We follow the iterative ﬁltering approach, starting with

Original: S = D and iteratively removing some instances with the highest predictability scores using the greedy slicing strategy.
Revised: S = D and iteratively removing some instances with the highest predictability scores using the greedy slicing strategy.

Revised: Slice size k and number of partitions m are determined by the available computation budget.

Original: At each ﬁltering phase, we train models (linear classiﬁers in our implementation) on m different random partitions of the data, and collect their predictions on their corresponding test set.
Revised: At each ﬁltering phase, we train models (linear classiﬁers in our implementation) on m different random partitions of the data, and collect their predictions on their corresponding test set.

Original: For each instance i , we compute its predictability score as the ratio of the number of times its label y
Revised: For each instance i , we compute its predictability score as the ratio of the number of times its label y

Original: i is predicted correctly, over the total number of predictions for it.
Revised: i is predicted correctly, over the total number of predictions for it.

Original: We rank the instances according to their predictability score and use the greedy slicing strategy of removing the top- k instances whose score is not less than the early-stopping threshold τ .
Revised: We rank the instances according to their predictability score and use the greedy slicing strategy of removing the top- k instances whose score is not less than the early-stopping threshold τ .

Original: We repeat this process until fewer than k instances pass the τ threshold in a ﬁltering phase or fewer than n instances remain.
Revised: We repeat this process until fewer than k instances pass the τ threshold in a ﬁltering phase or fewer than n instances remain.

Original:  E XPERIMENTAL A NALYSIS
Revised:  E XPERIMENTAL A NALYSIS

Original: We evaluate A F L ITE across various domains (synthetic, natural language processing, computer vision), different tasks in a given domain (language inference and question answering in NLP), differ- ent datasets for a given task (SNLI and MNLI in natural language inference), and different representations for a given dataset (pre-computed embeddings from ESIM+GLoVe , BERT , RoBERTa for the SNLI dataset).
Revised: We evaluate A F L ITE across various domains (synthetic, natural language processing, computer vision), different tasks in a given domain (language inference and question answering in NLP), different datasets for a given task (SNLI and MNLI in natural language inference), and different representations for a given dataset (pre-computed embeddings from ESIM+GLoVe , BERT , RoBERTa for the SNLI dataset).

Original: 3. S YNTHETIC E XPERIMENTS
Revised: 3. S YNTHETIC E XPERIMENTS

Original: We demonstrate the utility of A F L ITE in a synthetic data setting.
Revised: We demonstrate the utility of A F L ITE in a synthetic data setting.

Original: Our dataset consists of twodimensional data, arranged in concentric circles, at four different levels of separation, as shown in the Figure 2.
Revised: Our dataset consists of twodimensional data, arranged in concentric circles, at four different levels of separation, as shown in the Figure 2.

Original: As is evident, a linear function might not be adequate for separating the two classes; it requires a more complex non-linear model such as an SVM with an RBF kernel.
Revised: As is evident, a linear function might not be adequate for separating the two classes; it requires a more complex non-linear model such as an SVM with an RBF kernel.

Original: We add class-speciﬁc artiﬁcially constructed features (artifacts) sampled from two different Gaussian distributions.
Revised: We add class-speciﬁc artiﬁcially constructed features (artifacts) sampled from two different Gaussian distributions.

Original: These features are only added to 75% of the data in each class, while for the rest of the data, we insert random (noise) features.
Revised: These features are only added to 75% of the data in each class, while for the rest of the data, we insert random (noise) features.

Original: These artifacts make the task solvable through a linear function.
Revised: These artifacts make the task solvable through a linear function.

Original: Furthermore, for the ﬁrst dataset, with the largest separation, we ﬂipped the labels of some examples with artifacts, making the data slightly adversarial even to the RBF.
Revised: Furthermore, for the ﬁrst dataset, with the largest separation, we ﬂipped the labels of some examples with artifacts, making the data slightly adversarial even to the RBF.

Original: Both models can clearly leverage the artifacts, and demonstrate improved performance over a baseline without artifacts.
Revised: Both models can clearly leverage the artifacts, and demonstrate improved performance over a baseline without artifacts.

Original:  Once we apply A F L ITE , as expected, the number of examples with artifacts is reduced considerably, making the task hard once again for the linear model, but still solvable for the non-linear one.
Revised: [Table] Once we apply A F L ITE , as expected, the number of examples with artifacts is reduced considerably, making the task hard once again for the linear model, but still solvable for the non-linear one.

Original: The ﬁltered dataset is shown in the bottom half of Fig. 2, and the captions indicate the performance of a linear and an SVM model.
Revised: The ﬁltered dataset is shown in the bottom half of Fig. 2, and the captions indicate the performance of a linear and an SVM model.

Original: For the ﬁrst dataset, we see that A F L ITE removes most of those examples with ﬂipped labels.
Revised: For the ﬁrst dataset, we see that A F L ITE removes most of those examples with ﬂipped labels.

Original: 3. NLP E XPERIMENTS
Revised: 3. NLP E XPERIMENTS

Original: We evaluate A F L ITE on two NLP tasks, namely NLI and question answer sentence selection.
Revised: We evaluate A F L ITE on two NLP tasks, namely NLI and question answer sentence selection.

Original: We use two popular NLI large-scale datasets – SNLI (Bowman et al., 2015) and MNLI (Wang et al., 2018).
Revised: We use two popular NLI large-scale datasets – SNLI (Bowman et al., 2015) and MNLI (Wang et al., 2018).

Original: For the answer sentence selection task, we use QNLI which is a transformed version of the
Revised: For the answer sentence selection task, we use QNLI which is a transformed version of the

Original: SQuAD question answering dataset (Rajpurkar et al., 2016) converted to binary classiﬁcation where systems determine whether a sentence contains the answer to a question.
Revised: SQuAD question answering dataset (Rajpurkar et al., 2016) converted to binary classiﬁcation where systems determine whether a sentence contains the answer to a question.

Original: [Table] SNLI Each instance in the SNLI dataset consists of a premise-hypothesis pair that belongs to one out of three possible categories ( entailment , contradiction , or neutral ) based on the relationship 
Revised: SNLI Each instance in the SNLI dataset consists of a premise-hypothesis pair that belongs to one out of three possible categories ( entailment , contradiction , or neutral ) based on the relationship between the premise and the hypothesis.

Original: For SNLI, we experiment with three different feature representations derived from strong baseline models: Φ BERT and Φ RoBERTa which are based on BERT (Devlin et al., 2019) and RoBERTa (Liu et al., 2019), large-scale pretrained masked language models, plus Φ ESIM+GLoVe which uses the  ESIM model (Chen et al., 2016) with GLoVe word embeddings (Pennington et al., 2014).
Revised: For SNLI, we experiment with three different feature representations derived from strong baseline models: Φ BERT and Φ RoBERTa which are based on BERT (Devlin et al., 2019) and RoBERTa (Liu et al., 2019b), large-scale pretrained masked language models, plus Φ ESIM+GLoVe which uses the [Table] ESIM model (Chen et al., 2016) with GLoVe word embeddings (Pennington et al., 2014).

Original: In all cases, feature representation Φ is trained on a random sample of 10% of the original training instances, and feature representations are extracted from the ﬁnal layer before the output layer.
Revised: In all cases, feature representation Φ is trained on a random sample of 10% of the original training instances, and feature representations are extracted from the ﬁnal layer before the output layer.

Original: These features are pre-computed for all remaining instances while we discard the instances ( 10% of training) used for training the embeddings in the subsequent steps of our algorithm.
Revised: These features are pre-computed for all remaining instances while we discard the instances ( 10% of training) used for training the embeddings in the subsequent steps of our algorithm.

Original: Additionally, to measure the ability of a weaker adversary to ﬁlter biases only learned by a stronger model, we evaluate the ﬁltered datasets (for SNLI) with three different models: (i) ESIM with ELMo embeddings (Peters et al., 2018), (ii) BERT-large, and (iii) RoBERTa-large models.
Revised: Additionally, to measure the ability of a weaker adversary to ﬁlter biases only learned by a stronger model, we evaluate the ﬁltered datasets (for SNLI) with three different models: (i) ESIM with ELMo embeddings (Peters et al., 2018), (ii) BERT-large, and (iii) RoBERTa-large models.

Original: Table 1 shows the results for SNLI.
Revised: Table 1 shows the results for SNLI.

Original: In all cases, applying A F L ITE substantially reduces overall model accuracy, with typical drops of 15-35% depending on the models used for learning the feature representations and those used for evaluation of the ﬁltered dataset.
Revised: In all cases, applying A F L ITE substantially reduces overall model accuracy, with typical drops of 15-35% depending on the models used for learning the feature representations and those used for evaluation of the ﬁltered dataset.

Original: In general, performance is lowest when using the strongest model (RoBERTa) for learning feature representations.
Revised: In general, performance is lowest when using the strongest model (RoBERTa) for learning feature representations.

Original: Results also highlight the ability of weaker adversaries to produce datasets that are still challenging for much stronger models with a drop of 13.7% for RoBERTa using Φ ESIM+GLoVe as feature representation.
Revised: Results also highlight the ability of weaker adversaries to produce datasets that are still challenging for much stronger models with a drop of 13.7% for RoBERTa using Φ ESIM+GLoVe as feature representation.

Original: We also include a model that uses Pointwise Mutual Information (PMI) between words in a given instance and the target label as a feature.
Revised: We also include a model that uses Point-wise Mutual Information (PMI) between words in a given instance and the target label as a feature.

Original: The baseline captures the extent to which datasets exhibit word-association artifacts.
Revised: The baseline captures the extent to which datasets exhibit word-association artifacts.

Original: While this baseline is relatively weaker than other models, we still show that its performance reduce from 54.5% on D to 41 .
Revised: While this baseline is relatively weaker than other models, we still show that its performance reduce from 54.5% on D to 41 .

Original: 9% on the D ( φ RoBERTa ) dataset.
Revised: 9% on the D ( φ RoBERTa ) dataset.

Original: It might seem unsurprising that reducing the size of the training set results in lower performance.
Revised: It might seem unsurprising that reducing the size of the training set results in lower performance.

Original: To control for the confounding factor of the dataset size, we create another ﬁltered dataset D 92 k , sampled randomly from D such that its size is approximately equal to the size of D ( φ RoBERTa ) dataset.
Revised: To control for the confounding factor of the dataset size, we create another ﬁltered dataset D 92 k , sampled randomly from D such that its size is approximately equal to the size of D ( φ RoBERTa ) dataset.

Original: All models achieve nearly the same performance as their performance on the full dataset – even when trained on just one-ﬁfth the original dataset size.
Revised: All models achieve nearly the same performance as their performance on the full dataset – even when trained on just one-ﬁfth the original dataset size.

Original: This result further points to the fact that current benchmark datasets contain signiﬁcant redundancy within its instances.
Revised: This result further points to the fact that current benchmark datasets contain signiﬁcant redundancy within its instances.

Revised: Finally, to demonstrate the value of the iterative, ensemble-based A F L ITE algorithm, we compare with a baseline where using a single model, we ﬁlter out the most predictable examples in a single iteration — a non-iterative, single-model version of A F L ITE .

Revised: A RoBERTa-large model trained on this subset (of the same size as D ( φ RoBERTa ) ) achieves a dev accuracy of 72 . 1% .

Revised: Compared to the performance of RoBERTa on D ( φ RoBERTa ) ( 62 .

Revised: 6% , see Table 1), it makes this baseline a sensible yet less effective approach.

Revised: In particular, this illustrates the need for an iterative procedure involving models trained on multiple partitions of the remaining data in each iteration.

Original: We also report the k-nearest neighbors distances between examples in the train and heldout data in
Revised: We also report the k-nearest neighbors distances between examples in the train and heldout data in

Original: Table 3.
Revised: Table 3.

Original: We consider distances for examples within each class, as well as examples across classes.
Revised: We consider distances for examples within each class, as well as examples across classes.

Original: The distances are computed using cosine similarity between pooled features from BERT-based model (features for the [CLS] token, indicating a sentence-pair feature) trained on the original SNLI dataset.
Revised: The distances are computed using cosine similarity between pooled features from BERT-based model (features for the [CLS] token, indicating a sentence-pair feature) trained on the original SNLI dataset.

Original: Distances are measured between samples from the heldout data, and their nearest neighbors in the training data, before and after ﬁltering.
Revised: Distances are measured between samples from the heldout data, and their nearest neighbors in the training data, before and after ﬁltering.

Original: Distances generally increase after ﬁltering, indicating that A F L ITE promotes selecting a diverse set of examples from the dataset.
Revised: Distances generally increase after ﬁltering, indicating that A F L ITE promotes selecting a diverse set of examples from the dataset.

Original: The only exception to the rule is the neutral class, where distances to other classes decrease – this is not surprising since the neutral class is known to be associated with the least number of artifacts (Gururangan et al., 2018).
Revised: The only exception to the rule is the neutral class, where distances to other classes decrease – this is not surprising since the neutral class is known to be associated with the least number of artifacts (Gururangan et al., 2018).

Original: [Table] [Table]
Revised: [Table] [Table]

Revised: ( Lexical overlap , Subsequence , and Constituent ).

Revised: The NLI-Diagnostics dataset is broken down into the full dataset ( All ), as well as the instances requiring logical reasoning ( Logic ) and the ones requiring world and commonsense knowledge ( Knowledge ).

Revised: For Adversarial NLI, we ﬁnetuned both models on the in-distribution training data for each round ( Rd1 , Rd2 , and Rd3 ).

Original: MNLI and QNLI
Revised: MNLI and QNLI

Original: Following the same procedure described above, we apply A F L ITE on the MNLI and QNLI datasets.
Revised: Following the same procedure described above, we apply A F L ITE on the MNLI and QNLI datasets.

Original: Since RoBERTa resulted in the largest drops in performance across the board in SNLI, we only experiment with RoBERTa as adversary for MNLI and QNLI.
Revised: Since RoBERTa resulted in the largest drops in performance across the board in SNLI, we only experiment with RoBERTa as adversary for MNLI and QNLI.

Original: While RoBERTa achieves over 90% on both original datasets, its performance drops to 66 .
Revised: While RoBERTa achieves over 90% on both original datasets, its performance drops to 66 .

Original: 2% for MNLI and to 77 .
Revised: 2% for MNLI and to 77 .

Original: 7% for QNLI on the reduced datasets.
Revised: 7% for QNLI on the reduced datasets.

Original: Similarly, partial input baseline performance also decreases substantially on both dataset compared to their performance on the original dataset.
Revised: Similarly, partial input baseline performance also decreases substantially on both dataset compared to their performance on the original dataset.

Original: Table 2 shows these results.
Revised: Table 2 shows these results.

Original: We show that A F L ITE consistently result in reduced accuracy on the ﬁltered datasets across multiple NLP benchmark datasets, even after controlling for the size of the training set.
Revised: We show that A F L ITE consistently result in reduced accuracy on the ﬁltered datasets across multiple NLP benchmark datasets, even after controlling for the size of the training set.

Original: 3. I MAGENET E XPERIMENTS
Revised: 3.2. O UT - OF - DISTRIBUTION NLI

Revised: We measure the performance of A F L ITE on three other benchmarks for NLI evaluation, which provide out-of-distribution examples to challenge reliance on dataset biases in the original SNLI data (Glockner et al., 2018; Naik et al., 2018). Such benchmarks approximate the performance of NLI models in the wild. NLI Diagnostics (Wang et al., 2018) is a set of hand-crafted examples designed to demonstrate model performance on several ﬁne-grained semantic categories, such as logical reasoning and commonsense knowledge.

Revised: HANS (McCoy et al., 2019) contains evaluation examples designed to avoid common structural heuristics (such as word overlap) which could be used by models to correctly predict NLI inputs, without true inferential reasoning.

Revised: Adversarial NLI (Nie et al., 2019) consists of premises collected from Wikipedia and other news corpora, and human generated hypotheses, arranged at different tiers of the challenge they present to a model, using a human and model in-the-loop procedure.

Revised: Given that these benchmarks are collected independently of the original SNLI task, the biases from SNLI are less likely to carry over; however these benchmarks might contain their own biases (Liu et al., 2019a).

Revised: A F L ITE assigns a predictability score to all samples in a dataset, resulting in an ordering of the data.

Revised: Filtering out examples from the head of the data distribution based on this order yields more accurate benchmarks for measuring true model performance.

Revised: On the other hand, transferability to out-of-distribution data would involve a greater balance between examples from the head and the tail ends of the data distribution.

Revised: Hence we evaluate A F L ITE for generalization using a larger ﬁltered subset, amounting to only a third of the full training data.

Revised: We present these results on all the above benchmarks in Table 4.

Revised: On the two diagnostic datasets (HANS and NLI-Diagnostics), we perform a zero-shot evaluation of the two models.

Revised: Adversarial NLI allows to test for transfer

Revised: [Table]

Revised: capabilities, by ﬁnetuning these models on each of the three training datasets ( Rd1 , Rd2 and Rd3 ).

Revised: On each of the benchmarks above, the model trained on the A F L ITE data consistently outperforms the model trained on the full SNLI data.

Revised: challenging examples in the HANS benchmark, which targets models purely relying on lexical and syntactic cues.

Revised: Similarly, our model performs better on the instances in NLI-Diagnostics that require logical reasoning and commonsense knowledge, as opposed to instances that can be solved through lexical entailment alone.

Revised: 3. I MAGE N ET E XPERIMENTS

Original: We evaluate A F L ITE on image classiﬁcation through ImageNet (ILSVRC2012) classiﬁcation.
Revised: We evaluate A F L ITE on image classiﬁcation through ImageNet (ILSVRC2012) classiﬁcation.

Original: On
Revised: On

Original: ImageNet, we use the state-of-the-art EfﬁcientNet-B7 model as our core feature extractor Φ (Tan &
Revised: ImageNet, we use the state-of-the-art EfﬁcientNet-B7 model as our core feature extractor Φ (Tan &

Original: Le, 2019).
Revised: Le, 2019).

Original: The EfﬁcientNet model is learned from scratch on a ﬁxed 20% sample of the ImageNet training set, using AutoAugment data augmentation (Cubuk et al., 2019).
Revised: The EfﬁcientNet model is learned from scratch on a ﬁxed 20% sample of the ImageNet training set, using AutoAugment data augmentation (Cubuk et al., 2019).

Original: We then use the 2560dimensional features extracted by EfﬁcientNet-B7 as then underlying representation for A F L ITE to use to ﬁlter the remaining dataset.
Revised: We then use the 2560dimensional features extracted by EfﬁcientNet-B7 as then underlying representation for A F L ITE to use to ﬁlter the remaining dataset.

Original: In Table 4, we evaluate the robustness of the ﬁltered dataset by considering ImageNet accuracy across the EfﬁcientNet and ResNet model families (He et al., 2016).
Revised: In Table 5, we evaluate the robustness of the ﬁltered dataset by considering ImageNet accuracy across the EfﬁcientNet and ResNet model families (He et al., 2016).

Original: When lowering the size of the training set – down to 20% of the original, we ﬁnd a large drop in performance.
Revised: When lowering the size of the training set – down to 20% of the original, we ﬁnd a large drop in performance.

Original: The EfﬁcientNet models seem to suffer less – from 84% to 73% on EfﬁcientNet-B7 versus 80 .
Revised: The EfﬁcientNet models seem to suffer less – from 84% to 73% on EfﬁcientNet-B7 versus 80 .

Original: 6% to 56 .
Revised: 6% to 56 .

Original: 5% on
Revised: 5% on

Original: ResNet-152.
Revised: ResNet-152.

Original: However, the biggest performance drop comes from training and evaluating on the
Revised: However, the biggest performance drop comes from training and evaluating on the

Original: A F L ITE -ﬁltered dataset: the top performer is still EfﬁcientNet-B7, but its accuracy drops to 24 .
Revised: A F L ITE -ﬁltered dataset: the top performer is still EfﬁcientNet-B7, but its accuracy drops to 24 .

Original: 5% top-1.
Revised: 5% top-1.

Original: This is despite controlling for dataset size, as well as discrepancy between the training and validation sets.
Revised: This is despite controlling for dataset size, as well as discrepancy between the training and validation sets.

Original: Overall, these results suggest that image classiﬁcation – even within a subset of the closed world of ImageNet – is far from solved.
Revised: Overall, these results suggest that image classiﬁcation – even within a subset of the closed world of ImageNet – is far from solved.

Original: These results echo other ﬁndings that suggest that common biases that naturally occur in web-scale image data, such as towards canonical poses (Alcorn et al., 2019) or towards texture rather than shape (Geirhos et al., 2018), are problems for ImageNet-trained classiﬁers.
Revised: These results echo other ﬁndings that suggest that common biases that naturally occur in web-scale image data, such as towards canonical poses (Alcorn et al., 2019) or towards texture rather than shape (Geirhos et al., 2018), are problems for ImageNet-trained classiﬁers.

Original: Indeed, the randomly-selected ImageNet images in Figure 1 suggest that the A F L ITE algorithm learns to identify subsets of the data that are particularly challenging.[Table]
Revised: Indeed, the randomly-selected ImageNet images in Figure 1 suggest that the A F L ITE algorithm learns to identify subsets of the data that are particularly challenging.

Original:  R ELATED W ORK
Revised:  R ELATED W ORK

Original: Our proposed framework for artifact reduction is related to the adversarial ﬁltering (AF) algorithm in
Revised: Our proposed framework for artifact reduction is related to the adversarial ﬁltering (AF) algorithm in

Original: Zellers et al.
Revised: Zellers et al.

Original: (2018), yet distinct in two key ways: our approach is (i) much more broadly applicable (by not requiring over generation of data instances) [Table] and (ii) considerably more lightweight (by not requiring re-training a model at each iteration of AF).
Revised: (2018), yet distinct in two key ways: our approach is (i) much more broadly applicable (by not requiring over generation of data instances), and (ii) considerably more lightweight (by not requiring re-training a model at each iteration of AF).

Original: Variants of this AF approach have recently been used to create other datasets such as HellaSwag (Zellers et al., 2019) and ANLI (Bhagavatula et al., 2019) by iteratively perturbing dataset instances until a target model cannot ﬁt the resulting dataset.
Revised: Variants of this AF approach have recently been used to create other datasets such as HellaSwag (Zellers et al., 2019) and ANLI (Bhagavatula et al., 2019) by iteratively perturbing dataset instances until a target model cannot ﬁt the resulting dataset.

Original: While effective, these approaches run into three main pitfalls.
Revised: While effective, these approaches run into three main pitfalls.

Original: First, dataset curators need to explicitly devise a strategy of collecting or generating perturbations of a given instance.
Revised: First, dataset curators need to explicitly devise a strategy of collecting or generating perturbations of a given instance.

Original: Second, the approach runs the risk of distributional bias where a discriminator can learn to distinguish between machine generated instances and human-generated ones.
Revised: Second, the approach runs the risk of distributional bias where a discriminator can learn to distinguish between machine generated instances and human-generated ones.

Original: Finally it requires re-training a model at each iteration, which is computationally expensive especially when using a large model such as
Revised: Finally it requires re-training a model at each iteration, which is computationally expensive especially when using a large model such as

Original: BERT (Devlin et al., 2019) as the adversary.
Revised: BERT (Devlin et al., 2019) as the adversary.

Original: In contrast, A F L ITE focuses on addressing dataset biases from existing datasets instead of adversarially perturbing instances.
Revised: In contrast, A F L ITE focuses on addressing dataset biases from existing datasets instead of adversarially perturbing instances.

Original: A F L ITE was earlier proposed by Sakaguchi et al.
Revised: A F L ITE was earlier proposed by Sakaguchi et al.

Original: (2019) to create the Winogrande dataset.
Revised: (2019) to create the Winogrande dataset.

Original: This paper presents more thorough experiments, theoretical justiﬁcation and results from generalizing the proposed approach to multiple popular NLP and Vision datasets.
Revised: This paper presents more thorough experiments, theoretical justiﬁcation and results from generalizing the proposed approach to multiple popular NLP and Vision datasets.

Original: A F L ITE is also inspired by Gururangan et al.
Revised: A F L ITE is also inspired by Gururangan et al.

Original: (2018), who study lexical biased prevalent in the SNLI dataset (Bowman et al., 2015) and use point-wise mutual information (PMI) between a word and an inference class to determine the words that are highly indicative of the target label.
Revised: (2018), who study lexical biased prevalent in the SNLI dataset (Bowman et al., 2015) and use point-wise mutual information (PMI) between a word and an inference class to determine the words that are highly indicative of the target label.

Original: Instead of lexical features, we adopt a deeper representation of the instances using their pre-computed dense feature representations.
Revised: Instead of lexical features, we adopt a deeper representation of the instances using their pre-computed dense feature representations.

Original: We use an ensemble of linear classiﬁers trained on random subsets of the data to determine whether the dense feature representations are highly indicative of the target label.
Revised: We use an ensemble of linear classiﬁers trained on random subsets of the data to determine whether the dense feature representations are highly indicative of the target label.

Original: If so, we discard the corresponding instances and proceed iteratively.
Revised: If so, we discard the corresponding instances and proceed iteratively.

Original: Li & Vasconcelos (2019) recently proposed REPAIR, a method to remove representation bias by dataset resampling.
Revised: Li & Vasconcelos (2019) recently proposed REPAIR, a method to remove representation bias by dataset resampling.

Original: While resampling is a common technique for balancing datasets, the motivation in REPAIR is to learn a probability distribution over the dataset that favors instances that are hard for a given representation.
Revised: While resampling is a common technique for balancing datasets, the motivation in REPAIR is to learn a probability distribution over the dataset that favors instances that are hard for a given representation.

Original: This approach targets how to train better, less-biased models as opposed to creating datasets with fewer artifacts.
Revised: This approach targets how to train better, less-biased models as opposed to creating datasets with fewer artifacts.

Original: In addition, the implementation of REPAIR relies on intraining classiﬁcation loss as opposed to out-of-sample generalization accuracy.
Revised: In addition, the implementation of REPAIR relies on intraining classiﬁcation loss as opposed to out-of-sample generalization accuracy.

Original: RESOUND (Li et al., 2018) is another method that quantiﬁes the representation biases of datasets.
Revised: RESOUND (Li et al., 2018) is another method that quantiﬁes the representation biases of datasets.

Original: It uses the representation biases to assemble a new K-class dataset with smaller biases by sampling an existing
Revised: It uses the representation biases to assemble a new K-class dataset with smaller biases by sampling an existing

Original: C-class dataset ( C > K ).
Revised: C-class dataset ( C > K ).

Original: Arjovsky et al.
Revised: Arjovsky et al.

Original: (2019) argue that unstable, spurious correlations in the data would generalize poorly to novel test environments.
Revised: (2019) argue that unstable, spurious correlations in the data would generalize poorly to novel test environments.

Original: Thus, they propose Invariant Risk Minimization as an objective that promotes learning representations of the data which are stable across environments.
Revised: Thus, they propose Invariant Risk Minimization as an objective that promotes learning representations of the data which are stable across environments.

Original: Instead of learning optimal classiﬁers, our aim is to remove instances that exhibit artifacts in a dataset.
Revised: Instead of learning optimal classiﬁers, our aim is to remove instances that exhibit artifacts in a dataset.

Original:  C ONCLUSION
Revised:  C ONCLUSION

Original: We presented A F L ITE – a novel iterative greedy algorithm that adversarially ﬁlters out data points to arrive at a reduced dataset with more realistic problem distributions and considerably fewer spurious biases.
Revised: We presented A F L ITE – a novel iterative greedy algorithm that adversarially ﬁlters out data points to arrive at a reduced dataset with more realistic problem distributions and considerably fewer spurious biases.

Original: We apply A F L ITE to four widely-used datasets, including SNLI and ImageNet, where reported performance is extremely high – and show that state-of-the-art performance on the resulting ﬁltered dataset drops by 30 points for SNLI and drops from 84.4% to 24.5% Top-1 accuracy for ImageNet.
Revised: We apply A F L ITE to four widely-used datasets, including SNLI and ImageNet, where reported performance is extremely high – and show that state-of-the-art performance on the resulting ﬁltered dataset drops by 30 points for SNLI and drops from 84.4% to 24.5% Top-1 accuracy for ImageNet.

Original: In extensive analysis we show that A F L ITE is effective on real as well as synthetic datasets.
Revised: In extensive analysis we show that A F L ITE is effective on real as well as synthetic datasets.

Original: We hope that dataset creators will employ A F L ITE to identify unobservable artifacts before releasing new challenge datasets for the research community in order to have a more reliable estimate of model performance on future AI benchmarks
Revised: We hope that dataset creators will employ A F L ITE to identify unobservable artifacts before releasing new challenge datasets for the research community in order to have a more reliable estimate of model performance on future AI benchmarks.