Original: On the Limitations of Dataset Balancing: The Lost Battle Against Spurious Correlations
Revised: On the Limitations of Dataset Balancing: The Lost Battle Against Spurious Correlations

Original: Anonymous ACL submission

Original: Abstract
Revised: Abstract

Original: Recent work have shown that deep learning models in NLP are highly sensitive tolow-level correlations between simple featuresand speciﬁc output labels, leading to overﬁtting and lack of generalization.
Revised: Recent work has shown that deep learning models in NLP are highly sensitive to lowlevel correlations between simple features and speciﬁc output labels, leading to overﬁtting and lack of generalization.

Original: To mitigate this problem, a common practice is tobalance datasets by adding new instances orby ﬁltering out “easy” instances (Sakaguchiet al., 2020), culminating in a recent proposal to eliminate single-word correlations altogether (Gardner et al., 2021).
Revised: To mitigate this problem, a common practice is to balance datasets by adding new instances or by ﬁltering out “easy” instances (Sakaguchi et al., 2020), culminating in a recent proposal to eliminate single-word correlations altogether (Gardner et al., 2021).

Original: In this opinion paper, we identify that despite these efforts,increasingly-powerful models keep exploitingever-smaller spurious correlations, and as a result even balancing all single-word features isinsufﬁcient for mitigating all of these correlations.
Revised: In this opinion paper, we identify that despite these efforts, increasingly-powerful models keep exploiting ever-smaller spurious correlations, and as a result even balancing all single-word features is insufﬁcient for mitigating all of these correlations.

Original: In parallel, a truly balanced datasetmay be bound to “throw the baby out with thebathwater” and miss important signal encoding common sense and world knowledge.
Revised: In parallel, a truly balanced dataset may be bound to “throw the baby out with the bathwater” and miss important signal encoding common sense and world knowledge.

Original: Wehighlight several alternatives to dataset balancing, focusing on enhancing datasets with richercontexts, allowing models to abstain and interact with users, and turning from large-scaleﬁne-tuning to zero- or few-shot setups.
Revised: We highlight several alternatives to dataset balancing, focusing on enhancing datasets with richer contexts, allowing models to abstain and interact with users, and turning from large-scale ﬁne-tuning to zero- or few-shot setups.

Original:  Introduction
Revised:  Introduction

Original: Effective human communication relies on our ability to understand extra-textual context based oncommon sense, world knowledge or shared cultural experiences, a property often cited as Grice’ssecond maxim of quantity: “Do not make your contribution more informative than is required” (Grice,1975, 1989).
Revised: Effective human communication relies on our ability to understand extra-textual context based on common sense, world knowledge or shared cultural experiences, a property often cited as Grice’s second maxim of quantity: “Do not make your contribution more informative than is required” (Grice, 1975, 1989).

Original: Studies have estimated that only 12%of the information conveyed by text is mentionedexplicitly (Graesser, 2013; Tandon et al., 2020).
Revised: Studies have estimated that only 12% of the information conveyed by text is mentioned explicitly (Graesser, 2013; Tandon et al., 2020).

Original: Toillustrate this, consider the question “ who is thepresident of the U.S. ?”.
Revised: To illustrate this, consider the question “ who is the president of the U.S. ?”.

Original: To answer it, a humanreader is likely to presume many unstated propositions, as exempliﬁed in Tab. 1.
Revised: To answer it, a human reader is likely to presume many unstated propositions, as exempliﬁed in Tab. 1.

Original: In contrast to humans, supervised models often fail to generalize and understand implicit context, instead resorting to low-level correlations in
Revised: In contrast to humans, supervised models often fail to generalize and understand implicit context, instead resorting to low-level correlations in

Original: Who is the president of the U.S.?
Revised: Who is the president of the U.S.?

Original: Context
Revised: Context

Original: Answer ∅
Revised: Answer [Table]

Original: Joe Biden
Revised: Josiah

Original: The year Donald Trump The West Wing, season
Revised: [Table]

Original: Josiah “Jed” Bartlet the data, leading to ampliﬁed bias (Zhao et al., 2017; Stanovsky et al., 2019) and brittle performance (Schwartz et al., 2017; Gururangan et al., 2018).
Revised: Bartlet the data, leading to ampliﬁed bias (Zhao et al., 2017; Stanovsky et al., 2019) and brittle performance (Schwartz et al., 2017; Gururangan et al., 2018).

Original: To address this, recent approaches have suggested mitigating such correlations by balancing the dataset via either adding or removing certain instances (Goyal et al., 2017; Hudson and Manning, 2019; Zellers et al., 2018; Sakaguchi et al., 2020).
Revised: To address this, recent approaches have suggested mitigating such correlations by balancing the dataset via either adding or removing certain instances (Goyal et al., 2017; Hudson and Manning, 2019; Zellers et al., 2018; Sakaguchi et al., 2020).

Original: In parallel, developers keep building larger and larger pretrained models (Devlin et al., 2019;
Revised: In parallel, developers keep building larger and larger pretrained models (Devlin et al., 2019;

Original: Liu et al., 2019; Raffel et al., 2020), which, when ﬁne-tuned on these datasets, consistently manage to reach human performance.
Revised: Liu et al., 2019; Raffel et al., 2020), which, when ﬁne-tuned on these datasets, consistently manage to reach human performance.

Original: Taken together, these trends lead to an arms-race between data curation and model development (Fig. 1).
Revised: Taken together, these trends lead to an arms-race between data curation and model development (Fig. 1).

Original: In this position paper, we question the value ofmitigating spurious correlations via dataset balancing, by showing that their existence in large trainingsets is both inevitable and to some extent even desired, as they are an inherent property of naturallanguage understanding.
Revised: In this position paper, we question the value of mitigating spurious correlations via dataset balancing, by showing that their existence in large training sets is both inevitable and to some extent even desired, as they are an inherent property of natural language understanding.

Original: We build on a recent result by Gardner et al.
Revised: We build on a recent result by Gardner et al.

Original: (2021), who assumed thatevery single-word feature correlation is spurious ,i.e., can be used to mislead a model.
Revised: (2021), who assumed that every single-word feature correlation is spurious , i.e., can be used to mislead a model.

Original: We extendtheir argument, showing that balancing single-wordfeatures is insufﬁcient for eliminating all spuriouscorrelations, and that balancing feature combination is needed for that purpose.
Revised: We extend their argument, showing that balancing single-word features is insufﬁcient for eliminating all spurious correlations, and that balancing feature combination is needed for that purpose.

Original: On the other hand,we show that balancing too much leads to datasetsthat contain no learnable signal either.
Revised: On the other hand, we show that balancing too much leads to datasets that contain no learnable signal either.

Original: We concludeby questioning whether mitigating all spurious correlations via dataset balancing is practical .
Revised: We conclude by questioning whether mitigating all spurious correlations via dataset balancing is practical .

Original: Following, we show that this practice is also undesired .
Revised: Following, we show that this practice is also undesired .

Original: We show that ignoring these correlationswill hinder the learning of fallback options for bothworld knowledge facts ( Joe Biden is the presidentof the U.S. ) and common sense knowledge ( a person is happy when receiving a gift ), thus preventing models from using this knowledge in casesof uncertainty.
Revised: We show that ignoring these correlations will hinder the learning of fallback options for both world knowledge facts ( Joe Biden is the president of the U.S. ) and common sense knowledge ( a person is happy when receiving a gift ), thus preventing models from using this knowledge in cases of uncertainty.

Original: We conclude that the existence ofspurious correlations in training sets should not besolved by creating more balanced datasets.
Revised: We conclude that the existence of spurious correlations in training sets should not be solved by creating more balanced datasets.

Original: We then discuss alternatives to mitigating spurious correlations.
Revised: We then discuss alternatives to mitigating spurious correlations.

Original: We argue that models should betrained to understand constructions emanating froman apriori theory of language, such as negation, sarcasm, humor, and metaphors.
Revised: We argue that models should be trained to understand constructions emanating from an apriori theory of language, such as negation, sarcasm, humor, and metaphors.

Original: We also suggestadopting modeling approaches that identify whenthe context is insufﬁcient, and the model shouldnot fallback to default assumptions, but rather output an “I don’t know” response (e.g., unanswerable questions, Rajpurkar et al., 2018; Sulem et al.,2021) or interact with the user to clear ambiguities.
Revised: We also suggest adopting modeling approaches that identify when the context is insufﬁcient. We argue that in such cases, the model should not fallback to default assumptions, but rather abstain or interact with the user to clear ambiguities.

Original: We conclude by questioning the basic procedureof large-scale ﬁne-tuning, and suggest focusing onzero- and few-shot learning.
Revised: Finally, we question the basic procedure of large-scale ﬁne-tuning, and suggest focusing on zero- and few-shot learning instead (Liu et al., 2021b).

Original:  Dataset-Model Arms Race
Revised:  Dataset-Model Arms Race

Original: This section provides a view of recent research in
Revised: This section provides a view of recent research in

Original: NLP as an arms race between models and datasets.
Revised: NLP as an arms race between models and datasets.

Original: Below we describe the conditions leading to this arms race, and present our main research question, challenging its value for making progress in NLP.
Revised: Below we describe the conditions leading to this arms race, and present our main research question, challenging its value for making progress in NLP.

Original: Dataset balancing via augmentation
Revised: Models exploit spurious correlations

Original: While pretrained models consistently perform well across multiple tasks, various studies have pointed out that this is often achieved by exploiting spurious correlations in datasets, rather than improving on the underlying task (Glockner et al., 2018; Gururangan et al., 2018; Elazar et al.,  2021).
Revised: While pretrained models consistently perform well across multiple tasks, various studies have pointed out that this is often achieved by exploiting spurious correlations in datasets, rather than improving on the underlying task (Glockner et al., 2018; Gururangan et al., 2018; Elazar et al., 2021), and that this phenomenon becomes more prominent as the models grow in size (Li et al., 2021).

Original:  Various dataset curators have tried to prevent models from learning spurious correlations by modifying their training data via a careful control for the training label distribution, effectively striving for a balanced dataset.
Revised: Mitigating spurious correlations via balancing Various dataset curators have tried to prevent models from learning spurious correlations by modifying their training data via a careful control for the training label distribution, effectively striving for a balanced dataset.

Original: One approach, popular in visual question answering datasets, is to add examples in order to balance the dataset (Goyal et al., 2017;  Hudson and Manning, 2019).
Revised: One approach is to add examples in order to balance the dataset (Goyal et al., 2017; Sharma et al., 2018; Hudson and Manning, 2019).

Original: For instance, the VQA2.0 dataset (Goyal et al., 2017) is built by taking every (question q , image i , answer a ) triplet in the VQA dataset (Antol et al., 2015), and adding another triplet with the same question q , but a different image i (cid:48) , guaranteed to lead to a different answer a (cid:48) .
Revised: For instance, the VQA2.0 dataset (Goyal et al., 2017) is built by taking every (question q , image i , answer a ) triplet in the VQA dataset (Antol et al., 2015), and adding another triplet with the same question q , but a different image i (cid:48) , guaranteed to lead to a different answer a (cid:48) .

Original: See Fig.
Revised: See Fig.

Original: 2 for an example.
Revised: 2 for an example.

Original: Filtering as balancing
Revised: Filtering as balancing

Original: A complementary  approach to augmentation is ﬁltering examples out from datasets such that spurious correlations are minimized.
Revised: A complementary balancing approach to augmentation is ﬁltering examples out from datasets such that spurious correlations are minimized.

Original: This approach was taken in the creation of the SWAG dataset (Zellers et al., 2018), using “adversarial ﬁltering” (AF).
Revised: This approach was taken in the creation of the SWAG dataset (Zellers et al., 2018), using “adversarial ﬁltering” (AF).

Original: In AF, dataset instances that are easily solved by an adversarial model are ﬁltered out.
Revised: In AF, dataset instances that are easily solved by an adversarial model are ﬁltered out.

Original: The AF approach was picked up by manyfollow-up datasets such as DROP (Dua et al., 2019), HellaSWAG
Revised: The AF approach and similar approaches were picked up by many datasets such as ReCoRD (Zhang et al., 2018),

Original:  (Zellers et al., 2019),  and WinoGrande (Sakaguchi et al., 2020).
Revised: DROP (Dua et al., 2019), HellaSWAG (Zellers et al., 2019), α NLI (Bhagavatula et al., 2020), and WinoGrande (Sakaguchi et al., 2020).

Original: Here we argue that approaches like AF converge to removing all low-level correlations, 2 and there- fore a fully balanced dataset.
Revised: Here we argue that approaches like AF converge to removing all low-level correlations, 2 and therefore a fully balanced dataset.

Original: As this approachrelies on an external model, applying it with everstronger models with higher capacity, will allowthese models to pick up on subtler correlations.
Revised: As this approach relies on an external model, applying it with ever stronger models with higher capacity, will allow these models to pick up on subtler correlations (Li et al., 2021).

Original: Atthe extreme, the remaining instances that could notbe solved by a fully capable model will have no statistical signal that can be exploited by that model,i.e., a balanced dataset.
Revised: At the extreme, the remaining instances that could not be solved by a fully capable model will have no statistical signal that can be exploited by that model, i.e., a balanced dataset.

Original: We henceforth refer to bothaugmentation and ﬁltering as balancing methods.
Revised: We henceforth refer to both augmentation and ﬁltering as balancing methods.

Original: Large models solve the new datasets
Revised: Large models solve the new datasets

Original: In parallel to the efforts in dataset balancing, the leadingmodeling approach in recent years in NLP is pretraining large language models on raw text corpora, followed by ﬁne-tuning them on superviseddownstream applications.
Revised: In parallel to the efforts in dataset balancing, the leading modeling approach in recent years in NLP is pretraining large language models on raw text corpora, followed by ﬁne-tuning them on supervised downstream applications.

Original: These models continueto grow in size (Peters et al., 2018; Devlin et al. Liu et al., 2019; Radford et al., 2019; Raffel et al., 2020), and their ﬁne-tuning performanceimproves accordingly.
Revised: These models continue to grow in size (Peters et al., 2018; Devlin et al., 2019; Liu et al., 2019; Radford et al., 2019; Raffel et al., 2020), and their ﬁne-tuning performance improves accordingly.

Original: This in turn leads to moreaggressive balancing, setting in motion a kind ofarms race between datasets and models (Fig. 1).
Revised: This in turn leads to more aggressive balancing, setting in motion a kind of arms race between datasets and models (Fig. 1).

Original: Evidently, a similar trend emerges for the previously mentioned datasets: (1) the ﬁrst baselines, reﬂecting the state of the art at the time ofdataset creation, perform  poorly, e.g., 52% accuracy on SWAG,  47 F1 on DROP, 47% on HellaSWAG,  and 53% AUC on WinoGrande; (2)model developers introduce increasingly larger andheavily-parameterized models, hill-climbing onthese datasets; and eventually (3) models essentially solve the dataset within a year or two, oftenoutperforming humans: 86% on SWAG (Devlinet al., 2019), 90 F1 on DROP (Chen et al., 2020),93% on HellaSWAG(He et al., 2020), and 88%
Revised: Evidently, a similar trend emerges for the previously mentioned datasets: (1) the ﬁrst baselines, reﬂecting the state of the art at the time of dataset creation, perform relatively poorly, e.g., 59% on SWAG, 47% on ReCoRD, 47 F1 on DROP, 47% on HellaSWAG, 69% on α NLI, and 79% on WinoGrande; (2) model developers introduce increasingly larger and heavily-parameterized models, hill-climbing on these datasets; and eventually (3) models essentially solve the dataset within a year or two, often outperforming humans: 86% on SWAG (Devlin et al., 2019), 94% on ReCoRD (He et al., 2021b), 88 F1 on DROP (Chen et al., 2020), 93% on HellaSWAG

Original: AUC on WinoGrande (Raffel et al., 2020).
Revised: (He et al., 2021b), 92% on α NLI (He et al., 2021a), and 90% on WinoGrande (Raffel et al., 2020).

Original: (4) newlarge-scale datasets are collected with more aggressive pruning techniques, thus repeating the cycle.
Revised: (4) new large-scale datasets are collected with more aggressive pruning techniques, thus repeating the cycle.

Original: Based on these ﬁndings, our main researchquestion is whether dataset balancing is the mostpromising method for mitigating spurious correlations.
Revised: Based on these ﬁndings, our main research question is whether dataset balancing is the most promising method for mitigating spurious correlations.

Original: We note that an arms race between modelsand datasets might spur advances.
Revised: We note that an arms race between models [Table] and datasets might spur advances.

Original: Here we question a speciﬁc aspect of this arms race: the improvement of datasets by using more aggressive ﬁlteringtechniques.
Revised: Here we question a speciﬁc aspect of this arms race: the improvement of datasets by using more aggressive ﬁltering techniques.

Original: Next we turn to present practical andconceptual limitations of this practice.
Revised: Next we turn to present practical and conceptual limitations of this practice.

Original:  The Lost Battle Against Spurious 
Revised:  The Lost Battle Against Spurious Correlations

Original: Correlations

Original: So far we have identiﬁed dataset balancing as acommon way to mitigate spurious correlations.
Revised: So far we have identiﬁed dataset balancing as a common way to mitigate spurious correlations.

Original: [Table]

Original: Next, we outline how different works deﬁne spurious correlations (Sec. 3.1), and then question whether dataset balancing is a viable way for mitigating them; we note that balancing too little is bound to leave spurious correlations in the data (Sec. 3.2), while balancing too much discards meaningful signal (Sec. 3.3).
Revised: Next, we outline how different works deﬁne spurious correlations (Sec. 3.1), and then question whether dataset balancing is a viable way for mitigating them; we note that balancing too little is bound to leave spurious correlations in the data (Sec. 3.2), while balancing too much discards meaningful signal (Sec. 3.3).

Original: We ﬁnish by questioning whether this practice is even desired (Sec. 3.4).
Revised: We ﬁnish by questioning whether this practice is even desired (Sec. 3.4).

Original: 3. What are Spurious Correlations?
Revised: 3. What are Spurious Correlations?

Original: Mitigating spurious correlations is frequently used as motivation for developing new balancing approaches.
Revised: Mitigating spurious correlations is frequently used as motivation for developing new balancing approaches.

Original: However, the term spurious correlations is often not clearly and consistently deﬁned.
Revised: However, the term spurious correlations is often not clearly and consistently deﬁned.

Revised: The basic deﬁnition is a set of features that are correlated but not causally related.

Revised: In NLP, several deﬁnitions of spurious correlations are typically used.

Original: One conceptual deﬁnition, denoted here ingenuine (e.g., Wang and Culotta, 2020; Rogers, 2021) is a feature correlated with some output label for no apparent reason.
Revised: One conceptual deﬁnition, denoted here ingenuine (e.g., Wang and Culotta, 2020; Rogers, 2021) is a feature correlated with some output label for no apparent reason.

Original: Such features often result from the annotation process (referred to as annotation artifacts ; Gururangan et al., 2018).
Revised: Such features often result from the annotation process (referred to as annotation artifacts ; Gururangan et al., 2018).

Original: For instance, Gururangan et al.
Revised: For instance, Gururangan et al.

Original: (2018) have shown that the words “cat” and “sleeping” are correlated with contradictions in the SNLI dataset (Bowman et al., 2015).
Revised: (2018) have shown that the words “cat” and “sleeping” are correlated with contradictions in the SNLI dataset (Bowman et al., 2015).

Original: This deﬁnition is appealing: we want our models to learn real information about the world, and not properties of a given dataset.
Revised: This deﬁnition is appealing: we want our models to learn real information about the world, and not properties of a given dataset.

Original: However, it is also somewhat subjective, and could include features that might be referred to as genuine, such as the word “not” indicating NLI contradictions.
Revised: However, it is also somewhat subjective, and could include features that might be referred to as genuine, such as the word “not” indicating NLI contradictions.

Original: Further, genuine features, i.e., those representing a real phenomenon in the world (e.g., “amazing” as a feature for positive sentiment), are also likely to lead models make to erroneous predictions in some contexts (e.g., negation or sarcasm; Gardner et al., 2021).
Revised: Further, genuine features, i.e., those representing a real phenomenon in the world (e.g., “amazing” as a feature for positive sentiment), are also likely to lead models make to erroneous predictions in some contexts (e.g., negation or sarcasm; Gardner et al., 2021).

Original: Such features could thus harm generalization, so some might consider them spurious as well.
Revised: Such features could thus harm generalization, so some might consider them spurious as well.

Original: In an alternative deﬁnition, denoted ungeneralizable , a spurious feature is one that works well for speciﬁc examples but does not hold in general (Chang et al., 2021;
Revised: In an alternative deﬁnition, denoted ungeneralizable , a spurious feature is one that works well for speciﬁc examples but does not hold in general (Chang et al., 2021;

Original: Yaghoobzadeh et al., 2021).
Revised: Yaghoobzadeh et al., 2021).

Original: This deﬁnition does not address the nature of the feature (genuine or[Table] not), but does make an implicit assumption thatsuch features are of high importance (e.g., highpointwise mutual information values with thecorresponding label; Gururangan et al., 2018).
Revised: This deﬁnition does not address the nature of the feature (genuine or not), but does make an implicit assumption that such features are of high importance (e.g., high pointwise mutual information values with the corresponding label; Gururangan et al., 2018).

Original: This deﬁnition is no longer subjective in terms ofthe genuineness of the feature, but is still subjectivein the level of effect on generalizability (i.e., whatis a high value of PMI?).
Revised: This deﬁnition is no longer subjective in terms of the genuineness of the feature, but is still subjective in the level of effect on generalizability (i.e., what is a high value of PMI?).

Original: Gardner et al.
Revised: Gardner et al.

Original: (2021) relaxed the last constraint,and assumed that every simple correlation betweensingle word features and output labels is spurious(henceforth every-word ).
Revised: (2021) relaxed the last constraint, and assumed that every simple correlation between single word features and output labels is spurious (henceforth every-word ).

Original: They then deﬁned aclass of competent datasets, where the marginalprobability for every feature is uniform over theclass label, i.e., ∀ x ∈ X , y ∈ Y, p ( y | x  )[-= 
-]
Revised: They then deﬁned a class of competent datasets, where the marginal probability for every feature is uniform over the class label, i.e., for any feature x i and label y ∈ Y , p ( y | x i )[-= 
-]

Original:  | Y | , thuslimiting models from picking up any correlationbetween single features and output labels.
Revised: = 1 | Y | , thus limiting models from picking up any correlation between single features and output labels.

Original: We next extend the every-word approach beyond single words, showing that models that canexploit single word features can also exploit somefeature interactions, and therefore these should alsobe considered spurious.
Revised: We next extend the every-word approach beyond single words, showing that models that can exploit single word features can also exploit some feature interactions, and therefore these should also be considered spurious.

Original: Tab.
Revised: Tab.

Original: 2 summarizes the different deﬁnitions of spurious correlations.
Revised: 2 summarizes the different deﬁnitions of spurious correlations.

Original: 3. Balancing too Little Leaves someSpurious Features
Revised: 3. Balancing too Little Leaves some Spurious Features

Original: Gardner et al.
Revised: Gardner et al.

Original: (2021) assumed that as each wordcan appear in certain contexts that change its semantic meaning (e.g., negation, sarcasm), eachword is potentially spurious.
Revised: (2021) assumed that as each word can appear in certain contexts that change its semantic meaning (e.g., negation, sarcasm), each word is potentially spurious.

Original: Here we note thatthe same argument can be applied to feature interactions, such as word n -grams.
Revised: Here we note that the same argument can be applied to feature interactions, such as word n -grams.

Original: We start with a toyexample to illustrate our argument for bigrams, andthen extend it for larger values of n .
Revised: We start with a toy [Table] example to illustrate our argument for bigrams, and then extend it for larger values of n .

Original: Consider the toy dataset for the task of sentiment analysis shown in Tab. 3, with vocabulary
Revised: Consider the toy dataset for the task of sentiment analysis shown in Tab. 3, with vocabulary

Original: V ={ good,bad,not,very }, and label set Y ={ + , − }.
Revised: V ={ good, bad, not, very }, and label set Y = { + , − }.

Original: The Train split is balanced with respect to singleword features, i.e., ∀ w ∈ V, y ∈ Y : p ( y | w ) = 1 | Y |(a balanced or competent dataset).
Revised: The Train split is balanced with respect to single-word features, i.e., it is a balanced or competent dataset: [Equation]

Original: Assume the semantics of this dataset is that of English, while ‘ + ’means positive sentiment and ‘ − ’ means negative.
Revised: Assume the semantics of this dataset is that of English, while ‘ + ’ means positive sentiment and ‘ − ’ means negative.

Original: A model trained on Train can achieve perfect training accuracy by learning the correct semantics.
Revised: A model trained on Train can achieve perfect training accuracy by learning the correct semantics.

Original: However, achieving perfect training accuracy can also be done by learning correlations between twoword features and the target label (i.e., memorizing all the training examples).
Revised: However, achieving perfect training accuracy can also be done by learning correlations between twoword features and the target label (i.e., memorizing all the training examples).

Original: In this case, the model would make the wrong prediction for the ﬁrst test example in Test (as it has learned that very good is a feature that indicates positive sentiment), and similarly, will make a random prediction for the second test example, which does not contain any two-word features seen during training.
Revised: In this case, the model would make the wrong prediction for the ﬁrst test example in Test (as it has learned that very good is a feature that indicates positive sentiment), and similarly, will make a random prediction for the second test example, which does not contain any two-word feature seen during training.

Original: This example highlights that balancing singleword features does not guarantee resiliency to spurious correlations, and therefore in order to mitigate all spurious correlations, balancing pairs of features is also required.
Revised: This example highlights that balancing singleword features does not guarantee resiliency to spurious correlations, and therefore in order to mitigate all spurious correlations, balancing pairs of features is also required.

Original: One can construct similar examples for larger values of n , by similarly considering multi-word expressions and common co-occurrences (e.g., “jaw dropping”, “worst day ever”).
Revised: One can construct similar examples for larger values of n , by similarly considering multi-word expressions and common co-occurrences (e.g., “jaw dropping”, “worst day ever”).

Original: These could serve as spurious correlations in the same way single words do.
Revised: These could serve as spurious correlations in the same way single words do.

Original: Another example is sarcasm.
Revised: Another example is sarcasm.

Original: A model that fails to understand sarcastic contexts will misinterpret  statements that appear in such contexts, even if it perfectly understands the base meaning of these statements.
Revised: A model that fails to understand sarcastic contexts will misinterpret [Table] statements that appear in such contexts, even if it perfectly understands the base meaning of these statements.

Original: Thus, the entire reasoning process of such a model, whether relying on simple features, feature interactions, or other types of understanding, will result in mispredictions of certain inputs, and thus can be considered spurious.
Revised: Thus, the entire reasoning process of such a model, whether relying on simple features, feature interactions, or other types of understanding, will result in mispredictions of certain inputs, and thus can be considered spurious.

Original: As a result, to truly mitigate all spurious correlations in a dataset, balancing feature combinations is required as well.
Revised: As a result, to truly mitigate all spurious correlations in a dataset, balancing feature combinations is required as well.

Original: Accordingly, balancing too little will leave some spurious correlations in the dataset.
Revised: Accordingly, balancing too little will leave some spurious correlations in the dataset.

Original: 3. Too much Balancing Prevents Learning 
Revised: 3. Too much Balancing Prevents Learning Valuable Semantic Knowledge

Original: Valuable Semantic Knowledge

Original: We observed that balancing too little does not allow models to fully eliminate spurious correlations.
Revised: We observed that balancing too little does not allow models to fully eliminate spurious correlations.

Original: Here we show that too much balancing can prevent models from learning valuable knowledge.
Revised: Here we show that too much balancing can prevent models from learning valuable knowledge.

Original: Consider the training data for learning the XOR function presented in Tab. 4 (left).
Revised: Consider the training data for learning the XOR function presented in Tab. 4 (left).

Original: This dataset contains enough learnable signal when considering feature interactions despite being balanced for single words.
Revised: This dataset contains enough learnable signal when considering feature interactions despite being balanced for single words.

Original: Nonetheless, balancing this dataset for pairs of features would result in no information, and thus prevent any model from learning this function (Tab. 4, right).
Revised: Nonetheless, balancing this dataset for pairs of features would result in no information, and thus prevent any model from learning this function (Tab. 4, right).

Original: Now consider a given natural language dataset
Revised: Now consider a given natural language dataset

Original: [Table]

Original: D .
Revised: D .

Original: Deﬁne n to be the length of the longest document in D .
Revised: Deﬁne n to be the length of the longest document in D .

Original: By deﬁnition, balancing every combination of up to n features(including) leaves nolearnable signal in D .
Revised: By deﬁnition, balancing every combination of up to n features leaves no learnable signal in D .

Original: We conclude that balancing too much can prevent models from learningsemantic knowledge.
Revised: We conclude that balancing too much can prevent models from learning semantic knowledge.

Original: Combining the two observations, we are left withthe question of the potential intersection betweenbalancing too much and balancing too little: does asweet spot exist for which no spurious correlationsare found in the dataset, but enough learnable signalis left?
Revised: Combining the two observations, we are left with the question of the potential intersection between balancing too much and balancing too little: does a sweet spot exist for which no spurious correlations are found in the dataset, but enough learnable signal is left?

Original: And even if so, would a balancing algorithm(whether by augmentation or ﬁltering) be able toﬁnd it?
Revised: And even if so, would a balancing algorithm, whether by augmentation or ﬁltering, be able to ﬁnd it?

Original: We leave these questions for future work,but note that addressing them is a prerequisite forthe theoretical and practical application of datasetbalancing for mitigating spurious correlations
Revised: We leave these questions for future work, but note that addressing them is a prerequisite for the theoretical and practical application of dataset balancing for mitigating spurious correlations.

Original: 3. Dataset Balancing is Undesired
Revised: 3. Dataset Balancing is Undesired

Original: Even if a sweet-spot exists between balancing toolittle and too much, do we really want to ﬁndit?
Revised: Even if a sweet spot exists between balancing too little and too much, do we really want to ﬁnd it?

Original: Here we argue that perhaps not.
Revised: Here we argue that perhaps not.

Original: The practiceof dataset balancing is designed to prevent models from learning that some words or expressionshave a common fallback meaning that can stemfrom dataset idiosyncrasies (e.g., “cat” as an indicator of contradiction) but also from cultural andhistorical contexts (e.g., Biden is the U.S. president).
Revised: The practice of dataset balancing is designed to prevent models from learning that some words or expressions have a common fallback meaning that can stem from dataset artifacts (e.g., “cat” as an indicator of contradiction) but also from cultural and historical contexts (e.g., Biden is the U.S. president in 2022).

Original: Fallback meanings are crucial for understanding language, as contexts are often underspeciﬁed (Graesser, 2013).
Revised: Fallback meanings are crucial for understanding language, as contexts are often underspeciﬁed (Graesser, 2013).

Original: Indeed, relying on fallbackmeanings might make models fail to process someinputs correctly, and might not generalize to otherdomains where the fallback meaning is different.
Revised: Indeed, relying on fallback meanings might make models fail to process some inputs correctly, and might not generalize to other domains where the fallback meaning is different.

Original: Here we argue that the ability to use them is a central ability of language understanding.
Revised: We argue that the ability to use them is a central ability of language understanding.

Original: For example, substantial efforts are made toteach models world knowledge , such as that thepresident of the U.S. is Joe Biden, the capital of
Revised: For example, substantial efforts are made to teach models world knowledge , such as that the president of the U.S. is Joe Biden, the capital of

Original: Brazil is Brasília, and France is the soccer world champion.
Revised: Brazil is Brasília, and France is the soccer world champion.

Original: These efforts include building world knowledge datasets (Wang et al., 2021), developing methods for enhancing models with this information (Zhang et al., 2019; Peters et al., 2019), and evaluating how well models capture it (Rubinstein et al., 2015; Roberts et al., 2020).
Revised: These efforts include building world knowledge datasets (Wang et al., 2021), developing methods for enhancing models with this information (Zhang et al., 2019; Peters et al., 2019), and evaluating how well models capture it (Rubinstein et al., 2015; Roberts et al., 2020).

Original: But many of these world-knowledge facts are context dependent: the capital of the Brazil has changed in 1960, the president of the U.S., as well as soccer world champions potentially change every 4 years, etc.
Revised: But many of these world-knowledge facts are context dependent: the capital of the Brazil has changed in 1960, the president of the U.S., as well as soccer world champions potentially change every 4 years, etc.

Original: Another example is common sense knowledge , such as “people are happy when they receive a gift”, “an elephant is taller than a zebra”, and “a statue that doesn’t ﬁt into a suitcase is too large”.
Revised: Another example is common sense knowledge , such as “people are happy when they receive a gift”, “an elephant is taller than a zebra”, and “a statue that doesn’t ﬁt into a suitcase is too large”.

Original: A large body of work has been carried out to create benchmarks that measure the common sense abilities of models (Liu and Singh, 2004; Levesque et al., 2012; Zellers et al., 2018; Sakaguchi et al., 2020; Bisk et al., 2020), as well as augmenting models with such abilities (Qin et al., 2020; Bosselut et al., 2021).
Revised: A large body of work has been carried out to create benchmarks that measure the common sense abilities of models (Liu and Singh, 2004; Levesque et al., 2012; Zellers et al., 2018; Sakaguchi et al., 2020; Bisk et al., 2020), as well as augmenting models with such abilities (Qin et al., 2020; Bosselut et al., 2021).

Original: Common sense reasoning is, by deﬁnition, stochastic and reliant on understanding presupposed, underspeciﬁed context.
Revised: Common sense reasoning is, by deﬁnition, stochastic and reliant on understanding presup- posed, underspeciﬁed context.

Original: One could imagine a person unhappy to receive a gift (e.g., because it is not what they wanted), a fantastically large zebra compared to a tiny elephant, and a suitcase with multiple compartments which prevent a small statue from ﬁtting in it.
Revised: One could imagine a person unhappy to receive a gift (e.g., because it is not what they wanted), a fantastically large zebra compared to a tiny elephant, and a suitcase with multiple compartments which prevent a small statue from ﬁtting in it.

Original: These examples illustrate that a model that learns these correlations and relies exclusively on them to make predictions is limited and is bound to make mistakes in some contexts.
Revised: These examples illustrate that a model that learns these correlations and relies exclusively on them to make predictions is limited and is bound to make mistakes in some contexts.

Original: One way to avoid these mistakes is to balance these correlations out, and prevent models from knowing these assertions to begin with.
Revised: One way to avoid these mistakes is to balance these correlations out, and prevent models from knowing these assertions to begin with.

Original: We argue that this solution is not a desired solution.
Revised: We argue that this solution is not a desired solution.

Original: In essence, an interpreter’s task (be it human or machine) is to infer the most probable context in which a statement is made, and as a result, it should have a fallback option for such world knowledge and common sense assertions.
Revised: In essence, an interpreter’s task (be it human or machine) is to infer the most probable context in which a statement is made, and as a result, it should have a fallback option for such world knowledge and common sense assertions.

Original: Discussion
Revised: Discussion

Original: We recognize that a balanced dataset may not be balanced with respect to the appearance of common-sense or world-knowledge assertions in a given context .
Revised: We recognize that a balanced dataset may not be balanced with respect to the appearance of common-sense or world-knowledge assertions in a given context .

Original: E.g., a model might balance-out the general fact that Joe Biden is the U.S. president, but not that he is the president in 2022.
Revised: E.g., a model might balance-out the general fact that Joe Biden is the U.S. president, but not that he is the president in 2022.

Original: As in many cases much of the context is unobserved (Graesser, 2013), the question is whether we want models to make a prediction in cases of uncertainty based on the fallback option.
Revised: As in many cases much of the context is unobserved (Graesser, 2013), the question is whether we want models to make a prediction in cases of uncertainty based on the fallback option.

Revised: We argue that doing so is a desired strategy in many cases (though a preferred strategy might be to interact of abstain from making a decisive prediction, see Sec. 4.2).

Revised: We also acknowledge that correlations in the real world can be misleading.

Revised: For instance, people often mistake the biggest commercial city in some countries for their capital (e.g., Istanbul in Turkey), potentially due to the high correlation between the two.

Original: We argue that doing so
Revised: In such cases, relying on the fallback option might lead to prediction error. However, we argue that following the human strategy of relying on a fallback option in cases of uncertainty will promote models’ communication abilities. We want to stress that balancing methods can result in mitigating some of the spurious correlations, and therefore lead to increased generalization (Le Bras et al., 2020; Swayamdipta et al., 2020).

Revised: Moreover, the process of ﬁltering the data naturally results in smaller datasets, which leads to lower training costs (Swayamdipta et al., 2020).

Revised: While such

Revised: Current Practice

Revised: Proposal

Original: [Table]
Revised: [Table]

Original: Richer contexts (§4.1)
Revised: Richer contexts (§4.1)

Original: A closed label set
Revised: A closed label set

Original: Abstain/interact (§4.2)
Revised: Abstain/interact (§4.2)

Revised: Large

Original: [Table]
Revised: [Table]

Original: is a desired strategy in many cases (though see

Original: Sections 4.2 and 4.4 for counter-examples).

Original: We want to stress that balancing methods can result in mitigating some of the spurious correlations,and therefore lead to increased generalization (Le

Original: Bras et al., 2020; Swayamdipta et al., 2020).

Original: Moreover, the process of ﬁltering the data naturally results in smaller datasets, which leads to lower training costs (Swayamdipta et al., 2020).

Original: While suchcontribution is meaningful in terms of environmental concerns (Strubell et al., 2019; Schwartz et al. ,2020), it is orthogonal to our research question.
Revised: learning (§4.3) contribution is meaningful in terms of, e.g., environmental concerns (Strubell et al., 2019; Schwartz et al., 2020), it is orthogonal to our research question.

Original: Overall, despite the important contributions of balancing techniques, this paper shows that even theperfect balancing method might not mitigate allspurious correlations in a satisfying way.
Revised: Overall, despite the important contributions of balancing techniques, this paper shows that even the perfect balancing method might not mitigate all spurious correlations in a satisfying way.

Original: So how can we make models more resilient tospurious correlations without balancing the data?
Revised: So how can we make models more resilient to spurious correlations without balancing the data?

Original: Below we discuss several ideas for doing this.
Revised: Below we discuss several ideas for doing this.

Original:  Ways to Move Forward
Revised:  Ways to Move Forward

Original: So far, we presented limitations of dataset balancing as a means to mitigate spurious correlations.
Revised: So far, we presented limitations of dataset balancing as a means to mitigate spurious correlations.

Original: Inthis section we discuss several alternatives to thispractice, summarized in Tab. 5.
Revised: In this section we discuss several alternatives to this practice, summarized in Tab. 5.

Original: We note that noneof these proposals is particularly novel.
Revised: We note that none of these proposals is particularly novel.

Original: Rather, weintend to survey alternatives proposed in literatureand argue that these may be promising for addressing the drawbacks of spurious correlations, and thatmore efforts should be put into studying them.
Revised: Rather, we intend to survey alternatives proposed in literature and argue that these may be promising for addressing the drawbacks of spurious correlations, and that more efforts should be put into studying them.

Original: 4. Augmenting Datasets with Rich Contexts
Revised: 4. Augmenting Datasets with Rich Contexts

Original: The implicit assumption of dataset balancing isthat in order to mitigate spurious correlations themodel has to unlearn them, that is, they should beremoved altogether from the training set.
Revised: The implicit assumption of dataset balancing is that in order to mitigate spurious correlations the model has to unlearn them, that is, they should be removed altogether from the training set.

Original: We arguethat instead we should be focusing on learning andmodeling richer contexts.
Revised: We argue that instead we should be focusing on learning and modeling richer contexts.

Original: As an example, consider negation.
Revised: As an example, consider negation.

Original: A modelthat generalizes well, should learn the meaning ofwords such as not , and should be able to negate newwords, even those that were seen only in positivecontexts at training time.
Revised: A model that generalizes well, should learn the meaning of words such as not , and should be able to negate new words, even those that were seen only in positive contexts at training time.

Original: For example, if a model only sees during training words like “amazing” or “happy” with positive sentiment, and thus learns that these words bear positive meaning, we would still expect it to interpret their negated appearance (e.g., not amazing ) as an indicator of negative sentiment.
Revised: For example, if a model only sees during training words like “amazing” or “happy” with positive sentiment, and thus learns that these words bear positive meaning, we would still expect it to interpret their negated appearance (e.g., not amazing ) as an indicator of negative sentiment.

Original: Such generalization is crucial for language learning, and should ideally allow models to not rely exclusively on spurious correlations.
Revised: Such generalization is crucial for language learning, and should ideally allow models to not rely exclusively on spurious correlations.

Original: Despite the immense progress in the ﬁeld in the past decade, negation still poses a challenge to modern NLP tools (Hossain et al., 2020).
Revised: Despite the immense progress in the ﬁeld in the past decade, negation still poses a challenge to modern NLP tools (Hossain et al., 2020, 2022).

Original: We suggest taking into account different types of contexts during dataset design.
Revised: We suggest taking into account different types of contexts during dataset design.

Original: In particular, collecting training examples with contexts such as negation (Morante and Blanco, 2012), humor (Weller and Seppi, 2019; Annamoradnejad and
Revised: In particular, collecting training examples with contexts such as negation (Morante and Blanco, 2012), humor (Weller and Seppi, 2019; Annamoradnejad and

Original: Zoghi, 2020), sarcasm (Davidov et al., 2010; Oprea and Magdy, 2020), or metaphors (Tsvetkov et al., 2014; Mohammad et al., 2016).
Revised: Zoghi, 2020), sarcasm (Davidov et al., 2010; Oprea and Magdy, 2020), or metaphors (Tsvetkov et al., 2014; Mohammad et al., 2016).

Original: This recommendation applies to both supervised tasks, and perhaps more so to pretrained data.
Revised: This recommendation applies to both supervised tasks, and perhaps more so to pretrained data.

Original: We suggest adding documents with such contexts throughout the pretraining corpus, or as a continued pretraining step to existing large-scale models.
Revised: We suggest adding documents with such contexts throughout the pretraining corpus, or as a continued pretraining step to existing large-scale models.

Original: To incorporate contexts from a wide range of phenomena, we can leverage the vast literature on broad-coverage semantics (Baker et al., 1998;
Revised: To incorporate contexts from a wide range of phenomena, we can leverage the vast literature on broad-coverage semantics (Baker et al., 1998;

Original: Steedman and Baldridge, 2007; Banarescu et al., 2013; Abend and Rappoport, 2013).
Revised: Steedman and Baldridge, 2006; Banarescu et al., 2013; Abend and Rappoport, 2013).

Original: This line of work proposes theories of language, composing inventories of linguistic constructions with an algebraic formulation of their inter-relations in terms of truth value, factuality, and more.
Revised: This line of work proposes theories of language, composing inventories of linguistic constructions with an algebraic formulation of their inter-relations in terms of truth value, factuality, and more.

Original: These inventories often include the phenomena discussed above, such as negation, sarcasm, and presupposition.
Revised: These inventories often include the phenomena discussed above, such as negation, sarcasm, and presupposition.

Original: 4. Interaction and Abstention to Cope with 
Revised: 4. Interaction and Abstention to Cope with Underspeciﬁed Contexts

Original: Underspeciﬁed Contexts

Original: Most NLP tasks are designed with a closed label set that forces models to make a concrete prediction for each test instance, without an option to abstain or interact with the user to get more information.
Revised: Most NLP tasks are designed with a closed label set that forces models to make a concrete prediction for each test instance, without an option to abstain or interact with the user to get more information.

Original: Even for tasks with a large label set (e.g., language modeling), models still have to output a valid vocabulary item.
Revised: Even for tasks with a large label set (e.g., language modeling), models still have to output a valid vocabulary item.

Original: Here we argue that this practice creates an inductive bias towards using spuriouscorrelations in cases of uncertainty, as the modelhas “nothing to lose” in case of low uncertainty,and is encouraged to always make some prediction,potentially relying on spurious correlations.
Revised: Here we argue that this practice creates an inductive bias towards using spurious correlations in cases of uncertainty, as the model has “nothing to lose” in case of low certainty, and is encouraged to always make some prediction, potentially relying on spurious correlations.

Original: To further illustrate this point, consider the ambiguous sentence “ To my great surprise, the movieturned out different than what I thought .”
Revised: To further illustrate this point, consider the ambiguous sentence “ To my great surprise, the movie turned out different than what I thought .”

Original: , in thecontext of sentiment analysis.
Revised: , in the context of sentiment analysis.

Original: The reader cannot infer whether the writer is pleasantly surprised (a positive review) or disappointed (a negative review).
Revised: The reader cannot infer whether the writer is pleasantly surprised (a positive review) or disappointed (a negative review).

Original: We argue that in such cases models might leantowards a positive sentiment based on the words“great” and “surprise”, which are typically correlated with a positive sentiment.
Revised: We argue that in such cases models might lean towards a positive sentiment based on the words “great” and “surprise”, which are typically correlated with a positive sentiment.

Original: To test this, we ran a RoBERTa-large model (Liuet al., 2019) ﬁne-tuned on SST-2{+SST-
+} (Socher et al. on that example.
Revised: To test this, we ran a RoBERTa-large model (Liu et al., 2019) ﬁne-tuned on {+SST-
+} (Socher et al., 2013) on that example.

Original: 8 As expected, the modelreturns a positive label, with 99.99% conﬁdence.
Revised: As expected, the model returns a positive label, with 99.99% conﬁdence.

Original: Interestingly, three different interpretation methods (simple gradient visualization, Simonyan et al. ,2014; integrated gradient visualization, Sundararajan et al., 2017; and SmoothGrad, Smilkov et al.
Revised: Interestingly, three different interpretation methods (simple gradient visualization, Simonyan et al., 2014; integrated gradient visualization, Sundararajan et al., 2017; and SmoothGrad, Smilkov et al., 2017)

Original: all ﬁnd the word “great”  to be one of the twomost inﬂuential words on the model’s prediction.
Revised: all ﬁnd the word “great” or “surprise” to be one of the three most inﬂuential words on the model’s prediction.

Original: While this example does not prove the prevalenceof this problem, it does demonstrate its existence.
Revised: While this example does not prove the prevalence of this problem, it does demonstrate its existence.

Original: To address this problem, we suggest adopting ap- proaches that allow models to abstain and interact when they cannot make a decision with high conﬁdence (Chow, 1957; Hellman, 1970; Laidlaw and
Revised: To address this problem, we suggest adopting approaches that allow models to abstain and interact when they cannot make a decision with high conﬁdence (Chow, 1957; Hellman, 1970; Laidlaw and

Original: Feizi, 2019; Balcan et al., 2020).
Revised: Feizi, 2019; Balcan et al., 2020).

Original: See Fig.
Revised: See Fig.

Original: This can be achieved by building datasets with unanswerable questions (Ray et al., 2016;
Revised: This can be achieved by building datasets with unanswerable questions (Ray et al., 2016;

Original: Rajpurkar et al., 2018; Sulem et al., 2021), but also by designing models that abstain in cases of low certainty for all inputs, even those with an unambiguous gold label.
Revised: Rajpurkar et al., 2018; Sulem et al., 2021), but also by designing models that abstain in cases of low certainty for all inputs, even those with an unambiguous gold label.

Original: 9 We hypothesize that encouraging the model to provide this output when it is unsure, rather than making a semi-educated guess, potentially based on spurious correlations, could reduce its dependency on such correlations.
Revised: 12 We hypothesize that encouraging the model to provide this output when it is unsure, rather than making a semi-educated guess, potentially based on spurious correlations, could reduce its dependency on such correlations.

Original: 4. The End of Large-Scale Fine-Tuning?
Revised: 4. The End of Large-Scale Fine-Tuning?

Original: This paper has demonstrated the limitations of mitigating spurious correlations via dataset balancing.
Revised: This paper has demonstrated the limitations of mitigating spurious correlations via dataset balancing.

Original: A naive way to eliminate spurious correlations is to stop using large-scale datasets altogether.
Revised: A naive way to mitigate spurious correlations is to stop using large-scale datasets altogether.

Original: We  argue that for supervised learning (i.e., large-scale ﬁne-tuning), recent advances in zero- and few-shot learning might make this option possible.
Revised: We echo recent calls (Liu et al., 2021b) and argue that for supervised learning (i.e., large-scale ﬁne-tuning), recent advances in zero- and few-shot learning might make this option possible.

Original: Large pretrained models such as T5 (Raffel et al., 2020) or GPT-3 (Brown et al., 2020), trained on vast amounts of data, arguably learn enough about the world to acquire many of the skills currently learned through supervised datasets.
Revised: Large pretrained models such as T5 (Raffel et al., 2020) or GPT-3 (Brown et al., 2020), trained on vast amounts of data, arguably learn enough about the world to acquire many of the skills currently learned through supervised learning.

Original: Indeed, the large increase in the size and capacity of pretrained language models has led to a new wave of few-shot and zero-shot methods (Schick and Schütze, 2021;
Revised: Indeed, the large increase in the size and capacity of pretrained language models has led to a new wave of few-shot and zero-shot methods (Schick and Schütze, 2021;

Original: Shin et al., 2020; Gu et al., 2021), which are able to reach human-level performance on certain tasks using only a few dozens of training examples (Schick and Schütze, 2021).
Revised: Shin et al., 2020; Gu et al., 2022), which are able to reach human-level performance on certain tasks using only a few dozens of training examples (Schick and Schütze, 2021).

Original: Given these impressive results, it is not clear whether there is still value in ﬁnetuning models on large-scale datasets for all tasks.
Revised: Given these impressive results, it is not clear whether there is still value in ﬁnetuning models on large-scale datasets for all tasks.

Original: In the context of this work, focusing on few-shot learning might allow models to not learn some of the correlations that result from manual annotation (Schwartz et al., 2017; Gururangan et al., 2018;
Revised: In the context of this work, focusing on few-shot learning might allow models to not learn some of the correlations that result from manual annotation (Schwartz et al., 2017; Gururangan et al., 2018;

Original: Poliak et al., 2018), as they will not be exposed to many of them to begin with.
Revised: Poliak et al., 2018), as they will not be exposed to many of them to begin with.

Original: We note that this proposal is not a perfect solution.
Revised: We note that this proposal is not a perfect solution.

Original: First, some spurious correlations may be picked up by the small number of examples.
Revised: First, some spurious correlations may be picked up by the small number of examples.

Original: This is less of a problem in the zero-shot setting, or in cases where the model parameters are not updatedin few-shot settings (Brown et al., 2020), but studying the extent to which spurious correlations arepicked up in other few-shot settings is an importantavenue for future research.
Revised: This is less of a problem in the zero-shot setting, or in cases where the model parameters are not updated in few-shot settings (Brown et al., 2020), but studying the extent to which spurious correlations are picked up in other few-shot settings is an important avenue for future research.

Original: Second, some spuriouscorrelations might be picked up during the pretraining stage (Gehman et al., 2020; Birhane et al.
Revised: Second, some spurious correlations might be picked up during the pretraining stage (Gehman et al., 2020; Birhane et al.,

Original: ,2021; Dodge et al., 2021).
Revised: ; Dodge et al., 2021).

Original: Continuing to quantifythis phenomenon and ﬁnding ways to mitigate it isanother important line of research.
Revised: Continuing to quantify this phenomenon and ﬁnding ways to mitigate it is another important line of research.

Original: An important question in this context is the tasksfor which supervised learning is still needed.
Revised: An important question in this context is the tasks for which supervised learning is still needed.

Original: Itseems possible the excelling in language modeling tasks requires mastering the skills that standin the base of many NLP tasks, such as sentimentanalysis, syntactic parsing, and NER.
Revised: It seems plausible that excelling in language modeling tasks requires mastering the skills that stand in the base of many NLP tasks, such as sentiment analysis, syntactic parsing, and NER.

Original: However, itis similarly possible that this is not the case forother tasks, e.g., summarization, simpliﬁcation anddialogue.
Revised: However, it is similarly plausible that this is not the case for other tasks, e.g., summarization, simpliﬁcation and dialogue.

Original: We are cautious to make concrete recommendations for which tasks to apply this principle,but suggest the following intuitive rule of thumb:for datasets or tasks for which the state of the artis close to or surpasses the human baseline, weshould consider moving to few-shot setups.
Revised: We are cautious in making concrete recommendations for which tasks to apply this principle, but suggest the following intuitive rule of thumb: for datasets or tasks for which the state of the art is close to or surpasses the human baseline, we should consider moving to few-shot setups.

Original: Finally, dataset creation is still a valuable andimportant line of research.
Revised: Finally, dataset creation is still a valuable and important line of research.

Original: Our recommendation tostop building large scale training sets does not makethis task redundant, to both spur the design of bettermodels, and to better test their capabilities.
Revised: Our recommendation to stop building large scale training sets does not make this task redundant, to both spur the design of better models, and to better test their capabilities.

Original: Wesuggest instead of building large training sets andsmall validation and tests, authors should considerbuilding large test sets, as a means for achievingimproved statistical power (Card et al., 2020).
Revised: We suggest that instead of building large training sets and small validation and test sets, authors should consider building large test sets, as a means for achieving improved statistical power (Card et al., 2020).

Original:  A Note on Social-Bias Correlations
Revised: 4. A Note on Social-Bias Correlations

Original: So far, we discussed the problems with unlearningspurious correlations, and advocated instead formore elaborate context modeling.
Revised: So far, we discussed the problems with unlearning spurious correlations, and advocated instead for more elaborate context modeling.

Original: One exceptionmight be the case of social biases.
Revised: One exception might be the case of social biases.

Original: Textual dataoften reﬂects human stereotypes such as spuriouscorrelations between labels and protected group attributes, e.g., alignments between professions andgender or race.
Revised: Textual data often reﬂects human stereotypes such as spurious correlations between labels and protected group attributes, e.g., alignments between professions and gender or race.

Original: Unlike other types of knowledgediscussed in Sec. 3.4, in this case there is an incentive to prevent models from learning this typeof correlation as means for actively reducing theharms of such biases, especially in commercial andpublic-facing applications, such as machine translation (Stanovsky et al., 2019) or automated ﬁnancialdecision-making (Bartlett et al., 2021).
Revised: Unlike other types of knowledge discussed in Sec. 3.4, in this case there is an incentive to prevent models from learning this type of correlation as means for actively reducing the harms of such biases, especially in commercial and public-facing applications, such as machine translation (Stanovsky et al., 2019) or automated ﬁnancial decision-making (Bartlett et al., 2021).

Original: As a result,methods for dataset balancing are no longer undesired for mitigating such spurious correlations.
Revised: As a result, methods for dataset balancing are no longer undesired for mitigating such spurious correlations.

Original: Nonetheless, as demonstrated in Sec. 3, methods for dataset balancing are a limited solution for mitigating spurious correlations, including social-bias ones.
Revised: Nonetheless, as demonstrated in Sec. 3, methods for dataset balancing are a limited solution for mitigating spurious correlations, including social-bias ones.

Original: In contrast, the methods proposed in this section for mitigating spurious correlations might also assist in mitigating social biases, or at least slow down their ampliﬁcation (Zhao et al., 2017).
Revised: In contrast, the methods proposed in this section for mitigating spurious correlations might also assist in mitigating social biases, or at least slow down their ampliﬁcation (Zhao et al., 2017).

Original:  Related Work
Revised:  Related Work

Original: This paper discussed the arms-race between models and datasets.
Revised: This paper discusses the arms-race between models and datasets.

Original: Previous works criticized one side of this arms race—the increasing size of pretrained models—due to ethical and environmental concerns (Schwartz et al., 2020; Bender et al., 2021), or questioning its ability to learn meaningful abstractions from raw text (Bender and Koller, 2020; Merrill et al., 2021).
Revised: Previous works criticized one side of this arms race—the increasing size of pretrained models—due to ethical and environmental concerns (Schwartz et al., 2020; Bender et al., 2021), or questioning its ability to learn meaningful abstractions from raw text (Bender and Koller, 2020; Merrill et al., 2021).

Original: This work studies the second part of this arms race, regarding the efforts to mitigate spurious correlations through dataset balancing.
Revised: This work studies the second part of this arms race, regarding the efforts to mitigate spurious correlations through dataset balancing.

Original: The release of such datasets is often motivated by their potential to spur progress in modeling, and to help tease apart qualitative differences between models.
Revised: The release of such datasets is often motivated by their potential to spur progress in modeling, and to help tease apart qualitative differences between models.

Original: Liu et al.
Revised: Liu et al.

Original: (2021) showed that this is not necessarily the case, by observing that the ranking of reading comprehension models on small and synthetic benchmarks is similar to that of the  SQuAD dataset (Rajpurkar et al., 2016).
Revised: (2021a) showed that this is not necessarily the case, by observing that the ranking of reading comprehension models on small and synthetic benchmarks is similar to that of the (large) SQuAD dataset (Rajpurkar et al., 2016).

Original: Finally, Raji et al.
Revised: Raji et al.

Original: (2021) recently criticized the concept of benchmarks as a whole, arguing that they are only capturing speciﬁc skills and not “general” capabilities.
Revised: (2021) recently criticized the concept of benchmarks as a whole, arguing that they are only capturing speciﬁc skills and not “general” capabilities.

Original: Our paper raises relates concerns about training sets implicitly containing spurious correlations, and suggests reconsidering the practice of building large-scale training sets.
Revised: Our paper raises related concerns about training sets implicitly containing spurious correlations, and suggests reconsidering the practice of building large-scale training sets.

Revised: Finally, concurrent to this work, Eisenstein (2022) discussed several types of spurious correlations in the context of causality theory (Pearl, 2009), and used a toy example to demonstrate their different effects on models.

Revised: They concluded that domain knowledge is required to identify the correlations that are indeed spurious, i.e., those that might challenge the generalization ability of models.

Original:  Conclusion
Revised:  Conclusion

Original: Spurious correlations in large textual corpora can result in model brittleness, lack of generalization, and an inﬂated sense of the state of the art.
Revised: Spurious correlations in large textual corpora can result in model brittleness, lack of generalization, and an inﬂated sense of the state of the art.

Original: Mitigating their negative side-effects is an important research goal of the NLP community.
Revised: Mitigating their negative side-effects is an important research goal of the NLP community.

Original: In this paper we presented practical and conceptual limitations of dataset balancing as a means for doing so.
Revised: In this paper we presented practical and conceptual limitations of dataset balancing as a means for doing so.

Original: We proposed alternative ways for mitigating spurious correlations, including adding richer contexts to textual corpora, and allowing models to abstain or interact in cases of uncertainty.
Revised: We proposed alternative ways for mitigating spurious correlations, including adding richer contexts to textual corpora, and allowing models to abstain or interact in cases of uncertainty.

Original: We concluded by suggesting to reconsider the practice of ﬁne-tuning pretrained models on large-scale training sets.
Revised: We concluded by suggesting to reconsider the practice of ﬁne-tuning pretrained models on large-scale training sets.

Original:  Broader Impact and EthicalConsideration
Revised:  Broader Impact and Ethical Consideration

Original: Our work did not involve any new data or annotation collection, and as such did not require crowdsourced or in-house workers, or introduces anynew models and related risks.
Revised: Our work did not involve any new data or annotation collection, and as such did not require crowdsourced or in-house workers, or introduces any new models and related risks.

Original: Instead, we examineexisting resources and common data balancing approaches.
Revised: Instead, we examine existing resources and common data balancing approaches.

Original: In Section 4.4 we speciﬁcally discuss therelation between these practices and implicationson social bias in models.
Revised: In Section 4.4 we speciﬁcally discuss the relation between these practices and implications on social bias in models.

Revised: Acknowledgements

Revised: We would like to thank Matt Gardner and Will Merrill for the in-depth discussion.

Revised: We would also like to thank Omri Abend, Yoav Goldberg, Inbal Magar, and the anonymous reviewers for their feedback.

Revised: This work was supported in part by research gifts from the Allen Institute for AI.