Finding Dataset Shortcuts with Grammar Induction (2022)

Abstract: Many NLP datasets have been found to contain shortcuts: simple decision rules that achieve surprisingly high accuracy. However, it is difficult to discover shortcuts automatically. Prior work on automatic shortcut detection has focused on enumerating features like unigrams or bigrams, which can find only low-level shortcuts, or relied on post-hoc model interpretability methods like saliency maps, which reveal qualitative patterns without a clear statistical interpretation. In this work, we propose to use probabilistic grammars to characterize and discover shortcuts in NLP datasets. Specifically, we use a context-free grammar to model patterns in sentence classification datasets and use a synchronous context-free grammar to model datasets involving sentence pairs. The resulting grammars reveal interesting shortcut features in a number of datasets, including both simple and high-level features, and automatically identify groups of test examples on which conventional classifiers fail. Finally, we show that the features we discover can be used to generate diagnostic contrast examples and incorporated into standard robust optimization methods to improve worst-group accuracy.

## introduction many nlp datasets have been found to contain shortcuts: simple decision rules that achieve surprisingly high accuracy. for example, it is possible to get high classification accuracy on paraphrase identification datasets by predicting that sentences with many common words are paraphrases of each other @xcite . such classifiers are said to be "right for the wrong reason" @xcite . shortcuts are a problem if they do not generalize to the intended test distribution @xcite . for example, paraphrase identification models might misclassify non-paraphrases that have many overlapping words. shortcuts have been reported in many established datasets (e.g., @xcite @xcite , as a consequence of annotation artifacts or so-called spurious correlations. typically, these discoveries are the result of human intuition about possible patterns in a particular dataset. our goal in this paper is to discover shortcuts automatically. if we can identify shortcuts, we can identify categories of examples on which conventional classifiers will fail, and try to mitigate these weaknesses by collecting more training data or using robust optimization algorithms. the main challenge to automatically identifying shortcuts is to develop a formal framework for describing patterns in language data that can capture both simple and high-level features and that makes it possible to search for these patterns efficiently. prior work has addressed only simple features like unigrams and bigrams @xcite @xcite , and it is difficult to extend this approach to more sophisticated patterns, like lexical overlap, without knowing the pattern in advance. other model-based approaches use black-box interpretability methods, by using gradient-based techniques to identify tokens or training instances that influence a particular prediction @xcite @xcite . these methods offer local, qualitative hints about the decision of a classifier on a particular test instance, but do not identify dataset-level features nor provide a way of measuring the strength of correlation. our approach is to use grammar induction to characterize and discover shortcut features ( §2). probabilistic grammars provide a principled framework for describing patterns in natural language, allowing us to formally model both simple fea- figure 1 : left: the most likely parse tree for an example from the snli validation set according to our synchronous grammar. the numbered nodes index non-terminal symbols and denotes an empty symbol. we highlight the subtrees that provide the strongest evidence in favor of the label contradiction, and show alternative spans generated by these non-terminals after conditioning on the class labels (right). in this work, we focus on both single-sentence (e.g., sentiment analysis) and sentence-pair classification datasets (e.g., nli, paraphrase detection). while we can use context-free grammars to model features in sentences, sentence-pair datasets present a particular challenge, as it is difficult to enumerate interactions between the pair of sentences. we propose to use synchronous context-free grammars, which formally model insertion, deletion, and alignment. we find that we can extract meaningful, dataset-specific structures that describe latent patterns characterizing these classification tasks. we apply our approach to four classification datasets: imdb, subj, snli and qqp. after illustrating the shortcut features ( §3), we explore whether state-of-the-art classifiers exploit these shortcuts by identifying minority examples in the datasets and then generating contrastive examples ( §4). then we demonstrate that these features can be used in robust optimization algorithms to improve generalization ( §5). finally, we compare this approach with model-based interpretability methods and n-gram features, which do not explicitly model syntax ( §6). overall, we find that grammar induction provides a flexible and expressive representation for modeling features in nlp datasets. ## remedying shortcuts once we have found shortcuts, to what extent can we can mitigate them using standard robust optimization algorithms? we conduct a small-scale experiment, focusing the contrasting examples we created for snli. we compare two methods: just train twice (jtt; @xcite and drift @xcite . note that jtt does not assume known shortcut features while drift does. specifically, jtt upsamples the subset of training examples that are misclassified by a weak model (we use bertbase trained for one epoch). drift takes as input a biased model and trains a new model with a regularization term that effectively upweighs the reward for instances that the biased model misclassifies (we set the biased model to be a logistic regression classifier trained on feature vectors indicating the set of scfg production rules that appear in each tree). full details are provided in appendix b. results ## related work finding shortcuts prior work has identified shortcuts in nlp datasets by developing diagnostic evaluation datasets @xcite @xcite , training partial input baselines @xcite , or calculating statistics of simple features, like n-grams, that can be enumerated explic- figure 2 : we compare the scfg features from snli with equivalent n-gram feature that discard the alignment information provided by the grammar (section 6). as we consider simpler features, the features appear in more examples but become less discriminative, and tend to have a weaker correlation with bert accuracy. defining spurious correlations a related line of work has addressed the question of which correlations should be considered spurious. @xcite suggest that all correlations between labels and low-level features, such as unigrams, are spurious. @xcite argues that such correlations will arise naturally in most language classification settings, and domain expertise is needed to determine which correlations might be harmful. our aim is to provide a summary of the shortcuts in a dataset, so that a practitioner can determine which are undesirable and remedy them if needed. feature importance our approach is related to methods for identifying important input features, like lime @xcite . lundberg and lee (2017) present a framework for measuring local feature importance based on shapley values @xcite , which can be applied to both model-specific and model-agnostic notions of importance, and covert et al. (2020) extend this approach to global importance scores. these methods require first identifying the set of features, while our focus is on inducing a richer set of features. we use a simple metric for feature importance, and leave more sophisticated metrics to future work. ## conclusion we have developed an approach for automatically finding dataset shortcuts by inducing datasetspecific grammars. we demonstrated that it reveals interesting shortcut features in four classification datasets and can be used as a diagnostic tool to identify categories of examples on which classifiers are more likely to fail. future work will explore approximate inference methods to scale this approach to datasets with longer sequences, and extensions to more expressive grammar formalisms.