  Much recent work in NLP has documented dataset artifacts, bias, and spurious
correlations between input features and output labels. However, how to tell
which features have "spurious" instead of legitimate correlations is typically
left unspecified. In this work we argue that for complex language understanding
tasks, all simple feature correlations are spurious, and we formalize this
notion into a class of problems which we call competency problems. For example,
the word "amazing" on its own should not give information about a sentiment
label independent of the context in which it appears, which could include
negation, metaphor, sarcasm, etc. We theoretically analyze the difficulty of
creating data for competency problems when human bias is taken into account,
showing that realistic datasets will increasingly deviate from competency
problems as dataset size increases. This analysis gives us a simple statistical
test for dataset artifacts, which we use to show more subtle biases than were
described in prior work, including demonstrating that models are
inappropriately affected by these less extreme biases. Our theoretical
treatment of this problem also allows us to analyze proposed solutions, such as
making local edits to dataset instances, and to give recommendations for future
data collection and model design efforts that target competency problems.
