Learning Which Features Matter: R o BERT a Acquires a Preference for Linguistic Generalizations (Eventually) (2020)

Abstract: One reason pretraining on self-supervised linguistic tasks is effective is that it teaches models features that are helpful for language understanding. However, we want pretrained models to learn not only to represent linguistic features, but also to use those features preferentially during fine-turning. With this goal in mind, we introduce a new English-language diagnostic set called MSGS (the Mixed Signals Generalization Set), which consists of 20 ambiguous binary classification tasks that we use to test whether a pretrained model prefers linguistic or surface generalizations during finetuning. We pretrain RoBERTa from scratch on quantities of data ranging from 1M to 1B words and compare their performance on MSGS to the publicly available RoBERTa_BASE. We find that models can learn to represent linguistic features with little pretraining data, but require far more data to learn to prefer linguistic generalizations over surface ones. Eventually, with about 30B words of pretraining data, RoBERTa_BASE does consistently demonstrate a linguistic bias with some regularity. We conclude that while self-supervised pretraining is an effective way to learn helpful inductive biases, there is likely room to improve the rate at which models learn which features matter.

## introduction self-supervised pretraining through language modeling on massive datasets has revolutionized nlp. one reason this method works is that pretraining shapes a model's hypothesis space, giving it inductive biases that help it learn linguistic tasks @xcite . numerous probing studies have provided support for this idea by showing that language models learn representations that encode linguistic features @xcite @xcite . however, feature learning is just the first step to acquiring helpful inductive biases. models must the rumor that a ceo is losing spread. a boy who is hugging the cat sneezed. a guest said that the boat is sinking. ## evaluation data we introduce msgs (mixed signals generalization set), pronounced "messages", a dataset we design to be used in poverty of the stimulus and inoculation experiments. with the goal of contrasting inductive biases that are helpful and harmful in most nlp applications, the tasks in msgs test a model's preferences for generalizations based on linguistic or surface features. features under study ## models, pretraining, & fine-tuning we test 13 roberta models in our main experiments: we pretrain 12 from scratch, and also test roberta base pretrained by @xcite . ## results & discussion we have several main findings: (1) models learn to represent both surface features and linguistic features with relatively little data; (2) roberta begins to acquire a linguistic bias with over 1b words of pretraining data; (3) increasing pretraining data strengthens linguistic bias; (4) there is considerable variation in models' preferences between specific pairs of linguistic and surface features. ## related work there is increasing interest in studying the inductive biases of neural networks. much of this work has grown out of numerous findings that these models often fail to generalize in ways that task designers intend. for example, @xcite and @xcite demonstrate that ambiguity in widely used nlu datasets like squad @xcite and multinli @xcite leads models like bert to adopt some surface generalizations, despite the fact that they represent linguistic features. this continues to be a problem for models like roberta base which show an overall linguistic bias in our experiments. however, for tasks like nli, the underlying linguistic feature depends on a combination of significant syntactic knowledge, semantic knowledge, and world knowledge. it stands to reason that representations and preferences for such high level features require more data to learn than the features we probe. other work has used the poverty of stimulus design to study inductive biases associated with particular neural architectures during syntactic gen-eralization. @xcite train rnns on a morphological prediction task using artificial languages derived from naturally occurring english text, finding that rnns show a recency bias in acquiring agreement rules. @xcite train a seq2seq models on generated data ambiguous between a surface and a structural generalization to learn the subject-auxiliary inversion rule in english question formation. they find that, while tree-structured models show a structural bias, sequence models do not. @xcite conduct related experiments on subject-auxiliary inversion and other english structural rules, and find that bert likely acquires a structural bias from pretraining. more abstract inductive biases have also been studied. using zero-shot learning in an artificial language, @xcite show that rnns lack a bias in favor of learning compositional meanings for new symbols. gandhi and @xcite and @xcite explore conditions under which neural networks exhibit a bias towards learning mutually exclusive meanings for new symbols. data augmentation and inoculation have also been explored previously as a way to influence how models generalize. @xcite and @xcite show that small amounts of inoculating data during training on textual entailment help bert overlook certain surface generalizations. @xcite study inoculation using a constructed language of numerical sequences. like us, they generate ambiguous datasets, though they only compare features that resemble our surface features. they find that it is relatively easy to nudge models away from shallow generalizations, but harder to nudge them towards deeper ones. finally, several earlier studies explored how increasing training data impacts linguistic knowledge in lms. unlike the present study, these studies evaluate lms using an unsupervised acceptability judgment task on minimal pairs (i.e. not during fine-tuning), and do not attempt to separate feature learning from feature preferences. @xcite find the greatest increase in sensitivity to acceptability contrasts occurs between training on 2m and 10m words. @xcite find that while lms learn agreement phenomena at a similarly early stage, other phenomena require more data to learn. finally, @xcite find that adopting architectures that build in linguistic bias, such as rnngs @xcite , has a big-ger effect on the acceptability task than increasing training data from 1m to 40m words. ## future work & conclusion our experiments shed light on the relationship between pretraining data and an inductive bias towards linguistic generalization. our results indicate that, although some abstract linguistic features are learnable from relatively small amounts of pretraining data, models require significant pretraining after discovering these features to develop a bias towards using them when generalizing. this gives some insight into why extensive pretraining helps general-purpose neural networks adapt to downstream tasks with relative ease. we also introduce msgs, a new diagnostic dataset for probing the inductive biases of learning algorithms using the poverty of the stimulus design and inoculation, and also introduce a set of 12 roberta models we pretrain on smaller data quantities. these models could prove to be a helpful resource for future studies looking to study learning curves of various kinds with respect to the quantity of pretraining data. finally, while our results naturally lead to the conclusion that we should continue to pursue models with ever more pretraining, such as gpt-3 (brown et al., 2020), we do not wish to suggest that this will be the only or best way to build models with stronger inductive biases. future work might use msgs as a diagnostic tool to measure how effectively new model architectures and selfsupervised pretraining tasks can more efficiently equip neural networks with better inductive biases.