How Can We Accelerate Progress Towards Human-like Linguistic Generalization? (2020)

Abstract: This position paper describes and critiques the Pretraining-Agnostic Identically Distributed (PAID) evaluation paradigm, which has become a central tool for measuring progress in natural language understanding. This paradigm consists of three stages: (1) pre-training of a word prediction model on a corpus of arbitrary size; (2) fine-tuning (transfer learning) on a training set representing a classification task; (3) evaluation on a test set drawn from the same distribution as that training set. This paradigm favors simple, low-bias architectures, which, first, can be scaled to process vast amounts of data, and second, can capture the fine-grained statistical properties of a particular data set, regardless of whether those properties are likely to generalize to examples of the task outside the data set. This contrasts with humans, who learn language from several orders of magnitude less data than the systems favored by this evaluation paradigm, and generalize to new tasks in a consistent way. We advocate for supplementing or replacing PAID with paradigms that reward architectures that generalize as quickly and robustly as humans.

## introduction the special session of the 2020 annual meeting of association for computational linguistics invites us to take stock of the progress made in the field in the last few years. there is no question that we have made significant progress in a range of applications: current machine translation systems for high-resource languages, for example, are undeniably better than those we had a decade ago. this opinion piece will focus on a different question: are we making progress towards the classic goal of mimicking human linguistic abilities in machines-towards a model that acquires language as efficiently as humans, and generalizes it as humans do to new structures and contexts ("tasks")? i will argue that an evaluation paradigm that has rapidly established itself as one of the main tools for measuring progress in the field-a paradigm i will term, for want of a catchier name, pretraining-agnostic identically distributed evaluation (paid)-encourages progress in a direction that is at best orthogonal to the goal of human-like generalization. because it does not consider sample efficiency, this approach rewards models that can be trained on massive amounts of data, several orders of magnitude more than a human can expect to be exposed to. and because benchmark scores are computed on test sets drawn from the same distribution as their respective training sets, this paradigm favors models that excel in capturing the statistical patterns of particular data sets over models that generalize as a human would. ## human-like generalization humans learn language from much more limited exposure than most contemporary nlp systems. an analysis of recordings taken in the environment of the child of an mit professor between the ages of 9 and 24 months found that the child heard or produced approximately eight million words over this 15-month period @xcite . children in lower socioeconomic status families in western societies receive significantly less linguistic input than that (around 3 million words per year, @xcite ; even more strikingly, members of the tsimane community in bolivia spend about 15 times less time per hour speaking to their children than do highly educated american families @xcite . if nlp systems were as sample-efficient as tsimane children, far fewer languages would be considered "low-resource languages". despite the limited amount of exposure to their language, humans generalize their linguistic knowl-edge in a consistent way to structures that are infrequent or non-existent in corpora @xcite , and quickly learn to do new things with language (what we sometimes refer to in nlp as "tasks"). as i discuss below, this is not the case for current deep learning systems: when tested on cases sampled from a distribution that differs from the one they were trained on, their behavior is unpredictable and inconsistent with that of humans @xcite , and they require extensive instruction on each new task @xcite . humans' rapid and consistent generalization abilities rely on powerful inductive biases, which likely arise from a combination of innate building blocks and experience with diverse learning problems @xcite . systems that generalize like humans would be useful not only for nlp, but also for the scientific study of human language acquisition and processing @xcite . but, as i will argue in the next two sections, it is unclear whether our dominant evaluation paradigms are getting us closer to this goal. ## pretraining-agnostic evaluation over the last two years, deep learning systems have obtained rapidly increasing scores on language understanding benchmarks such as glue @xcite or superglue @xcite . these benchmarks aggregate multiple supervised classification tasks-such as sentiment analysis, linguistic acceptability judgments, or entailment detection-and collate the scores obtained on those tasks into a leaderboard, with a single headline score for each model averaging its scores on each individual task. for each of these classification tasks, a data set that was generated by a particular process, often involving crowdsourcing, is randomly split into two: a training set, which the system is allowed to observe, and a held-out test set, on which it is evaluated. a standard recipe has emerged for achieving high scores on such benchmarks. a neural network-typically, one based on the transformer architecture @xcite -is pretrained on a denoising objective, such as filling in one or more blanks in a vast number of sentences. this network is then fine-tuned (performs transfer learning) on the benchmark's supervised tasks, each of which include a much smaller number of training examples than the pretraining corpus @xcite . the t5 model @xcite -the system that boasted the highest score on superglue at the time of writing-achieved an average accuracy of 88.9% on this benchmark, slightly lower than that of untrained human annotators (89.8%), and more than 20 percentage points higher than the score obtained just a few months earlier by bert @xcite . this jump in accuracy does not reflect significant modeling innovations: both bert and t5 are transformers trained on similar objectives that differ primarily in their scale. when ranking systems, leaderboards such as su-perglue do not take into account the amount of pretraining data provided to each model. pretraining corpora are not standardized, and the amount of pretraining data is not always easy to discern from the papers reporting on such systems. here is my attempt to reconstruct the recent evolution of pretraining corpus sizes. foot_0 bert, uploaded to arxiv in october 2018, was trained on 3.3 billion words; xlnet @xcite , was trained on 78 gb of text, or approximately 13 billion words; roberta @xcite , july 2019) was trained on 160 gb of text, or around 28 billion words; and t5 @xcite was trained on 750 gb of text, or approximately 130 billion words. when we rely on a single leaderboard to compare systems trained on corpora with such a large range of sizes, we are not comparing architectures, but rather interactions of architectures, corpus sizes, and computational resources available for training. while this may be a useful comparison for an engineer who seeks to plug an existing trained model into a larger pipeline, this approach is unlikely to advance us towards the goal advocated in this article. the 130 billion word corpus that t5 was trained on is much larger than the corpus that a human can expect to be exposed to before adulthood (fewer than 100 million words, see section 2). but a leaderboard that evaluates only bottom-line transfer learning accuracy inherently disadvantages a sample-efficient model pretrained on a few dozen million words compared to a model such as t5. for all we know, it is possible that architectures rewarded by paid, such as massive transformers, only work well when given an amount of data that is orders of magnitude greater than that available to humans. if that is the case, our exploration of the space of possible models could be going in a direction that is orthogonal to the one that might lead us to models that can imitate humans' sample efficiency (one example of such direction is neural networks with explicit symbolic structure, which are harder to scale up, but perform well on smaller data sets: @xcite . ## identically distributed training set and test set the remaining two letters of the paid acronym refer to the practice of evaluating success on classification tasks using training and test set generated using the same process. typically, a single data set is collected and is randomly split into a training portion and test portion. while this may seem reasonable from a machine learning perspective, it has become clear that this form of evaluation obscures possible mismatches between the generalizations that we as humans believe a system performing the task should acquire, and the generalizations that the system in fact extracts from the data. consider, for example, crowdsourced natural language inference (nli) data sets, in which workers are asked to generate a sentence that contradicts the prompt shown to them @xcite . one strategy that crowdworkers adopt when generating a contradiction is to simply negate the prompt, for example by inserting the word not. this strategy is often effective: the man is sleeping contradicts the man is not sleeping. conversely, it is much less likely that the worker would use the word not when asked to generate a sentence that is entailed by the prompt. taken together, such worker choices lead to a strong correlation between the presence of the word not in the hypothesis and the label contra-diction. it would be surprising if low-bias learners such as neural networks did not notice such a correlation, and indeed they do, leading them to respond contradiction with high probability any time the hypothesis contains a negation word @xcite . of course, relying on the presence of the word not is not a generally valid inference strategy; for example, the man is awake entails, rather than contradicts, the man is not sleeping. numerous generalization issues of this sort have been documented, for nli and for other tasks. in the syntactic domain, @xcite showed that bert fine-tuned on the crowdsourced multinli data set @xcite achieves high accuracy on the multinli test set, but shows very little sensitivity to word order when tested on constructed examples that require an analysis of the structure of the sentence; for example, this model is likely to conclude that the detective followed the suspect entails the suspect followed the detective. in short, the models, unable to discern the intentions of the data set's designers, happily recapitulate any statistical patterns they find in the training data. with a random training/test split, any correlation observed in the training set will hold approximately for the test set, and a system that learned it could achieve high test set accuracy. and indeed, we have models that excel in the paid paradigm, even exceeding the performance of human annotators on the test portion of the corpus used for fine-tuning @xcite , but, when tested on controlled examples, make mistakes that a human would rarely make. 2 the generalizations that a statistical model extracts from the data are always the result of the interaction between the model's inductive biases and the statistical properties of the data set. in the case of bert's insensitivity to word order in nli, the model does not seem to have a strong inductive bias one way or another; its sensitivity to word order varies widely depending on the weight initialization of the fine-tuning classifier and the order of the fine-tuning examples @xcite , and its syntactic behavior in the inference task can be made to be more consistent with human intuitions if the training set is augmented to include a larger number of examples illustrating the importance of word order @xcite . while bert is capable of learning to use syntax for inference given a sufficiently strong signal, then, it prefers to use other heuristics, if possible. this contrasts with human-like generalization in this task, which would likely start from the assumption that any language understanding task should recruit our knowledge of syntax: it would most likely be difficult to convince humans to ignore syntax when understanding a sentence, as bert does. ## the generalization leaderboard what is the way forward? my goal is not to argue that there is no value to the leaderboard approach, where a single number or a small set of numbers can be used to quickly compare models. despite the drawbacks of this approach-in particular, its tendency to obscure the fine-grained strengths and weaknesses of particular models, as i discuss below-hill climbing on a metric can enable a productive division of labor between groups that develop strong benchmarks, groups that propose new models and inference methods, and groups that have the engineering skills and computational resources necessary to train those models on the number of gpus they require to thrive. instead, my argument is that the current division of labor is unproductive. at the risk of belaboring the mountaineering metaphor, one might say that groups with access to engineering and computing resources are climbing the paid hill, while other groups, which document the same models' unreliable generalization behavior-or retrain them on smaller data sets to produce the learning curves that are often missing from engineering papers-are climbing the interpretability track hill, producing papers that are more and more sophisticated and well-respected but do not influence the trajectory of mainstream model development. this section describes some design decisions that can lead to better alignment between the two sets of research groups. many of these points are not new-in fact, some of these properties were standard in evaluation paradigms 10 or 20 years ago-but are worth revisiting given recent evaluation trends. standard, moderately sized pretraining corpora. to complement current evaluation approaches, we should develop standard metrics that promote sample efficiency. at a minimum, we should standardize the pretraining corpus across all models, as some conll shared tasks do. multiple leaderboards can be created that will measure performance on increasingly small subsets of this pretraining corpus size-including ones that are smaller than 100 million words. to make stronger contact with the human language acquisition literature, a leaderboard could compare models on their ability to learn various linguistic generalizations from the childes repository of child-directed speech @xcite . ## conclusion i have described the currently popular pretraining-agnostic identically distributed paradigm, which selects for models that can be trained easily on an unlimited amount of data, and that excel in capturing arbitrary statistical patterns in a fine-tuning data set. while such models have considerable value in applications, i have advocated for a parallel evaluation ecosystem-complete with a leaderboard, if one will motivate progress-that will reward models for their ability to generalize in a human-like way. human-like inductive biases will improve our models' ability to learn language structure and new tasks from limited data, and will align the models' generalization behavior more closely with human expectations, reducing the allure of superficial heuristics that do not follow linguistic structure, and the prevalence of adversarial examples, where changes to the input that are insignificant from a human perspective turn out to affect the network's behavior in an undesirable way.