What do Large Language Models Learn beyond Language? (2022)

Abstract: Large language models (LMs) have rapidly become a mainstay in Natural Language Processing. These models are known to acquire rich linguistic knowledge from training on large amounts of text. In this paper, we investigate if pre-training on text also confers these models with helpful ‘inductive biases’ for non-linguistic reasoning. On a set of 19 diverse non-linguistic tasks involving quantitative computations, recognizing regular expressions and reasoning over strings. We find that pretrained models significantly outperform comparable non-pretrained neural models. This remains true also in experiments with training non-pretrained models with fewer parameters to account for model regularization effects. We further explore the effect of text domain on LMs by pretraining models from text from different domains and provenances. Our experiments surprisingly reveal that the positive effects of pre-training persist even when pretraining on multi-lingual text or computer code, and even for text generated from synthetic languages. Our findings suggest a hithertho unexplored deep connection between pre-training and inductive learning abilities of language models

## introduction pretrained language models (lms) have shown singular succcess on a range of natural language understandings tasks, to the extent that they have become foundational for contemporary nlp systems. several works have investigated why pretraining works so well @xcite . in particular, studies have shown that the pretrained lms like bert capture linguistic knowledge about syntax @xcite , semantics @xcite and morphology @xcite . in fact, @xcite demonstrated that learned representations in pretrained lms even internally reflect the classical nlp pipeline. since most nlp benchmarks such as superglue @xcite naturally are focused on tasks such as textual entailment and reading comprehension that require linguistic knowledge and reasoning, it is unsurprising that lms have achieved strong results on these tasks. on the other hand, little work so far has explored the abilities of pretrained lms for learning nonlinguistic tasks. in this paper, we explore whether pretraining on text is inherently about learning language, or if pretraining also imbues lms with skills for symbolic manipulation and non-linguistic reasoning (for example, performing quantitative computation such as finding the median of a set of numbers, recognizing regular expressions, or identifying whether a string is a palindrome, as shown in figure 1 ). in other words, we investigate whether and how pretraining develops helpful inductive biases for non-linguistic reasoning. for this analysis, we create a set of 19 tasks from three categories of task paradigms: quantitative computation ( §3.1), recognizing regular expressions ( §3.2), and string reasoning ( §3.3). figure 1 shows an example for each category, and the full list of tasks is described in the our experiments ( §5) reveal that pretrained models overall perform substantially better and are more sample efficient on most tasks. however, there are significant differences and patterns in performance between task types, as well as variance between different lm architectures. since nonpretrained models do not have the benefit of regularization that comes from pretraining, a plausible reason for the discrepancy between them and pretrained lms might be underfitting of the nonpretrained models when trained on comparatively small dataset sizes. to account for this, we also comprehensively explore the effect of model size ( §6) of non-pretrained models for both transformer and rnn architectures. we find that the discrepancy in performance remains even for smaller neural models, indicating that the differences are not simply due to a mismatch in model and data sizes. finally, we investigate the role that pretraining data plays in influencing task performance on nonlinguistic tasks ( §7). we experiment with pretraining on different domains of text, pretraining on perturbed representations of natural language text (such as shuffled word order), pretraining on text of computer programs (no linguistic properties of natural languages), pretraining on multi-lingual and non-english text, and pretraining with synthetic text (data sampled from synthetic distributions). our analysis reveals that the advantages of pretraining surprisingly persist with various degrees across these variations, suggesting hithertho unexplored connections between pretraining and the learning abilities of language models. our contributions are: ## related work a body of work has investigated contextual word embeddings to determine whether they capture aspects of mathematical meaning for numbers @xcite . @xcite probed numerical supremacy on token embeddings of contextual language models such as elmo and bert. @xcite surveyed numerical understanding in nlp models using 7 sub-tasks such as measurement estimation and word problems. our work diverges from these in exploring a richer set of tasks including harder tasks such as set operations. further, previous methods explore mathematical reasoning tasks posed as language problems, which conflates the problems of language and mathematical learning and also makes the datasets susceptible to biases due to data collection. our analysis circumvents both these issues by design. some previous works have explored the ability of rnn and transformer architectures for learning regular languages @xcite @xcite , closing brackets @xcite , and dynamic counting @xcite . however, they focus on the learnability of these tasks with specific architectures, and do not look at pretrained lms, which are our focus here. finally, in our discussion, we conceptually stretch the notion of inductive bias. the idea of inductive bias is usually associated with specific model types @xcite , architectures @xcite and regularization approaches @xcite . we believe that extending this to refer to learning tasks with pretrained lms is both reasonable and useful. ## nilm in this section, we describe the tasks used for our analysis, which we refer to as nilm (measuring non-linguistic inductive bias in language models). the tasks correspond to three task paradigms: (1) quantitative computation, (2) regular expressions, and (3) string reasoning. each task in nilm is posed as a classification task. the descriptions for all the tasks with input and output examples, class labels and the input range are shown in ## models & variants next, we describe the lms and their variants used in nilm. we experiment with four language models, based on both transformer and rnn architectures. bert small. this is the bert-base-uncased model with 12 transformer encoder layers and the dimension of the representations is 768. bert tokenizer is based on the wordpiece model @xcite . bert large. this is the bert-large-uncased model which has 24 transformer encoders and representations have 1024 dimensions. deberta. this is a transformer based language model and its tokenizer is built using byte pair encoding @xcite . we consider the de-berta base model. it has 12 transformer encoder layers and representations have 768 dimensions. elmo. this is an lstm based language model @xcite . it has 3 layers and the output representations have 1024 dimensions. our experiments are based on pretrained and non-pretrained variants of these architectures. for pretrained variants, the weights are initialized with the pretrained weights. the tokenization on the training data is performed using the pre-built vocabulary. for the non-pretrained neural models, the weights are initialized randomly and updated during training. the tokenizer used is the same as in the pretrained variant. ## comparative evaluation next, we compare the performance of pretrained and non-pretrained models on tasks in nilm foot_2 . quantitative computation: figure 2 shows results on odd classification, even classification, odd even classification and decimal operation tasks. we find that pretrained lms outperformed non-pretrained model for all of these tasks. further, transformerbased lms outperformed the rnn-based elmo models in all the tasks foot_3 . we note that for the relatively easy tasks such as odd and even classifications, the pretrained lms show more stable training. however, for harder tasks such as decimal operations (where the baseline performance is around 10%), no models are able to learn the task well even with 10k labeled examples. figure 3 shows results on median, mean, mode and decimal & word operation tasks. the median task requires complex reasoning (sorting numbers and computing the middle element), and shows significantly lower performance than the mean and mode tasks for the non-pretrained models even with the maximum training set size. the pretrained lm models show little eventual difference in performance between these three tasks. on the other hand, for the easiest of these tasks (mode), non-pretrained models actually show higher performance than pretrained lms in the low data regime. recognizing regular expressions: figure 4 shows the comparative performance of pretrained lms on non-pretrained models on the two tasks involving recognizing regular expressions. for both tasks, we note that the pretrained lms can perfectly learn the tasks with many fewer labeled examples compared to the non-pretrained models. in both cases, the non-pretrained transformer-based models eventually reach optimal performance as well. however, curiously the elmo based non-pretrained models struggle with learning both tasks. string reasoning: figures 6 show the results on palindrome, anagram, isogram and tautonym classification. these tasks require character comparison within the string or with another string. again, the pretrained variants consistently outperformed non-pretrained models variants in all of these tasks. in particular, the non-pretrained models completely fail to learn the anagram and palindrome tasks even for the largest training set size. again, transformer based lms outperform lstm based lms. figure 7 shows the results on vowels classification, maximum frequent character, length of a string and parity check tasks. these tasks don't require intra-string comparisons. we see that most transformer-based variants eventually achieve optimal performance. for these simpler tasks, we again observe several instances where the transformerbased non-pretrained models actually outperform pretrained lms in the low data regime. figure 5 illustrates the effect of model sizes of non-pretrained model. the original 110 million parameter model has 12 encoder layers, 12 attention heads, and 768 dimensional representations. the 42 million parameter model has 8 encoder layers, 8 attention heads and 512 dimensional representations. the 29 million parameter model has 4 encoder layers, 8 attention heads and 512 dimensional representations. the 11 million parameter model has 4 encoder layers, 4 attention heads and 256 dimensional representations. the smallest 4 million parameter model has 2 encoder layers, 2 attention heads and 128 dimensional representations. as seen in the figure, reducing the model size significantly improves the average performance of the non-pretrained models over 6 representative tasks. however, the smallest models show a performance drop. most significantly, even the best performing intermediate-sized architectures are significantly worse than the pretrained lm models. this strongly suggests that the discrepancy between pretrained and non-pretrained models is not simply due to a mismatch between model and data sizes. ## effects of pretraining data we observe that pretrained lms consistently performed better than non-pretrained models. this leads to the natural question of what role the text data used for pretraining plays in the process. next, we investigate this in depth by experimenting with language models pretrained on different types of text. for this, we pretrain models using the bertsmall and deberta architectures and an mlm objective on different text datasets, and evaluate the performance of these models on nilm tasks. ## conclusion we explore the non-linguistic inductive biases of pretrained lms. while the general trend (that pretraining helps) is unsurprising, our analysis with models pretrained on different text corpora shows that this is not due to the model seeing related topics during pretraining. we find that these gains persist even in absence of any shared linguistic structure (in cross-lingual settings). our observation that this behavior is seen even when pretraining on synthetically generated languages is intriguing