Think Big, Teach Small: Do Language Models Distil Occam’s Razor?

1 Introduction . An effective and reliable interaction between humans and machines depends on a proper understanding of their priors . The alignment between these priors , in the form of knowledge and other predispositions used as bias , is especially critical for communication and learning ( Sun et al. , 2020 ) . For instance , after a human utters the prompt “ on off on off on ” , another human would expect “ off ” to be a more likely continuation than “ on ” . In contrast , after the prompt “ twinkle twinkle little ” we would expect “ star ” to be more likely . The former depends on an abstract pattern intrinsic to the sequence ( a simple algorithmic sequence alternating “ on ” and “ off ” ) , while the latter depends on some language pattern extrinsic to the sequence ( common use in the English language because of a popular song ) . Language models are a kind of AI systems extrapolating sequences of symbols , and are very good at exploiting the extrinsic patterns that are borrowed from humans after training over massive natural language corpora . However , it is only recently , with systems such as BERT ( Devlin et al. , 2018 ) , GPT-2 ( Radford et al. , 2019 ) and GPT-3 ( Brown et al. , 2020 ) , that we see how these models can also discover some intrinsic patterns provided in the prompts . This kind of inference is called ‘ few-shot learning ’ , even if there is no further training or fine-tuning of the model . Many of these language models are based on transformers and other attention-based architectures ( Vaswani et al. , 2017 ) , which do not feature recursion or loops , unlike some approaches based on recurrent neural networks . Some particularly well-chosen and optimised prompts have shown remarkable extrapolations ( Xu et al. , 2020 ; Izacard and Grave , 2020 ; Hendrycks et al. , 2021 ) , illustrating this new kind of abstraction on the go . However , it is unclear how elaborate or algorithmic these abstractions can be . More studies are needed to determine the optimal prompts and example order that make the model capture 35th Conference on Neural Information Processing Systems ( NeurIPS 2021 ) . a range of different patterns ( Zhao et al. , 2021 ; Lu et al. , 2021 ; Liu et al. , 2021 ; Webson and Pavlick , 2021 ; Kirstain et al. , 2021 ) . Even when the prompt template is fixed , it is an open question what the minimal set of examples should be for learning a concept , how this minimality is evaluated and what priors pretrained models incarnate for few-shot learning to work . Precisely , machine teaching is the area of artificial intelligence that looks for the optimal examples or curriculum that a teacher should use to make a learner capture a concept ( Zhu , 2015 ) . This is of enormous importance today in applications such as digital assistants , where we would like to teach them new concepts or routines with a few examples , in the same way humans teach or communicate with other humans ( Cakmak and Thomaz , 2014 ; Goodman and Frank , 2016 ; Degen et al. , 2020 ) . It is also fundamental for explainable AI , as the dual problem appears : how a machine can teach ( explain ) a concept to a human in an efficient way ( Khan et al. , 2011 ; Basu and Christensen , 2013 ; Hernández-Orallo and Ferri , 2021 ) . When dealing with rich representation languages ( e.g. , regular languages or Turing-complete languages ) , closer or similar to human language in expressive power , it has recently been shown that strong simplicity priors can make effective teaching possible with very few examples : the overall size of the example set is usually shorter than the algorithmic expression of the concept itself ( Telle et al. , 2019 ) . For instance , teaching the concept of what an odd number is becomes shorter by using a few examples than by giving the full definition of what an odd number is . Language models and machine teaching can now meet because of this new few-shot learning ability of language models . We can explore the question of how many examples are needed ( or how long they have to be ) to teach something to a language model . These examples and their template are seen as teaching prompts : we want the language model to continue the sequence as if it had captured the pattern that has been expressed in the prompt . From the machine teaching perspective , language models are general systems , as they have not been trained for simplicity or for any particular domain , other than a partial sample of human knowledge expressed as natural language . With their conjunction , we can now ask key questions for AI such as the relevance of simplicity or acquired priors to make learning from a few examples possible ( Chater and Vitányi , 2003 ) . In this paper , we examine how a machine teaching setting inspired by Occam ’ s razor can help us understand how language models ‘ learn ’ and choose prompts more efficiently . We use the term ‘ prior distillation ’ , overloading the notion of knowledge distillation within the field of machine learning , as we analyse whether a simplicity prior is acquired as the result of training large language models such as GPT-2 or GPT-3 . To complete the picture , we also compare them with some other AI systems ( e.g. , program induction systems ) and humans on the same task , which are known to have this prior . The key contributions of this paper are : • We analyse language models in the context of machine teaching , using minimal teaching sets for few-shot inference . • We compare the acquired priors of language models with humans and program induction systems ( Gulwani et al. , 2015 ) , when facing patterns preferred by a simplicity prior . • We find that language models such as GPT-3 have similar or better performance than humans . The performance negatively correlates with the complexity of the concept . • Language models and humans differ in their capability for producing explanations . The length of human explanations correlates with the complexity of the concept . Having a better understanding of language models and its relation to the way other systems , including humans , make inferences is fundamental research that can make the use of these systems safer and more effective , through more robust prompts to solve a variety of tasks . However , this better understanding can also be used maliciously to cunningly modify prompts that alter the results of some applications built on top of language models . A query may fail when using dates , numbers or other strings that have intrinsic patterns . For instance , GPT-3 usually continues “ Mary ’ s father has 5 daughters , Lucy , Anne , Alice , Jane and ” correctly , but fails on “ Mary ’ s father has 5 daughters , Ba , Baba , Bababa , Babababa and ” , systematically continued with ‘ Bababababa ’ . The rest of the paper follows with related work in language models , the comparison of machine and human learning , as well as Occam ’ s razor and machine teaching . Section 3 contains the experimental design , and introduces the systems we consider for comparison . Section 4 presents and analyses the results . Finally , we close the paper with a discussion of the implications of this work . The supplementary material includes additional experiments , plots , tables and specific analyses . 2 Related Work . It is only recently that deep language models are starting to show some minimal capabilities for few-shot learning . Systems such as GPT-3 solve “ tasks which require to perform simple on-the-fly computational reasoning , recognize a novel pattern that is unlikely to have occurred in training , or adapt quickly to an unusual task ” ( Brown et al. , 2020 ) . Examples of these tasks are simple arithmetic operations and the induction of symbolic manipulations such as anagrams or reversed words . Hendrycks et al . ( 2021 ) compare the performance of a text model with humans on 57 tasks including elementary mathematics , US history , law , and more . Sinha et al . ( 2019 ) show that language models obtain poor robustness and generalisation capabilities when compared to graph neural networks that work with symbolic inputs . The experiments also show that humans ( with limited time ) performed well in simple problems but the performance decreases with complex examples . There are also important limitations of language models on simple generalisation tasks ( Bender and Koller , 2020 ; Marcus and Davis , 2020 ) . On the one hand , recurrent neural networks , such as long-short-term-memory ( LSTM ) architectures , have shown the capacity of capturing syntax-sensitive generalisations such as long distance number agreement ( Lakretz et al. , 2019 ) , or language modelling for the game of chess ( Toshniwal et al. , 2021 ) . The abstract reasoning capacity of transformers and other deep learning architectures has been studied by Hupkes et al . ( 2020 ) . In sum , as language models based on transformers are not recursive , it is an open debate whether they are just memorising a huge corpus , or they really have the capacity of learning abstract concepts from data . The comparison between humans and AI systems has usually been performed with systems that are trained for a task , but not for a variety of tasks that are introduced with a few words or examples . For instance , Kühl et al . ( 2020 ) present an experiment that compares the performance of several machine learning algorithms and 44 human participants recognising patterns in 3x3 matrices . The results in this case show that humans , as expected , tend to achieve better performance than the AI systems when few examples are provided . This work suggests that many AI systems may not be competitive with humans when limited examples are given for a range of tasks . Further insight can be obtained when we try to look into the extracted models through explanations , and analyse them in the context of concept complexity . For instance , the relation between complexity of concepts and explanations is discussed by Ai et al . ( 2021 ) . 2.1 Occam ’ s Razor . Occam ’ s razor , the a priori preference for simple hypotheses over more complex ones , has pervaded philosophy of science , inductive inference , and human communication , and it is ubiquitous in statistics and machine learning , explicitly —through criteria such as the MML/MDL principles ( Wallace and Boulton , 1968 ; Rissanen , 1983 ) and the use of regularisation terms— or implicitly —in the optimisation algorithms or parameter-function maps in many methods , including deep learning ( Valle-Perez et al. , 2019 ; Shah et al. , 2020 ) . Occam ’ s razor is backed by strong theoretical and empirical reasons ( Solomonoff , 1964 ; Dingle et al. , 2018 ) , which oppose the idea that all hypotheses should be equally likely a priori , an assumption that would lead to the no free lunch theorems results ( Wolpert , 1996 ) , making learning impossible . While simplicity is always relative to a reference language , the invariance theorem ( see , e.g. , Li and Vitányi , 2008 ) ensures that the use of different ( optimal ) languages gives at most a constant difference that only depends on these two languages . While these constants may be very large , some level of agreement in what simplicity is can happen even for very different representations , as we explore in this paper . A prior for simplicity is hard to understand in large language models , a kind of architecture that is usually overparametrised , compared to the size of the evidence . While the model is not necessarily the most compressed representation of the data , it leads to continuations with low perplexity , which can be used to compress future data , as language models were introduced for communication ( Shannon , 1949 ) . It is this interpretation that sheds light on the role of priors , compression and simplicity in human communication and understanding ( Chater and Vitányi , 2003 ; Dowe et al. , 2011 ) . It is precisely in the context of few-shot inference , once the model has been trained , that we can reformulate the question as whether these models make inferences based on a preference for simpler hypotheses . We use the term ‘ distillation ’ , in a somewhat related way to ‘ knowledge distillation ’ ( Gou et al. , 2021 ) , to represent that the simplicity prior has emerged or have been distilled in the trained model , and used as ‘ learning ’ bias in few-shot inference . For further discussion about Occam ’ s razor , compression and inference , see Section 15 in the supplementary material .

Summary: This paper proposes to empirically investigate to what extent large language models like GPT-3 learn (or "distill") a simplicity prior, i.e., a prior that favors concepts with shorter description length. The authors investigate this by randomly choosing 8 concepts/programs in the P3 language whose total number of P3 instructions number from 1 to 8, respectively. Using the framework of Telle et al. (2019), the authors are able to determine the "witness set," the minimal number of examples necessary for a learner with the correct simplicity prior to distinguish the intended concept from others. This witness set is fed to a learner, which is then evaluated on 5 randomly selected test examples of the concept. The authors also consider feeding additional sets of 2 and then 3 examples to the learner, each with its own random test set. The authors evaluate the accuracy of GPT learners, human learners, and 2 inductive program learners on these 3 test sets. They also elicit explanations from the learners.