Mapping Language Models to Grounded Conceptual Spaces

1 INTRODUCTION . Large pre-trained language models ( LMs ) trained on text corpora have shown remarkable progress on a range of natural language understanding tasks ( Radford et al. , 2019 ; Brown et al. , 2020 ) . Such models have demonstrated their ability to generate fluent dialogue ( Brown et al. , 2020 ) , make commonsense inferences Zellers et al . ( 2019 ) , and reconstruct taxonomies and word relations ( Chen et al. , 2020 ) . However , it has been argued that true meaning can not be learned from the form of language alone ( i.e. , from text ) because it is a word ’ s use in the non-linguistic world that imparts it meaning ( Bender & Koller , 2020 ; Bisk et al. , 2020 ) . For example , although LMs might learn from textual co-occurrences that the words north and south are opposites , the grounded meaning of these words , i.e. , the direction that you should travel if you are told to go north , is something which these models , by definition , do not have access to during training . While it is indisputable that text-only models do not learn representations of concepts that are grounded in the non-text world , it is possible for the structure of relations between concepts in text form to be identical to what a grounded model would learn . In principle , it is therefore possible for a text-only model ’ s conceptual space to be isomorphic to the “ true ” , grounded conceptual space ( Merrill et al. , 2021 ) . In this work , we investigate whether this is the case by asking whether models can learn to ground an entire domain ( e.g. , direction ) after grounding only a subset of the points in that domain ( e.g. , left ) . Specifically , for generative LMs that have been trained only on large text corpora , we “ orient ” the models by showing them how some word forms ( that they have learned during training ) are used in simple text worlds—for example , what the direction north maps to in a textual representation of a grid world ( see Figure 1 ) . We then evaluate two types of generalisation . First ( §3.1 ) , we evaluate generalisation to unseen worlds . For example , if the model has seen several realisations of the word north in different grid worlds , can it correctly identify north in an unseen world ( e.g. , one of a different size or shape ) ? Second ( §3.2 ) , we evaluate generalisation to unseen but related concepts . For example , if the model has been shown grounded representations of north and east , can it correctly identify south and west , even though it was never shown to them ? We find that although the small language models ( GPT-2 models that contain an order of 100M parameters ) can not perform either generalisation well , the largest model ( a GPT-3 model containing 175B parameters ) can indeed learn groundings in the conceptual worlds we build . We analyse the predictions and errors made by models ( §3.3 ) and find that the errors made by the small models often come from them struggling to produce in-domain references and defaulting to random words . In contrast , the errors made by the largest model are quite intuitive , as they predict in-domain concepts that are often close to the true concept in the world ( e.g. , maroon versus dark red ) . 2 EXPERIMENTAL DESIGN . 2.1 MODELS . We test five autoregressive Transformer language models ( Vaswani et al. , 2017 ) of varying size , specifically the GPT-2 ( Radford et al. , 2019 ) and GPT-3 ( Brown et al. , 2020 ) models . The smallest model we use contains 124M parameters , and the others follow increasing model sizes ( 355M , 774M , 1.5B and 175B parameters ) . All models are pre-trained on differently filtered versions of the OpenAI Web-Text dataset Radford et al . ( 2019 ) , composed of 40GB of English web text available on the internet . We generate up to 5 tokens per prompt and , to improve the robustness of our analyses , generate 3 samples for every prompt . We use a temperature of 1 during generation and sample from the softmax probabilities produced at each time step using nucleus sampling ( Holtzman et al. , 2019 ) with p = 0.85 . We include detail on all model aspects , as well as discussion on their training data in Appendix A . 2.2 IN-CONTEXT LEARNING . Several studies ( Brown et al. , 2020 ; Reynolds & McDonell , 2021 ) have shown that instead of finetuning generative LMs–i.e. , a process that updates parameters learned by the model during pretraining , it is possible to achieve competitive performance by giving the model a small number of training examples within the prompt . This is often referred to as “ in-context learning ” or “ fewshot prompting ” . Specifically , a prompt includes n task examples that include a question prefix ( e.g. , “ World : ” ) followed by the question , and an answer prefix ( e.g. , “ Answer : ” ) followed by the answer to the question . After giving the model n examples in this manner , the prompt ends with a new question and only an answer prefix after which the model is expected to generate an answer to the last question , following the prompt format it has seen . By enumerating over all questions in the test set , we can obtain a model-generated answer for every test set question that we wish to evaluate . There are no gradient updates to any model parameters using this approach . FINAL 2.3 GROUNDED CONCEPT DOMAINS . The models that we test , by construction , can receive only text inputs . Thus , we focus on a set of grounded domains for which it is possible to faithfully represent the grounded meaning in text form . We briefly describe these domains below , summarise them in Figure 1 , and describe in detail , the prompt creation process for each generalisation task in Sections §3.2 and §3.1 . We discuss the possibility of expanding this set of concepts in future work in Section §4 . Spatial Terms We consider 6 spatial concepts : left , right , up , down , top , bottom . Each of the above concepts can be represented in a grid world using the position of a special character ( here , a ‘ 1 ’ ) 1 in the world . To do this , we create grid world environments of varying sizes ( where the number of rows and columns ranges from 1 to 8 ) , where each world consists of ‘ 0 ’ s and a single ‘ 1 ’ . Cardinal Directions We consider eight cardinal directions : north , south , east , west , northeast , northwest , southeast , southwest . These are similar to spatial terms , except that they include compositional terms ( e.g. , northeast ) . We use grid-worlds of the same format as spatial terms to represent these concepts . Colour Terms We consider colour terms in a three-dimensional space , using a dataset of 367 RGB colours ( Abdou et al. , 2021 ) that contains colour names ( e.g. , red , cyan , forest green ) each associated with an RGB code ( e.g. , ( 255 , 0 , 0 ) ) . Therefore , the world representation in this case is not a grid-world , but an RGB code associated with every colour name . Figure 1 shows example RGB codes and colours that serve as part of a prompt given to the models we test . 2.4 ROTATED WORLDS TO CONTROL FOR MEMORISATION . Motivation The GPT-x models that we use have been trained on the CommonCrawl corpus ( Radford et al . ( 2019 ) ) , a collection of documents that contains web text freely available on the internet . Since we provide instantiations of grounded concepts in text form , it is very plausible that the domains described above have been encountered verbatim during training . For example , for spatial terms such as left , a model might have seen instances of matrices and linear algebra terms with the word left in close proximity ; for colour terms , tables that map RGB colour codes to colour names are pervasive in web-text . We therefore include a control task in our experimental setup such that the model can not succeed using simple memorisation . Rather , success on a task requires the model to truly perform a conceptual mapping between the ungrounded and grounded representations . Isomorphism as a Control We use the concept of isomorphism to control for memorisation . Intuitively , imagine a situation where you are lost in the woods . Once pointed in the direction of north , you instantly know which way is south . However , this ability is not dependent on having been correctly pointed north–if someone were to incorrectly point east and tell you this was north , you would readily infer west to be south . This is because your reasoning depends on your knowledge of the relation between north and south , and between north and the world , rather than on having memorized the “ true ” grounding of each concept independently . By the same logic , if a model is learning a grounding function , this should be dependent on the world that it is being grounded in . For example , in different worlds where the word red grounds to different points in space , the word blue , by analogy , shares a fixed conceptual relation to the word red . Therefore , it should ground in correspondingly equidistant ways in the two worlds . A model ’ s ability to learn two different grounding functions f vs. g , should not be dependent on what the actual points ground to , as long as the structural relations between concepts in the space are preserved . Further , this should hold for all such isomorphic transformations that preserve the structure of the space , and importantly , it should not hold for random perturbations that distort the structure of the space , since such distortions would break the assumption that the relation between red and blue in ungrounded space is analogous to that in grounded space . Mechanism In the colour domain , since colour concepts exist in a 3D world of RGB codes , we rotate each point around a fixed axis by a certain degree to create a new isomorphic world . We repeat this control three times ( for 90◦ , 180◦ and 270◦ rotations ) and average over the rotations in our evaluations . For cardinal directions , we rotate each cardinal concept by 90◦ in two dimensions , and do this three times and average over rotations . Since the spatial terms exist as pairs , we simply 1As a control , we perform the same experiments by varying the special character , e.g. , replacing a 1 with a 2 or replacing them with letters such as a or b . This does not effect our results , see Appendix B swap the groundings of alternate terms ( e.g. , left and right ) . For random worlds that do not preserve the structure between word forms , we randomly assign a concept name ( e.g. , red ) to a point in the world , and we do this for all concept names and categories to obtain a random world for each . Figure 2 shows example transformations of colours on rotating by 90◦ , as well as randomly rotating points . Isomorphism and Rotated Worlds FINAL FINAL

Summary: This work seeks to probe large pre-trained language models for indications that they have induced a conceptual space that is structured similarly to one constructed from interactions with the real-world, despite being trained solely from text.  Most of this probing takes the form perceptual tasks -- can LMs quickly learn new concepts related to navigation, or colors, if given some related examples from the space?  In both navigation and color tasks, this seems to be the case, and a consistent trend emerges where larger LMs perform better on this task.  The experiments seem well-controlled, and the authors present an insightful discussion of model performance.