Do language models have coherent mental models of everyday things? (2023)

Abstract: When people think of everyday things like an egg, they typically have a mental image associated with it. This allows them to correctly judge, for example, that “the yolk surrounds the shell” is a false statement. Do language models similarly have a coherent picture of such everyday things? To investigate this, we propose a benchmark dataset consisting of 100 everyday things, their parts, and the relationships between these parts, expressed as 11,720 “X relation Y?” true/false questions. Using these questions as probes, we observe that state-of-the-art pre-trained language models (LMs) like GPT-3 and Macaw have fragments of knowledge about these everyday things, but do not have fully coherent “parts mental models” (54-59% accurate, 19-43% conditional constraint violation). We propose an extension where we add a constraint satisfaction layer on top of the LM’s raw predictions to apply commonsense constraints. As well as removing inconsistencies, we find that this also significantly improves accuracy (by 16-20%), suggesting how the incoherence of the LM’s pictures of everyday things can be significantly reduced.

## introduction psychologists and cognitive scientists hypothesize that humans develop mental models of the world, namely internal, conceptual representations of the environment which we base our decisions and actions on @xcite . @xcite observed that 5-month-old human infants exhibit understanding of mechanical properties of objects in terms of arrangements and motions of surfaces, well before they can understand language. drawing loosely on this idea, but without making any claims about how lms reason internally @xcite figure 1 : while humans appear to have coherent mental pictures of everyday things (e.g., an egg, a), our question-asking probes suggest that lms do not (e.g., one lm answered that the egg white both surrounds and is surrounded by the shell, b). this model incoherence can be reduced by applying commonsense constraints (e.g., surrounds is asymmetric), resulting in a more coherent parts model (c). 2022; andreas, 2022), we investigate if pre-trained language models show evidence of coherent internal representations of everyday things, analogous to human mental models, via probing. we focus on mental models in the context of ordinary objects that we encounter in our everyday lives. such commonsense knowledge helps us understand how these everyday things work and how to interact with them. for example, when someone tries to make a fried egg, they know that it has a shell and that it can be cracked open to reveal the egg white and yolk inside. however, if a system does not have a coherent picture of such everyday things, thinking that the egg yolk surrounds the shell, then it might have to resort to ridiculous approaches such as trying to scrape the egg yolk off the shell into the pan. we explore a first version of this, in which we consider only knowledge about an object's parts and their relationships. we refer to this knowledge as a parts mental model. we first create a benchmark dataset of 100 everyday things, by asking human annotators to draw a graph representing their parts mental model (e.g., figure 2 ) depicting the parts of an everyday thing, spatial relationships, connections between its parts and functional dependencies (if any). then we probe two representative state-of-the-art lms with questions about these everyday things. we find that the lms' parts mental models are generally of poor quality. further, model predictions can violate basic consistency constraints e.g. transitivity. to alleviate this, we apply constraint reasoning to derive more accurate and consistent mental models of everyday things, correcting some of the lms' original inconsistencies. this is illustrated in figure 1 . our contributions are: 1. we present a benchmark dataset of parts mental models consisting of 100 everyday things, 2.2k parts and 11.7k relationships. 2. we show that sota lms like gpt-3 and macaw are poor at answering relationship queries between parts of everyday things. the parts mental models derived using their predictions are only 54-59% accurate, and significantly inconsistent (19-43% conditional violation τ ). 3. we propose a neuro-symbolic method that applies constraint reasoning on top of raw lm predictions as a way of obtaining more consistent (0% conditional violation τ ) and more accurate mental models (16-20% improvement). this suggests a broader cognitive architecture (lm + reasoner) for future systems, to better construct mental models than the lm alone. mental models: the idea of mental models @xcite ) is not new. many years ago, @xcite proposed that thinking itself is the manipulation of internal representations of the world. craik (1943) described mental models as a 'small-scale model' of external reality and of its own possible actions within someone's head. such a mental model is useful in many ways, including allowing one to try out various alternatives, make conclusions, react to future situations, learn from past events, and in general, improve competency. years later, when @xcite outlined the mental processes that underlie human reasoning, he based his discussion on the fundamental assumption that human beings can construct internal representations of spatial layouts, and specified mental models to be iconic. in his words, a mental model's "parts and the relations among them correspond to the parts of the layout and the relations among them." while coherent internal representations of spatial layouts are crucial for human reasoning, their role, coherence, and even existence in lms have not been systematically explored. in this work, we try to bridge this gap by proposing a benchmark dataset and methodology to compare human internal representations of spatial layouts of everyday things with those of lms. prior datasets: prior works on reasoning about object/body parts include @xcite which focused on human body parts and human interaction with other objects. the ptr benchmark @xcite is a qa dataset about objects and their parts, combining 5 everyday things: chair, table, bed, refrigerator, and cart, to create questions across 70k different scenes. @xcite used tangram puzzles to analyze shape naming, part naming and segmentation divergence across participants when they see a certain shape. contributing to this existing body of datasets, the dataset we introduce serves as a resource for researchers to study canonical parts mental models for a wide variety of everyday things, focusing on relationships between parts of objects, which is fundamental to how humans think and interact with these things. large language models: despite recent advances in lms, studies suggest that they still struggle at reasoning with real-world entities and concepts. @xcite found that when lms answer questions involving physical commonsense reasoning, their performance at that time was near chance level for questions involving spatial relations like "top" and "bottom." @xcite demonstrated the lack of conceptual consistency in lms by correlating models' answers on commonsense reasoning questions (csqa dataset) and their answers on associated conceptual questions from conceptnet knowledge base. to improve existing systems, progress has been made such as by imposing constraints with neuro-symbolic approaches @xcite and incorporating both textual and visual information @xcite . inspired by recent progress, we propose a constraint reasoning method that applies hard commonsense constraints (e.g., if 'a above b' is true then 'a below b' cannot be true) on top of raw lm predictions to produce more accurate and consistent mental models of everyday things. we define "parts mental model" for everyday things in this section. then in the rest of the paper, we describe how we collect a dataset for them, measure lms' coherence on them, and finally apply external reasoning to improve the accuracy and consistency of lms' parts mental model. here, we use parts mental model to mean a partsfocused subset of a complete mental model of an entity. we represent a parts mental model as a directed graph where parts of the everyday thing form the nodes of this graph and these nodes are connected with edges indicating how these parts are related to each other. based on prior works such as @xcite and @xcite , we selected 11 spatial orientation relations to focus on. in addition, we augmented these with relations describing connectivity and functional dependency. in total, we consider 14 relationships (across these 3 categories) between parts, listed in note that the notion of a single "parts mental model" for an everyday thing is somewhat uncon-strained (e.g., which parts to pick? what version of the entity are we talking about?). to make this task more well-defined, we also provide a predefined list of parts as a guide (details in section 4.1), and the task for annotators or a model is to specify relationships between them as they see appropriate, using our ontology of relationships. this is important so that we can do meaningful comparisons between language models and humans' notion of parts mental models of everyday things. figure 2 shows two examples of parts mental models in our dataset, where edges encode relationships between parts. e.g., in a tree, "trunk is above the roots"; in a flashlight, "bulb requires the batteries," etc. inspired by previous literature, we envision that such parts mental models would play a key role when one carries out daily activities involving these everyday things. ## everyday things dataset: parrot (parts and relations of things) we created a dataset of common entities that one would encounter in their daily life. for each everyday thing, our dataset (parrot) contains a "parts mental model" in the form of a graph, which depicts parts of the entity and relational information about the parts. such a graph encodes a partsfocused mental model of that everyday thing, potentially useful for reasoning about how the entity works and how to interact with it. ## measuring and improving parts mental models our proposed approach, parrot-con, foot_6 comprises two main components. 8 the first component "probing a pre-trained language model" sends an exhaustive list of relation queries to a lm querying for every relation between each pair of parts (e.g. all relationships between egg white, yolk, shell, shell membrane and air cell). this gives us a large set of candidate relation tuples along with the model's confidence in each of them. incorrect relation predictions can result in inconsistencies in the mental model. e.g, "egg white both surrounds and is surrounded by the egg shell." the second component "constraint reasoning" then applies a constraint satisfaction layer on top of these raw predictions to choose a subset of these relation tuples that are maximally probable and minimally conflicting with each other. note that parrot-con is a zero-shot approach, where both probing lms and constraint reasoning steps do not require any task-specific fine-tuning or re-training. ## conclusion do language models have coherent mental models of everyday things? to systematically study this question, we present a benchmark dataset, parrot, consisting of 300 human-constructed mental models for 100 everyday objects, including over 2k parts and 11.7k relationships between these parts. our experiments reveal that even sota lms generally have poor mental models (inaccurate and violating basic commonsense constraints) of everyday things, thus providing insight into their apparent knowledge and behavior not previously explored. we apply constraint reasoning on top of base lm predictions to construct more coherent mental models. our method, parrot-con, improves both accuracy (up to 20% improvement) and consistency (up to 43% improvement) of such parts mental models. this suggests a broader cognitive architecture (lm + reasoner) for future systems, to construct more coherent mental models than using the lm alone.