POSQA : Probe the World Models of LLM s with Size Comparisons (2023)

Abstract: Embodied language comprehension emphasizes that language understanding is not solely a matter of mental processing in the brain but also involves interactions with the physical and social environment. With the explosive growth of Large Language Models (LLMs) and their already ubiquitous presence in our daily lives, it is becoming increasingly necessary to verify their real-world understanding. Inspired by cognitive theories, we propose POSQA: a Physical Object Size Question Answering dataset with simple size comparison questions to examine the extremity and analyze the potential mechanisms of the embodied comprehension of the latest LLMs. We show that even the largest LLMs today perform poorly under the zero-shot setting. We then push their limits with advanced prompting techniques and external knowledge augmentation. Furthermore, we investigate whether their real-world comprehension primarily derives from contextual information or internal weights and analyse the impact of prompt formats and report bias of different objects. Our results show that real-world understanding that LLMs shaped from textual data can be vulnerable to deception and confusion by the surface form of prompts, which makes it less aligned with human behaviours.

## introduction the rapid growth of recent large language models (llms) such as chatgpt has led to their increased use in various applications @xcite @xcite @xcite . with the rapid growth of interest in developing embodied language models (elm) @xcite @xcite , recently there has been increas-ing interest in investigating whether llm have an aligned understanding of the real world as our human from cognitive and physiological perspectives @xcite @xcite . embodied language comprehension @xcite @xcite , a possible explanation for human cognition, suggests that the human develops an understanding of the physical world related by language by our physical experiences and sensory perceptions of the world around us. when we process languages, we reemulate or recreate the experiences mentioned in the language to understand and interact with those languages more meaningfully. although common sense physical reasoning has been widely explored previously with various benchmarks, such as piqa @xcite , mmlu-physics @xcite , utopia @xcite , and prost @xcite , few studies analyse the understanding of llms about object size, which is actually central to various fundamental aspects of cognition such as implicit memory, object recognition, conceptual processing, and perception-action coordination @xcite . therefore, inspired by cognitive experiments @xcite , we proposed posqa: a physical object size question-answering dataset containing 12,000 questions of size comparisons between pairs of objects to investigate whether the latest llms have aligned cognition with our human and identify the limits of their real-world understanding with various promptbased experiments. empirical findings suggest that under the zeroshot setting, the performance of popular llms such as gpt-3 is slightly better than random guessing. however, increasing the types and amount of external knowledge presented in the prompt about objects has a significant impact on the behaviour of llms. in particular, llms tend to develop their mental representation of objects referred to based on the given context in prompts rather than relying on their internal weights, even if the given context information is incorrect. to conclude, our contributions can be summarized into three folds: ## posqa posqa consists of 12,000 multiple choice questions designed to probe the physical world understanding ability of the language model in the size dimension. we design two types of questions, each of them containing 6,000 questions. general question a general question requires the answer "yes" or "no". we use two templates to generate general questions. template 1: is entity a bigger than entity b? template 2: is entity a smaller than entity b? we replace entity a and entity b with different entity names. for each template, we use the same entity a -entity b pair to generate a question, which is to avoid introducing bias. we generate 3,000 questions for each template, so there are 6,000 general questions in total. based on the actual size of each entity, we label each question with "yes" or "no". the general questions aim to evaluate the size knowledge of objects contained in the lms and the understanding of lms on yes/no labels. special question a special question begins with an interrogative word "which". we design two templates for special questions. template 3: which one is bigger between entity a and entity b? template 4: which one is smaller between entity a and entity b? similarly, we also use the same entity a -entity b pair to generate a question on each template. we generated a total of 6,000 questions, 3,000 questions for each template. the label of a special question is different from the general question. in the special question, we label each question with the actual entity name, which is exactly the same as entity a or entity b. the special questions are intended to test the understanding of the size of the lms and the understanding of the lms about the interrogative word 'which' of the question. ## methodology in this section, we cover the details of the proposed approach, first describing the designed prompt in section 4.1, followed by the models used in the experiments in section 4.2 and the introduction of the evaluation methods in section 4.3. ## discussion llms' ability to understand the size of physical objects in the real world remains a challenge. our experiments underscore that even the most advanced llms at our disposal struggle to consistently grasp the sizes of physical objects. specifically, gpt3.5-turbo registers an average accuracy score of 0.71 when directly addressing the two question types present in posqa. this performance reveals a pronounced disparity compared to human comprehension of size, particularly when object size information is explicitly provided. humans, when confronted with size comparison tasks, often engage in mental simulations, drawing upon their accumulated knowledge to envisage the sizes of objects @xcite . for a human, the ability to tackle size comparison effectively hinges on possessing adequate size-related information about the objects in question. similarly, llms should leverage the knowledge encoded in their internal weights to adeptly respond to size comparison queries. llms prefer to utilise the information in the given context rather than knowledge stored in their internal weights. our experiments demonstrate that giving useful information in a prompt can enhance the performance of llms. for instance, gpt3.5-turbo achieves 0.9 accuracy score on both two types of questions. however, llms cannot make good use of the external context information when they are only given partial information. it is noteworthy that adding error information to the prompt largely decreases the performance of llms. for example, gpt3.5-turbo can only get 0.38 accuracy and macro-f1 score on general questions and 0.29 accuracy and macro-f1 score on special questions. even research @xcite has shown that adding an instruction to ignore irrelevant information brings performance gains; a single piece of irrelevant information can distract the models and substantially degrade their performance. these results indicate that the context in the prompt is extremely important for llms and that llms will utilise the information in the context. llms are sensitive to the format of the query, even if they are semantically equivalent. in our experiments, we query llms with different format of size comparison questions from posqa. the results show that llms are not robust enough when faced with the same semantics, but different forms of the queries. for example, on ft5-xl, it achieves a 0.58 accuracy score on general questions versus a 0.72 accuracy score on special questions. based on the behaviour of the llms on answering size comparison questions, it is not certain which forms of questions the models are better at solving, and the performance is also influenced by the context added to the prompt. although the performance gap between these two types of questions in gpt3.5-turbo has narrowed, it is still a noteworthy problem when training robust llms in the future. as shown in ## conclusion we propose posqa: a physical object size question-answering dataset with two types of size comparison questions to probe the ability of llms to understand the size of physical world objects. we design different knowledge-augmented prompt settings to investigate the effect of the context in the prompt. our experiments demonstrate that llms still fail to demonstrate a robust understanding of the size of physical objects. the ability of llms to understand the size of physical objects in the real world remains a challenge for the future. the results also show that llms prefer to utilise the information in the given context rather than to use the knowledge stored in their internal weights. this also raises concerns about the robustness of llms' understanding ability to identify the useful and correct contextual information in the prompt.