POSQA: Probe the World Models of LLMs with Size Comparisons
Authors: Chang Shu
Jiuzhou Han
Fangyu Liu
Ehsan Shareghi
Nigel Collier
Venue: EMNLP 2023 Findings (2023)
Keywords: Embodied language comprehension
World Model
Large Language Models
AI Alignment

Abstract: Embodied language comprehension emphasizes that language understanding is not solely a matter of mental processing in the brain but also involves interactions with the physical and social environment. With the explosive growth of Large Language Models (LLMs) and their already ubiquitous presence in our daily lives, it is becoming increasingly necessary to verify their real-world understanding. Inspired by cognitive theories, we propose POSQA: a Physical Object Size Question Answering dataset with simple size comparison questions to examine the extremity and analyze the potential mechanisms of the embodied comprehension of the latest LLMs.

We show that even the largest LLMs today perform poorly under the zero-shot setting. We then push their limits with advanced prompting techniques and external knowledge augmentation. Furthermore, we investigate whether their real-world comprehension primarily derives from contextual information or internal weights and analyse the impact of prompt formats and report bias of different objects. Our results show that real-world understanding that LLMs shaped from textual data can be vulnerable to deception and confusion by the surface form of prompts, which makes it less aligned with human behaviours.

POSQA: Probe the World Models of LLMs with Size Comparisons
Chang Shu*
Jiuzhou Han
University of Cambridge
Monash University jiuzhou. www..haaam.haeeeeeeo
cs2175@cam. haa1755cam.ac.uk
Fangyu Liu' Google DeepMind liufangyu@google. com
Ehsan Shareghi Monash University ehsan. Shraennhhareg@@mmmmnne www.ash..mm...mmmmmmmmmmmmm edu
Nigel Collier
University of Cambridge nhc300cam ac. uk
Abstract
ing interest in investigating whether LLM have an aligned understanding of the real world as our human from cognitive and physiological perspec- tives (Prystawski et al., 2022; Binz and Schulz, 2022; Hagendorff et al., 2022: Mahowald et al.. 2023). Embodied language comprehension (Hor- chak et al., 2014; Buccino et al., 2016; Fischer and Zwaan, 2008; Barsalou, 1999), a possible explana- tion for human cognition, suggests that the human develops an understanding of the physical world re- lated by language by our physical experiences and sensory perceptions of the world around us. When we process languages, we reemulate or recreate the experiences mentioned in the language to un- derstand and interact with those languages more meaningfully.
Embodied language comprehension empha- sises that language understanding is not only a matter of mental processing in the brain, but also involves interactions with the physical and social environment. With the explosive growth of Large Language Models (LLLMs) and their already ubiquitous presence our daily lives, it is becoming increasingly necessary verify their real-world understanding. Inspired by cog- nitive theories, we propose POSQA: a Physical Object Size Question-Ansn Answering dataset with simple size comparison questions to examine the extremity and analyse the potential mech- anisms of the embodied comprehension of the latest LLM.
We show that even the largest LLMs today per- form poorly under the zero-shot setting. We then push their limits with advanced prompting techniques and external knowledge augmen- tation. Furthermore, we investigate whether their real-world comprehension primarily de- rives from contextual information internal weights and analyse the impact of prompt for- mats and report bias different objects. Our results show that real-world understanding that LLMs shaped from textual data can be vulnera- ble to deception and confusion by the surface form of prompts, which makes it less aligned with human behaviours.
Although common sense physical reasoning has been widely explored previously with various benchmarks, such as PIQA (Bisk et al., 2020b), MMLU-Physics (Hendrycks al., 2021), UTOPIA (Liu et al., 2022b), and PROST (Aroca-Ouellette et al., 2021), few studics analyse the understand- ing of LLMs about object size, which is actually central to various fundamental aspects of cogni- tion such as implicit memory, object recognition, conceptual processing, and perception-actionn co- ordination (Biederman and Cooper, 1992: Barsa- lou, 2008). Therefore, inspired by cognitive ex- periments (de Koning et al., 2017a), we pro- posed POSQA: a Physical Object Size Question- Answering dataset containing 12,000 questions of size comparisons between pairs of objects to in- vestigate whether the latest LLMs have aligned cognition with our human and identify the limits of their real-world understanding with various prompt- based experiments.
1 Introduction
The rapid growth of recent Large Language Models (LLMs) such as ChatGPT has led to their increased use in various applications Garrido-Merchán, 2023; Sobania et al., 2023; Lehn- ert, 2023; Guo et al., 2023; Nov et al., 2023; Jiao al., 2023). With the rapid growth of interest in developing Embodied Language Models (ELM) (Dasgupta et al., 2023; Driess et al., 2023; Vem- prala et al., 2023), recently there has been increas-
and
Empirical findings suggest that under the zero- shot setting, the performance of popular LLMs such as GPT-3 slightly better than random guess- ing. However, increasing the types and amount of external knowledge presented in the prompt about
Equal contribution Work done at U University of Cambridge. Code: https:// // thub, com/ hamm//cbbrdgeet///Pee

---

objects has a significant impact on the behaviour of LLMs. In particular, LLMs tend develop their mental representation of objects referred to based on the given context in prompts rather than relying on their internal weights, even if the given context information is incorrect.
PIQA dataset consists of more than 16,000 train- ing QA pairs, with additional 2K and 3K held for development and testing. The task is multiple choice question answering: Given a question and two possible solutions, a model or a human must choose the most appropriate solution, of which ex- actly one is correct. MMLU-Physics (Hendrycks et al., 2021) contains 206 samples of physics con- sisting of multiple choice questions the college and high school level evaluate the academic and professional understanding of the model in the physics domain. UTOPIA (Liu et al., 2022b) is a new multi-task physics alignment dataset that aims to benchmark how well current LMs can un- derstand and areas over some basic laws of physics. It leverages a physics engine to generate data for 39 subtasks covering six common scenes that in- volve understanding basic principles of physics. PROST (Aroca-Ouellette et al., 2021) is new probing dataset to evaluate the ability of pre-trained LMs to understand and reason about the physical world. It contains 18,736 multiple-choice questions made from 14 manually curated templates, cover- ing 10 physical reasoning concepts. The existing datasets contain questions from different dimen- sions, but they fail to effectively evaluate some particular aspect of the understanding of the phys- ical world of L.Mss. Since we want to probe the effect of context for in-context lcarning, is neces- sary to have a content-conttrllabbee and dimension- specific dataset. Based on the requirements, we propose POSQA: a Physical Object Size Question- Answering dataset which is also designed to test the size understanding ability of LLMs physical world objects.
To conclude, our contributions be summa- rized into three folds:
We propose simple but effective size com- parison dataset to probe the real-world under- standing of LLMs.
We analyse the limits of the real-world un- derstanding of LLMs with comprehensive prompt-based probing experiments.
We discuss the vulnerability and the alignment of the world knowledge of LLMs.
2
Background
2.1 World Models and World Knowledge
There has been a wide and long-lasting debate about whether LLMs really have their internal world models and to what extent their world knowl- edge aligns with humans. Mind's Eye (Liu et al., 2022b) proposed to augment language models with an external physical simulation engine for better understanding the physical phenomena. RAP (Hao al., 2023) suggests LLMs as both world model and a reasoning agent and includes a principled planning algorithm for strategic exploration a vast reasoning space. Xiang et al. (2023) deploys an embodied agent in a world model to endow LLMs with a diverse set of embodied experiences by fine-tuning. Although the world model of LLM can be effectively augmented and they indeed dis- play certain level of real-world understanding, there still a lack of sufficient study on the bound- ary of the world understanding of LLMs. Similarly to other research on LLMs inspired by cognitive sci- ence and psychology (Binz and Schulz, 2022; Bisk et al., 2020a; Mahowald et al., 2023; Prystawski et al., 2022), we propose to audit the real-world understanding of LLM with questions as simple as size comparison.
POSQA
POSQA consists of 12,000 multiple choice ques- tions designed to probe the physical world under- standing ability of the language model the size dimension. We design two types of questions, each of them containing 6,000 questions. Table shows the statistics of POSQA. The size comparison cov- 92 entities, ranging from proton to universe. The entity and size information are obtained from Nikon Universcale, which aims to allow people to see and understand the relative size of the full range of known objects in our universe. We design four manually written templates to construct the two types of size questions. We show the templates
2.2 Physical World Understanding Datasets
PIQA (Bisk et al., 2020b) is a popular data set for physical commonsense reasoning to benchmark progress in physical commonsense understanding.

---

Question Type
Bigger
Smaller
Total
feature represents the exponent of size which is stored in the size feature. The text feature contains a textual description of the entity.
General Question
3,000
3,000
6,000
Special Question
3,000
3,000
6,000
Methodology
Table I: The statistics of POSQA.
In this section, cover the details of the proposed approach. first describing the designed prompt in Section 4.1, followed by the models used in the experiments in Section 4.2 and the introduction of the evaluation methods in Section 4.3.
in detail below.
General Question A general question requires the answer "yes" or "no". We use two templates to generate general questions.
4.1 Prompt Design
TEMPLATE 1: Is Entity A bigger than Entity B? TEMPLATE Is Entity A smaller than Entity B? We replace Enrity A and Enrity B with differ- ent entity names. For each template, we use the same Entity A - Entity B pair to generate a ques- tion, which is to avoid introducing bias. We gen- erate 3,000 questions for each template, so there are 6,000 general questions in total. Based on the actual size of each entity, we label each question with "yes" "no". The general questions aim to evaluate the size knowledge objects contained in the LMs and the understanding of LMs on yes/no labels.
We construct different prompts that are used in our experiments.
4.1.1 Plain Question Prompt
Plain question prompt to query the model with single plain question without any hint or knowl- edge. We aim to test how the model performs on POSQA without any external auxiliary information. The model answers the query purely based on the knowledge stored within its weights.
4.1.2 Relevant Knowledge-augmentte Prompt
External knowledge has been shown to be helpful for various NLP tasks, including common sense reasoning (Liu et al., 2022a). We consider two kinds of knowledge in our experiments: (1) Ex- act Size Information from POSQA (2) Generated Knowledge from GPT-3. Knowledge is considered as the context that is concatenated with a ques- tion. We use the mnoweedee-augment prompt to query LLMs to see how the context would affect the model's prediction.
Special Question A special question begins with an interrogative word "which". We design two templates for special questions.
TEMPLATE 3: Which one is bigger between Enrity A and Entity B?
TEMPLATEE Which one is smaller between Entity and Entity B?
Similarly, we also use the same Entity A - Entity B pair to generate question on each template. We generated a total of 6,000 questions, 3,000 ques- tions for each template. The label of a special question is different from the general question. In the special question, we label each question with the actual entity name, which is exactly the same as Entiry A or Entity B. The special questions are intended to test the understanding of the size of the LMs and the understanding of the LMs about the interrogative word 'which' of the question.
Exaet Size Information The exact size informa- tion of each object in POSQA stored as an entity feature. For a size comparison question, first we retrieve the exact size the two entities respectively, then we rewrite the original prompt to knowledge- augmented prompt. In particular, to describe the size of the two objects before the question. The sentence is: The size of Entiry is Exact Size of Entity A. The size of Entity B Exact Size of Entity B.
add sentence
Entity Feature We collect the features of the 92 entities, including scale, size, magnitude, and text. The scale feature stores the size information of the entity in a specific size unit. For example. the scale of the Solar System is 9 billion km and the scale an Atom 100 pm. The size feature stores the absolute value of the size of the entity representing in scientific notation. The magnitude
Generated Knowledge We generate the entity- related knowledge statement by querying LLM. We consider two types of knowledge of entities: (1) general knowledge, which describes the general in- formation of an entity; (2) size knowledge, which describes the size information an entity. The purpose is to investigate which knowledge is more

---

useful to the model when answering the size-related questions. We query GPT-3 using the prompt 'Gen- erate knowledge about entity in one sentence." for extraction of general knowledge and 'Generate size knowledge entity one sentence." for size knowledge extraction. The knowledge generated from GPT-3 is stored and used context in the knowledge-angmeence prompt. Then we concate- nate the knowledge generated with the size com- parison question to query LLMs.
4.2 Models
Previous work (Wei et al., 2022a) (Sanh et al., 2022) has shown that instruction-tuned language models a collection of NLP tasks formatted with instructions substantially improve the ability of language models to perform an unseen task from an instruction, especially zero-shot performance. In this work, we do experiments on three kinds of instruction-tuned model: Flan-T5 (from 80M to 3B) (Chung et al., 2022), InstructGPT (175B) (Ouyang et al., 2022), and recent ChatGPT. Flan- T5 is instruction-tur -uned initialise from prior public checkpoints of T5 (Raf- fel al., 2020). InstructGPT uses reinforcement learning from human feedback (Christiano et al., 2017) (Stiennon et al., 2020) to fine-tune GPT-3 (Brown et al., 2020) to follow a broad class of writ- ten instructions. ChatGPT uses the same training methods as InstructGPT, but with slight differences in data collection setup. It can interact in the form of a conversational dialogue and provide human- like responses.
4.1.3 Adversarial Prompt with Knowledge Perturbation
1,836 NLP tasks that
addition the above useful prompt, we also design some adversarial prompts with knowledge perturbation. We aim to test how the model behaves when the given context not useful or against the knowledge stored in its internal weights. We con- sider three settings: (1) Partial Information Pro- vided (2) Masking Particular Information (3) Coun- terfactual Size Information.
Partial Information Provided In Exact Size In- formation Prompt, provide the exact size infor- mation of two entities the context. To investigate what extent the model would utilise the context information, instead giving two entity size infor- mation, we only provide one of them. This is to test whether the model could extract useful information from its internal weights to use together with the context information.
4.3 Evaluation
In this part, we describe the evaluation process and the evaluation metrics use.
4.3.1 Answer Mapping
Since we query the LLMs to generate the answer to the question, cannot be guaranteed that all the generated answers are exactly the labels. We use an answer mapping process to map the generated answer to the answer label. For general questions, the labels are yes tains 'yes" YYEE', we assume its predicted label is yes. If the predicted answer contains the nno/'NO', we assume that its predicted label is no. For special questions, the labels are entity names. We calcu- late the Levenshtein distance (Li and Liu, 2007) between the predicted entity and the two candidate entities, respectively. The Levenshtein distance a string metric for measuring the difference between two sequences, and smaller distance means the two strings more similar. We choose the can- didate entity with smaller Levenshtein distance the predicted label.
Masking Particular Information To further in- vestigate whether the exact size helps the model when answering size comparison questions, we manually mask important information in the con- text. For example, in Exact Size Information Prompt, we mask the exact size or entities, respec- tively, see the performance gap between using the masked prompt and the unmasked prompt. We replace the exact size of the entities with the mask token [MASK].
If the predicted answer con-
Counterfactual Size Information Instead of providing the true size information in the context, we replace it with the wrong size information to investigate what predictions the model would make when the extemal context knowledge contradicts the knowledge stored in its weights. If the model will fully utilise the counterfactual size information when answering the size comparison questions. In particular, we swap the size information the two entities in the prompt to mislead the model.
4.3.2 Metries
We consider four metrics our experiments. Ac- curacy and Macro-F1 scores are two commonly
https://coeeai..com/blooo

---

Prompt
FTS Small
FTS Base
FTS FTS--arge Large
FTS XL
GPT3 Davinci
GPT3.5 Turbo
Turbo
Metric
Acc
Macro F1
Acc
Macro-F Fl
Acc
Macro
Ace
Macro Fl
Acc
Macro
Acc
Macro F1
Question
0.58
0.69
0.69
General Knowledge Information
0.50
0,34
0.52
0.52
0.64
0.63
0.67
0.67
0.51
0.35
0.79
0.79
+ Size Knowledge Information
0.50
0.40
0.55
0.53
0.63
0.61
0.76
0.76
0.56
0.48
0.85
0.85
+ Exact Size Information
0.50
0,50
0,50
0.54
0.62
0.77
0.87
0.88
0.73
0.89
0.90
0.89
+ Only Entity Size
0.50
0.64
0.69
Only Tail Entity Gold Size
0.50
0,34
0.49
0.34
0.55
0.54
0.66
0.63
0.51
0.34
0.75
0.74
+ Masking Size Information
0.50
0.49
0.49
0.34
0.53
0.41
0.74
0.74
0.53
0.39
0.80
0.80
+ Masking Eniity Information
0.50
0.33
0.50
0.37
0.49
0.42
0.63
0.58
0.52
0.46
0.70
0.69
Counterfactual Size Information
0.50
0.47
0.33
0.47
0.23
0.23
0.42
0.38
0.38
Table 2: The results of using different prompt settings various models the general question of POSQA.
Results and Analysis
General Question
1.0
F15-Small
FTS Base
5.1 Baseline
FT5-Large
Context Misleading Rate
0.8
FTS-XL
GPT3 Davincl
The baseline to query LLMs using the Plain Question prompt and the results are presented Table for general questions and Table for spe- cial questions. When the number of parameters of the model exceeds 250M (Flan-T5-Base), the ability to answer the special size comparison ques- tions begins emerge. GPT3.5- -urbo achieves the best accuracy score and Macro-F1 score on both types of questions. It surprising that sig- nificantly smaller models, such as Flan-TS-Large (780M) and Flan-T5-XL (3B), exhibit superior per- formance. For example, Flan-T5-XL outperforms GPT3-Davinci (175B) by 0.06 precision in answer- ing general questions and 0.03 precision in answer- ing special questions. In general, LLMs perform better special questions. For example, the curacy increases by 0.14 on Flan-T5-XL. and 0.04 on GPT3.5-Tume respectively, from answering general questions to special questions.
0.4
0.0
0.0
Context Effective Rate
Figure 1: The results of CER and CMR using different knowledge-augmented prompts (except for Counterfac- tual Size Information) on various models on general question of POSQA.
used metrics for the evaluation of the performance of the model in classification tasks. Accuracy con- siders global precision and recall of the categories. while Macro-Fl computes the average of the Fl scores obtained by individual categories.
Our empirical investigation also reveals that the GPT3-Davinci model tends to provide an initial incorrect answer, despite subsequently offering a correct explanation for the given question. This phenomenon occurs especially when directly query- ing GPT3-Davinci with general questions get the "yes" or "no" answers. We speculate that it could be attributed to the use of first-word sampling techniques during the decoding process. However, this phenomenon does not occur in GPT3.5-Turbo which has been optimised for dialogue scenario. Even GPT3.5-Turbo, one of the most powerful LLMs, can only achieve an average 0.71 accuracy score these two types of questions. This sug- gests that although larger LMs may possess certain advantages most situations, they may still lack real-world understanding when it comes to answer- ing basic size comparison questions.
To explore the influence of context on the prompt, we propose two quantitative evaluation metrics: Context Effective Rate (CER) and Con- text Misleading Rate (CMR). We calculate CER and CMR by comparing the output of a model us- ing prompt that contains context information with the output of model using prompt that does not contain any context information. Specifically, CER evaluates how many incorrectly answered ques- tions can be correctly answered after adding the context in the prompt. CMR evaluates how many correctly answered questions can be incorrectly answered after adding the context to the prompt.

---

M
FTS Small
FTS Base
FTS FTS--arge Large
FTS XL
GPT3 Davinci
GPT3.5 Turbo
Gurb.
Prompti, Metric++
Acc
Macro F1
Acc
Macro-F Fl
Acc
Macro
Acc
Macro Fl
Acc
Macro
Acc
Macro F1
Question
0.51
0.49
0.50
0.63
0.72
0.71
0:73
Size Knowledge Information
0.52
0.50
0.55
0.53
0.74
0.73
0.79
0.78
0.76
0.76
0.83
0.83
+ General Knowledge Information
0.52
0.50
0.54
0.52
0.67
0.66
0.72
0,71
0.71
0.71
0.80
0.30
+ Exact Size Information
0.52
0,50
0,56
0.54
0.77
0.77
0.88
0.88
0.89
0.89
0.90
0.89
+ Only Entity Gold Size
0.50
0.74
Only Tail Entity Gold Size
0.51
0.49
0.52
0.51
0.64
0.63
0.77
0.76
0.69
0.68
0.76
0.76
+ Masking Size Information
0.52
0.49
0.51
0.49
0.68
0.68
0.76
0.76
0.67
0.67
0.64
0.63
+ Masking Emiit Information
0.51
0.47
0.54
0.52
0.71
0.69
0.84
0.84
0.84
0.83
0.82
0.82
Counterfactual Size Information
0.52
0.47
0.32
0.32
0.18
0.18
0.25
0.25
0.29
0.29
Table 3: The results of using different prompt settings various models on the special question of POSQA.
curacy score and the Macro-F1 score on special questions. Even for FT5-XL, can achieve 0.87 accuracy score and 0.88 Macro-Fl score on gen- eral questions and special questions, respectively, which is quite close to GPT3.5- Turbo with regard to the size comparison ability.
Special Question
1.0
FT5- F15-Small
FTS Base
FTS-Large
Context Misleading Rate
0.8
FT5-XL
GPT3 Davinci
GPTa.5-Turbo
In addition Gold Size Information, the gener- ated knowledge information and size information are also helpful to LLMs in answering the size com- parison question. The results also show that size knowledge is more effective than general knowl- edge as contextual information. Specifically, on GPT3.5-Turbo using size knowledge information can increase the accuracy and Macro-F1 scores by 0.06 general questions and 0.03 on special ques- tions from using general knowledge information. Although there is a significant improvement after providing LLMs with useful contextual informa- tion, the result still fall short of the human-level understanding of the real world, especially when adding the ground truth exact size information. Fur- thermore, the LLMs' performance in answering dif- ferent types of question has an obvious difference. despite the two types of questions being seman- tically equivalent for humans. This observation indicates that LLMs are more sensitive to question formats than humans. In summary, the findings suggest that supplementing original questions with additional information can enhance an LLM's s real- world understanding. However, even with this aug- mentation, LLMMs ability to achieve human-level understanding of the real world is still limited, and their sensitivity to question formats remains chal- lenge.
0.4
0.0
0.0
0.6
Context Effective Rate
Figure 2: The results of CER and CMR using different knowledge-augmented prompts (except for Counterfac- tual Size Information) on various models on special question of POSQA.
5.2 Prompts with Augmented Knowledge
The performance of using Relevant Knowledge- Enhanced Prompt as contextual information to query LLMs is shown in Table 2 and Table 3. Ac- cording to the presented results, augmenting the original size comparison questions with supplemen- tary knowledge about the mentioned objects can significantly enhance LLM'ss performance, sug- gesting that these models can effectively use con- textual information improve their real-world un- derstanding. In particular, as the model size scales beyond 780M (Flan-T5-Largee its ability to utilise prompt information increases greatly. Among these three Knowledge-augment augmented prompts, adding ex- act size information of the two compared objects in the context is the most effective way, which adheres the intuition. By using this prompt, 3P7 Turbo achieves 0.9 accuracy score on both general questions and special questions, which shows great ability utilising the exact size information in the context. GPT3-Davimm also obtains the 0.89 ac-
5.3 Context vs. Weights
By using the Adversarial Prompt with Knowledge Perturbation, we further explore the influence of the contextual information. The results in Table and Table provide valuable insights into how provid- ing additional information can help improve LLM''s

---

ability of LMs to utilise contextual information in the prompt is also enhanced. For example, the re- sults of G are scattered in the lower right of the diagram, which means that a powerful LLM should have high CER and low CMR.
Counterfactual Size Information Prompt
1.0
Question Type
Question
Context Misleading Rate
0.8
Special Question
When providing LLMs with counterfactual size information that is not helpful answering size comparison questions, accuracy and Macro-Fi scores decrease to a large extent. It should be noted that FT5-XL has the lowest accuracy score and Macro-F1 score on both types of questions, with only 0.23 and 0.18 respectively. Figure 3 illustrates the results of CER and CMR using the Counterfac- tual Size Information prompt various models for two types of questions. The CER is low all models, while the CMR increases as the scale of the models grows. shows a great ability to utilise counterfactual contextual information for larger LMs (e.g. GPT3-Davinci), which is revealed by the low CER and high CMR on general and special questions.
0.6
Model
0.4
FTS-Small
FTS-Base
FTS-Large
0.2
FTS-XL.
Divinni
GPT3.5-Turbo
0.0
0.0
0.2
0.6
0.8
1.0
Context Effective Rate
Figure 3: The results of CER and CMR using Counter- factual Size Information prompt on various models on two types of questions of POSQA.
real-world understanding. When we only provide LLMs with partial information, the performance drops significantly compared jointly providing information about both objects. For instance, on GPT3.5-Turbo the accuracy score decreases by 0.21 on general questions and by 0.16 on special questions when only providing the exact size infor- mation of the head entity. The results also reveal an imbalance in the utilisation of information, as the LMs tend to benefit more from extra information about the tail objects than the head objects.
The results also highlight the importance of con- textual information in LLMs' real-world under- standing, as providing counterfactual information significantly decreases their performance with high CMR in answering both general and special ques- tions. This indicates that the LLMs rely more on contextual information rather than their inter- nal weights learnt during pre-traie pre-training, which flects their in-context learning capabilities. In sum- mary, the findings provide valuable insights into the strengths and limitations of LLMs real-world understanding. They demonstrate that while pro- viding additional information can enhance LLMs' performance and their sensitivity to contextual in- formation. However, this also raises concerns about the robustness of LLMs' real-world understanding, as they may be easily induced perform harmful actions in real-world scenarios.
Masking either of the size or entity information would influence the performance of the LLMs. In- terestingly, masking size information decreases the performance more in answering the special ques- tions, while masking entity information decreases the performance more in answering the general questions. For example, on GPT3.5-Turbo, the accuracy score is 0.8 general questions while only 0.64 on special questions when masking size information. When masking entity information, GPT3.5-Turbo only gets a 0.7 accuracy score on general questions while 0.82 on special questions. Even when key information is masked, the con- text still provides some useful information which LLMs can utilise when answering on one type of question.
Discussion
LLMs' ability to understand the size of physical objects in the real world remains a challenge. Our experiments underscore that even the most advanced L.MSs at our disposal struggle to consis- tently grasp the sizes of physical objects. Specif- ically, GPT3.5-Turbo registers an average accu- racy score of 0.71 when directly addressing the two question types present in POSQA. This perfor- mance reveals a pronounced disparity compared to human comprehension of size, particularly when
Figure/ 1 and Figure/ 2 show the CER and CMR results using different knowledge-augmented prompts (except for Counterfactual Size Informa- tion) on various models on general and special ques- tions, respectively. As the scale of LMs grows, the

---

object size information is explicitly provided. Hu- mans, when confronted with size comparison tasks, often engage in mental simulations, drawing upon their accumulated knowledge to envisage the sizes objects (de Koning et al., 2017b). For a human, the ability to tackle size comparison effectively hinges possessing adequate size-related infor- mation about the objects in question. Similarly, LLMs should leverage the knowledge encoded in their internal weights to adeptly respond size comparison queries.
Krippendortf's Alpha
Human (Online)
Human (Offline)
Human (Online)
0.740
0.780
Human (Offline)
0.780
0,791
ChatGT
0.644
0.687
Table 4: The internal and mutual consistency among human annotators in different settings and ChatGPT is measured by Krippendorff's Alpha.
Aligmment of World Models To further investi- gate the alignment of LLM world models, ran- domly sampled 100 examples and annotated them with four human annotators in two settings: (1) Onl one Annotators are free to access any external or online sources of knowledge, and (2) Offline: Annotators are prohibited from relying on external resources during annotation. Surprisingly, online annotations lag behind offline annotations in terms of accuracy, with online accuracy at 0.86 and of- fline accuracy at 0.88. ChatGPT is slightly behind human performance, reaching an accuracy of 0.77. As shown Table 4, we compute the Krippendorff Alpha (Krippendorff. 2011) to assess the internal and mutual agreement between the online and of- fline annotators and ChatGPT. The scores indicate that ChatGPT is more consistent with human an- notators without accessing external knowledge source, which arouses curiosity that ChatGPT may share characteristics and bias similar that of hu- man intuition or fast thinking, which could be fur- ther investigated from the psycholinguistic point of view.
LLMs prefer utilise the information in the given context rather than knowledge stored in their internal weights. Our experiments demon- strate that giving useful information in prompt can enhance the performance of LLMs. For instance. GPT3.5-Turbo achieves 0.9 accuracy score on both two types of questions. However, LLMs cannot make good use of the external context information when they are only given partial information. is noteworthy that adding error information to the prompt largely decreases the performance of LLMs, For example, G can only get 0.38 Ac- curacy and Macro-Fl score on general questions and 0.29 Accuracy and Macro-F1 score on spe- cial questions. Even research (Shi et al., 2023) has shown that adding an instruction to ignore ir- relevant information brings performance gains; A single piece irrelevant information can distract the models and substantially degrade their perfor- mance. These results indicate that the context the prompt is extremely important for LLMS and that LLMs will utilise the information in the context.
Conclusion
We propose POSQA: Physical Object Size Question-Answering dataset with two types of size comparison questions to probe the ability of LLMs to understand the size of physical world objects. We design different settings to investigate the effect of the context in the prompt. Our experiments demonstrate that LLMs still fail to demonstrate robust understanding of the size of physical objects. The ability of LLMs to understand the size of physical objects in the real world remains a challenge for the future. The results also show that LLMs prefer to utilise the information in the given context rather than to use the knowledge stored in their internal weights. This also raises concerns about the robustness of LLMs' understanding ability to identify the useful and cor- rect contextual information in the prompt.
LLMs are sensitive to the format of the query, even if they are semantically equivalent. In our experiments, we query LLMs with different for- mat of size comparison questions from POSQA. The results show that LLMs are not robust enough when faced with the same semantics, but different forms of the queries. For example, on FT5-XL, achieves a 0.58 accuracy score on general questions versus a 0.72 accuracy score on special questions. Based on the behaviour of the LLMs on answering size comparison questions, it is not certain which forms of questions the models are better at solv- ing, and the performance is also influenced by the context added to the prompt. Although the perfor- mance gap between these two types of questions in GPT3.5-Turbo has narrowed, is still a noteworthy problem when training robust LLMs in the future.
auggeentd prompt

---

Acknowledgement
sharing within the research community, seeking develop LLMs that better engage with the world and benefit society. Continuous evaluation and im- provement are integral to our approach, and we welcome feedback from the research community and the wider public, as it contributes to the re- sponsible development and application of LLMs, aligning with principles of fairesss privacy protec- tion, transparency, and public benefit.
We grateful to acknowledge that the work of the joint first author, CS,, has been jointly supported by a donation from Toshiba Europe and the Engi- neering and Physical Sciences Research Council of UKRI (grant number 2752931).
Limitations
Our datasets comprise a modest total of 92 objects. Consequently, rather than serving as comprehen- sive evaluation toolkits that encapsulate the breadth of LLM world models, they may be best suited for probing or auditing LLM performance in real- world understanding, preferably in tandem with broader benchmarks. Additionally, the dataset only captures rudimentary relationships between two ob- peets-specificallly size comparisons. The incorpo- ration of more intricate interactions and dynamics among multiple objects might provide a deeper in- sight into the LLM world model. Moreover, due to resource constraints, our experiments were limited to Flan-T5, GPT3-Davinci, and GPT3.5-Turbo
References
Stephane
Cory Paik. Alessandro Ron- cone, and Katharina Kann. 2021. PROST: physical reasoning about objects through space and time. In Findings of the Association for Computational Lin- guistics: ACI/IJCNIP 2021, Online Event, August 1-6, 2021, volume ACL/IJCNLP 2021 of Findings of ACL, pages 4597-4608. Association for Computa- tional Linguistics.
Lawrence W Barsalou. 1999.
Perceptual symbol sys- tems. Behavioral and bruin sciences, 22(4):577-660.
Lawrence W Barsalou. 2008 Grounded cognition. Annu. Rev. Psychol., 59:617-645.
Ethics Statement
Irving Biederman and Eric E Cooper. 1992. Size invari- ance in visual object priming. Journal of Experimen- tal Psychology: Human Perception and Performance, 18(1):121.
The purpose of this research project is to evaluate the world understanding capabilities of Language Models (LLMs) by synthesizing new datasets from existing knowledge bases, with the aim of advanc- ing Natural Language Processing (NLP) research and improving LLM performance. We are com- mitted to conducting this research with the highest ethical standards, ensuring privacy and ethical con- siderations. No personally identifiable information or sensitive data is collected, stored, or processed, the datasets are solely derived from publicly available knowledge bases and are anonymized. We actively mitigate biases in the underlying data by carefully selecting and preprocessing the knowl- edge base. The datasets created will be used ex- clusively for evaluating LLM world understanding and will not be used for any commercial, discrim- inatory, or unethical purposes. We prioritize re- sponsible data usage, securely storing the data and making it accessible only to authorized researchers. Transparency and reproducibility are key, as document the dataset synthesis process for others reproduce and validate the results. Our research adheres to ethical guidelines, institutional policies, industry standards, and relevant regulations. Ad- ditionally, we foster collaboration and knowledge
Marcel Binz and Eric Schulz. 2022. Using cog- nitive psychology to understand GPT-3. CoRR. 14576.
Yonatan Bisk, Ari Holtzman, Jesse Thomason, Jacob Andreas, Yoshua Bengio, Joyce Chai, Mirella Lapata, Angeliki Lazaridou, Jonathan May, Aleksandr Nis- nevich, Nicolas Pinto, and Joseph Turian. 2020a. Experience grounds language. In Proceedings of the 2020 Conference Empirical Methods in Natural Language Processing, EMNLP 2020, Online, Novem- ber 16-20, 2020, pages 8718-8735. Association for Computational Linguistics.
Yonatan Bisk, Rowan Zellers, Ronan Le Bras, Jianfeng Gao, and Yejin Choi. 2020b. PIQA: reasoning about physical commonsense in natural language. In The Thirty- AAAI Conference on Artificial Intelli- gence, AAAI 2020, The Thirty-Secona Innovative Ap- plications of Artificial Intelligence Conference, IAAI 2020, The Tenth AAAI Symposium on Educational Advances in Artificial Intelligence, EAAI 2020, New York, NY, USA, February 7-12, 2020, pages 7432- 7439. AAAI Press.
Tom B. Brown, Benjamin Mann. Nick Ryder, Melanie Subbiah, Jared Kaplan, Prafulla Dhariwal, Arvind Neclakantan, Pranav Shyam, Girish Sastry, Amanda Askell, Sandhini Agarwal, Ariel Herbert-Voor Voss, Gretchen Krueger, Tom Henighan, Rewon Child,

---

Aditya Ramesh, Daniel M. Ziegler, Jeffrey Wu, Clemens Winter, Christopher Hesse, Mark Chen, Eric Sigler, Mateusz Litwin, Scott Gray, Benjamin Chess, Jack Clark, Christopher Bemer, Sam McCandlish, Alec Radford, Ilya Sutskever, and Dario Amodei. 2020. Language models are few-shot learners. In Ad- vances in Neural Information Processing Systems 33: Annual Conference on Neural Information Process- ing Systems 2020, NewrlPS 2020. December 6-12, 2020, virtual.
Martin H Fischer and Rolf A Zwaan. 2008 Embodied language: A review of the role of the motor system in language comprehension. Quarterly journal of experimental psychology, 616:)225--800
Roberto Bizzal-Brizuela and Eduardo C. Garrido- Merchán. 2023. Chatgpt is not all you need, A state of the art review of large generative Al models. CoRR, abs/2301.c0 0.6605...55....
Biyang Guo, Xin Zhang, Ziyuan Wang, Minqi Jiang. Jinran Nie, Yuxuan Ding, Jianwei Yue, and Yupeng Wu. 2023. How close is chatgpt to human experts? comparison corpus, evaluation, and detection. CoRR, abs/2301.07597.
Giovanni Buccino. Ivan Colagè, Nicola Gobbi, and Gior- gio Bonaccorso. 2016. Grounding meaning in expe- rience: A broad perspective on embodied language. Neuroscience & Biobehavioral Reviews, 69:69-78.
Paul F. Christiano, Jan Leike, Tom B. Brown, Miljan Martic, Shane Legg, and Dario Amodei. 2017. Deep reinforcement learning from human preferences. In Advances in Newral Information Processing Systems Neural Information Pro-
Thilo Hagendorff, Sarah Fabi, and Michal Kosin- ski. 2022. Machine intuition: Uncovering human- like inturtive decision-making in GPT-3.5. CoRR, abs/2212.052006
30: Annual Conference cessing Systems 2017, December 4-9, 2017, Long Beach, CA, USA, pages 4299-4307.
Shibo Hao, Yi Gu, Haodi Ma, Joshua Jiahua Hong. Zhen Wang. Daisy Zhe Wang, and Zhiting Hu. 2023. Reasoning with language model is planning with world model. CoRR, abs/2305. 14992.
Hyung Won Chung, Le Hou, Shayne Longpre, Barret Zoph, Yi Tay, William Fedus, Eric Li, Xuezhi Wang, Mostafa Dehghani, Siddhartha Brahma, Albert Web- son, Shixiang Shane Gu, Zhuyun Dai. Mirac Suz- gun, Xinyun Chen, Aakanksha Chowdhery, Sharan Narang, Gaurav Mishra, Adams Yu, Vincent Y. Zhao, Yanping Huang. Andrew M. Dai, Hongkun Yu, Slav Petrov, Ed H. Chi, Jeff Dean, Jacob Devlin, Adam Roberts, Denny Zhou, Quoc V. Le, and Jason Wei. 2022. Scaling instruction-finttiee CoRR, abs/2210.1 11416.
Dan Hendrycks, Collin Burns, Steven Basart, Andy Zou, Mantas Mazeika, Dawn Song, and Jacob Stein- hardt. 2021. Measuring massive multitask language understanding. In 9th International Conference Learning Representations, ICIR 2021, Virtual Event, Austria, May 3-7, 2021. OpenReview.net.
language models.
Oleksandr V Horchak, Jean-Christophe Giger, Maria Cabral, and Grzegorz Pochwatko. 2014. From demonstration to theory in embodied language com- prehension: a review. Cognitive Systems Research, 29:66-85.
NAC Cressie and HJ Whitford. 1986. How to use the two sample t-test. Biometrical Journal, 28(2):131-
148.
Wenxiang Jiao, Wenxuan Wang, Jen-tse Huang, Xing Wang, and Zhaopeng Tu. 2023. chatgpt A good translator? A preliminary study. CoRR, abs/2301.08745.
Ishita Dasgupta. Christine Kleser-Chenn Kenneth Marino, Arun Ahuja, Sheila Babayan, Felix Hill, and Rob Fergus. 2023. Collaborating with language mod- els for embodied reasoning. CoRR, abs/2302.00763.
Takeshi Kojima, Shixiang Shane Gu, Machel Reid, Yu- taka Matsuo, and Yusuke Twasawa. 2022. Large language models are zero-shot reasoners. CoRR, abs/2205.11916.
Björ B de Koning, Stephanie I Wassenburg, Lisanne T Bos, and Menno Van der Schoot. 201 Size does matter: Implied object size is mentally simulated dur- ing language comprehension. Discourse processes, 54(7):493-503.
Klaus Krippendorff 2011. Computing krippendorff's relaabrelibbiiit..
Björ B. de Koning, Stephanie I. Wassenburg. Lisanne T. Bos, and Menno van der Schoot. 2017b. Size does matter: Implied object size is mentally simulated dur- ing language comprehension. Discourse Processes,
Kay Lehnert. 2023. AI insights into theoretical physics and the swampland program: A journey through the cosmos with chatgpt. CoRR, abs/2301.08 155.
54:493 503.
Yujian Li and Liu. 2007. normalized levenshtein distance metric. IEEE Trans. Pattern Anal. Mach. Intell., (9: 1091-0099999
Danny Driess, Fei Xia, Mehdi S. M. Sajjadi. Corey Lynch, Aakanksha Chowdhery, Brian Ichter. Ayzaan Wahid, Jonathan Tompson, Quan Vuong, Tianhe Yu. Wenlong Huang, Yevgen Chebotar, Pierre Ser- manet, Daniel Duckworth, Sergey Levine, Vincent Vanhoucke, Karol Hausman, Mare Toussaint, Klaus Greff, Andy Zeng, Igor Mordatch, and Pete Florence. 2023. Palm-e: An embodied multimodal language model. CoRR, htt/2333..3377.
Jiacheng Liu, Alisa Liu, Ximing Lu, Sean Welleck, Pe- ter West, Ronan Le Bras, Yejin Choi, and Hannaneh Hajishirzi. 2022a. Generated knowledge prompting for commonsense reasoning. In Proceedings of the 60th Annual Meeting of the Association for Compu- national Linguistics (Volume 1: Long Papers), ACL

---

2022, Dublin, Ireland, May 22-27. 2022, pages 3154- 3169. Association for Computational Linguistics.
Nisan Stiennon, Long Ouyang, Jeff Wu, Daniel M. Ziegler, Ryan Lowe, Chelsea Voss, Alec Radford. Dario Amodei, and Paul F. Christiano. 2020. Learn- ing summarize from human feedback. CoRR, abs/2009.01325.
Ruibo Liu, Jason Wei, Shixiang Shane Gu, Te-Yen Wu, Soroush Vosoughi, Claire Cui, Denny Zhou, and An- drew M. Dai. 2022b. Mind's eye: Grounded lan-
guage model reasoning through simulation. CoRR, abs/2210.05359.
Sai Vemprala, Rogerio Bonatti, Arthur Bucker, and Ashish Kapoor. 2023. Chatgpt for roboties: Design principles and model abilities.
Kyle Mahowald, Anna A. Ivanova, Idan Asher Blank, Nancy Kanwisher, Joshua B. Tenenbaum, and Evelina Fedorenko. 2023. Dissociating language and thought in large language models: cognitive perspective. CoRR, 0ass22301.06622
Jason Wei, Maarten Bosma, Vincent Y. Zhao, Kelvin Guu, Adams Wei Yu, Brian Lester, Nan Du, An- drew M. Dai, and Quoc V. Le. 2022a. Finetuned language models are zero-shot learners. In The Tenth International Conference on Learning Representa- tions, ICLR 2022, Virtual Event, April 25-29, 2022. OpenReview.net.
Oded Nov, Nina Singh, and Devin M. Mann. 2023. Putting chatgpt's medical advice to the (turing) test. CoRR, abs/2301. 10035.
Long Ouyang. Jeff Wu, Xu Jiang, Diogo Almeida, Car- roll L. Wainwright, Pamela Mishkin, Chong Zhang. Sandhini Agarwal, Katarina Slama, Alex Ray, John Schulman, Jacob Hilton, Fraser Kelton, Luke Miller, Maddic Simens, Amanda Askell, Peter Welinder, Paul F. Christiano, Jan Leike, and Ryan Lowe. 2022. Training language models to follow instructions with human feedback. CoRR, Abs/2203.02155.
Jason Wei, Xuezhi Wang. Dale Schuurmans, Maarten Bosma, Ed H. Chi, Quoe Le, and Denny Zhou. 2022b. Chain of thought prompting elicits reasoning in large language models. CORR, abs/2201.11903.
Jiannan Xiang, Tianhua Tao, Yi Gu, Tianmin Shu, Zirui Wang, Zichao Yang, and Zhiting Hu. 2023. Language models meet world models: Embodied experiences enhance language models. CoRR, abs/2305.10626.
Ben Prystawski, Paul H. Thibodeau, and Noah D. Goodman. 2022. seychologically-inoonne informed chain-
A Appendix
of-thought prompts for metaphor understanding in
large language models. CoRR, abs/2209.00 .01141.
A.1 Error Analysis
Colin Raffel, Noam Shazeer, Adam Roberts, Katherine Lee, Sharan Narang, Michael Matena, Yanqi Zhou, Wei Li, and Peter J. Liu. 2020. Exploring the limits of transfer learning with a unified text-to-text trans- former. J. Mach. Learn. Res., 21:140:1-140:61
Empirically, we realised that the error counts vary largely regarding different entity mentions in the prompts. According to Figure 4, the AIDS virus the most challenging entity for LLMs to distinguish the physical size relationship with other objects, compared to that of Mars, which is the casiest. Furthermore, as shown in Figure 5, it is more difficult for LLMs to correctly compare two objects with similar size magnitude.
Victor Sanh, Albert Webson, Colin Raffel, Stephen H. Bach, Lintang Sutawika, Zaid Alyafeai, Antoine Chaffin, Arnaud Stiegler, Arun Raja, Manan Dey, M Saiful Bari. Canwen Xu, Urmish Thakker, Shanya Sharma Sharma, Eliza Szezechla, Taewoon Kim. Gunjan Chhablani. Nihal V. Nayak. Debajyoti Datta, Jonathan Chang, Mike Tian-Jian Jiang, Han Wang, Matteo Manica, Sheng Shen, Zheng Xin Yong, Harshit Pandey, Rachel Bawden, Thomas Wang, Tr- ishala Neeraj. Jos Rozen, Abheesht Sharma, An- drea Santilli, Thibault Févry, Jason Alan Fries, Ryan Teehan, Teven Le Scao, Stella Biderman, Ieo Gao, Thomas Wolf, and Alexander M. Rush. 2022. Multi- task prompted training enables zero-shor shot task gener- alization. In The Tenth International Conference on Learning Representations, ICLR 2022, Virtual Event, April 25-29, 2022. OpenReview.net.
A.2 GPT3-Davinci using Chain-of-Thougee
Existing research (Kojima et al., 2022) et al., 2022b) has shown that the use of CoT could sig- nificantly improve the performance of LLMs. We randomly sample 100 test cases (50 questions for each type) and report the results using Zero-Shot- CoT (Kojima et al., 2022) on GPT3-Davinci in Table 5. The results show that after using CoT, ac- curacy and Macro-F1 scores are notably improved across all settings both types of questions.
Freda Shi, Xinyun Chen, Kanishka Misra, Nathan Scales, David Dohan, Ed H. Chi, Nathanael Schärli, and Denny Zhou, 2023. Large language models can be easily distracted by irrelevant context. CoRR, abs/2302.00093.
A.3 GPT3-Davinci using Different Temperature Parameters
To investigate the effect of different temperature parameters, we performed experiments three dif- ferent temperatures (0, 0.5, 1) in 100 randomly sam- pled test cases (50 questions for each type). This
Dominik Sobania, Martin Briesch, Carol Hanna, and Justyna Petke. 2023. An analysis of the auto- matic bug fixing performance of chatgpt. CoRR, 108653..

---

the wwprrcaaooosseoo
Ho 2010 REA
biyy
Figure 4: The number of mistakes related to each entity.
Question
Prompt
Accuracy
Macro-F1
Accuracy Macro-F1
w/o CoT
with CoT
General
Plain Question
0.52
0.38
0.62
0.62
+ Gold Size Information
0.78
0.78
0.88
0.88
+ General Knowledge Information
0.50
0.33
0.66
0.65
Size Knowledge Information
0.62
0.56
0.76
0.76
Special
Plain Question
0.84
0.83
0.90
0.87
+
Gold Size Information
0.94
0.93
0.94
0,93
+ General Knowledge Information
0.84
0.81
0.92
0.91
+ Size Knowledge Information
0.88
0.86
0.94
0.93
Table The results of GPT3-Davinci using Chain-of-Thought (CoT) in different prompts on 100 randomly chosen test cases (50 questions for each type).
result is shown in Table 6. The results show that setting the temperature to 0 can give the best results. To alleviate the randomness effect of temperature, in all our experiments, we set the temperature as 0, that the LLMs become deterministic.
context is significant when choosing the prompt.
A.5 Statistical Significance
We assess the statistical significance of the differ- ences or relationships observed in our experiments using two-sample t-test (Cressie and Whitford, 1986). The results in Table show that the p-value (p 0.05) is sufficiently low to reject the null hy- pothesis, indicating that there is significant differ- ence between the various experiment settings with GPT3. on the POSQA data set.
A.4 GPT3-Davinci using Gold Size Information Prompt Templates
Since LLMs is sensitive to the format of the prompt, we use different prompt templates to probe their in- fluence. We mainly tested the gold size information prompt template and we list all templates in Table 7. We conducted experiments on the same 100 ran- domly sampled test cases (50 questions for each type), and the result is shown in Table 8. The effect of using different templates is notable in the general question, and using Template has the best per- formance. Interestingly, Template 4 decreases the performance on both general and special questions, which means that the order of the question and

---

Question
Prompt
Accuracy
Macro-F1
Accuracy
Macro-F1
Accuracy
Macro-F1
Temperature
0.5
General
Plain Question
0.52
0.38
0.52
0.38
0.52
0.38
+ Gold Size
0.78
0.78
0.80
0.80
0.78
0.78
General Knowledge
0.50
0.33
0.50
0.33
0.50
0.33
Size Knowledge
0.62
0,56
0.62
0.56
0.60
0,52
Special
Plain Question
0.84
0.83
0.82
0.80
0.82
0.81
Gold Size
0.94
0.93
0.90
0.90
0,92
0.92
+ General Knowledge
0.84
0.81
0.86
0.83
0.82
0.78
Size Knowledge
0.88
0.86
0.82
0.78
0.80
0.75
Table 6: The results of GPT3-Davinci using different temperatures in different prompts on 100 randomly chosen test cases (50 questions for each type).
Template 1:
The size of Entity A is Exact Size of Entity A. The size of Entity B is Exact Size of Entity B. Question Entity Exact Size of Entity A; Entity Exact Size of Entity B. Question.
Template 2:
Template 3:
Entity A is Exact Size of Entity A and Entity B is Exact Size of Entity B. 4. Question
Template 4:
Question The size Entity A Exact Size of Emtity The size of Entity is Exact Size of Entity B.
Table 7: Different gold size information prompt templates used in Table 8.
Question
Prompt
Accuracy
Macro-Fi
General
Plain Question
0.52
0.38
Gold Size Information using Template 1
0.78
0.78
Gold Size Information using Template 2
0.82
0.82
Gold Size Information using Template 3
0.88
0.88
Gold Size Information using Template 4
0.50
0.37
Special
Plain Question
0.84
0.83
Gold Size Information using Template 1
0.94
0.93
Gold Size Information using Template 2
0.94
0.95
Gold Size Information using Template 3
0.90
0.89
Gold Size Information using Template 4
0.86
0.84
Table 8: The results of GPT3-Davinci using different gold size information prompt templates on 100 randomly chosen lest cases (50 questions for each type).

---

Count of Errors of Magnitude Differences
250
200
Count
150
100
50
37
35
40
33
34
30
36
32
27
28
31
26
29
23
25
19
21
24
20
22
18
17
16
14
15
13
12
11
10
8
Magnitude
Figure The number of mistakes related to the magnitude difference between entities.


Reviews:
[{"review_id": "GzvOg1UIiz", "reviewer": "Reviewer_gnhd", "summary": "", "questions": "Give more details about the human annotation process used for the dataset.", "limitations": "", "rating": 4, "confidence": 3, "soundness": 4, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We extend our sincere gratitude to the reviewer for conducting a comprehensive evaluation of our paper. We highly value the opportunity to address the reviewer's insights and provide further insights into our research. Below, we address the concerns raised by the reviewer and provide additional clarifications:\n\n**Reasons to Address the Concerns Raised by the Reviewer:**\n\n- **Scope of Real-World Understanding:** We acknowledge the reviewer's concern that our study's focus on size comparison represents only a facet of real-world understanding. We concur that real-world understanding is multifaceted, and our work serves as an exploratory step into a specific aspect. We intend to clearly contextualise this aspect within the broader understanding of language models' capabilities in both our introduction and conclusion.\n\n- **Dataset Accessibility:** We sincerely apologise for the inaccessibility of the dataset link in the original submission. To address this, we are delighted to provide access to the [preview version of POSQA](https://anonymous.4open.science/r/POSQA-EF54/datasets). Upon acceptance, we commit to releasing the dataset on the Hugging Face Datasets platform to facilitate widespread usage and validation.\n\n- **Human Annotation Process:** We greatly appreciate the reviewer's interest in our human annotation process for ChatGPT comparison. We are pleased to provide detailed information about the [online](https://anonymous.4open.science/r/POSQA-EF54/annotation_online) and [offline](https://anonymous.4open.science/r/POSQA-EF54/annotation_offline) annotations, including text excerpts, labelled categories, and annotation durations. Our annotators, all with master's degrees and advanced English proficiency, utilised the [Potato Platform](https://potato-annotation-tutorial.readthedocs.io/en/latest/) for annotation. In our final version, we will comprehensively outline the annotation process in the appendix.\n\n**Addressing Typos, Grammar, and Presentation:**\n\nWe express our gratitude to the reviewer for their meticulous attention to detail. We will diligently rectify the typographical errors mentioned, and aim to augment the clarity\n\n**Reproducibility and Transparency:**\n\nWe acknowledge the reviewer's concern regarding the reproducibility of our results. In response, our revised submission will provide an in-depth exposition of our methodology, experimental design, and evaluation metrics. Moreover, we will offer a comprehensive overview of the dataset's structure and components, ensuring accessibility for external researchers to comprehend and replicate our approach.\n\n**Future Directions and Contributions:**\n\nWe deeply value the reviewer's recognition of our study's contribution to comprehending LLMs' real-world understanding. We concur with the reviewer that our findings open avenues for investigating the intricate interplay between context and model weights. In our final version, we will elaborate on potential research directions within this domain, fostering further exploration and collaboration within the research community.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "1. Interesting, simple, and focused (size comparison) approach to probe real-world understanding in LLMs. The POSQA dataset is the main contribution of the paper.\n2. The paper lays out the methodology and evaluation metrics in detail to systematically test the LLMs.\n3. The results are relevant and interesting. They show that LLMs are still far from humans in real-world understanding. Moreover, model’s preference between context and weights is an important finding and can potentially lead to interesting future investigations.\n", "reasons_to_reject": "1. The results might not be very general for actual “real-world understanding” of LLMs. Size comparison is still just one part of the bigger picture about real-world understanding.\n2. The link to the dataset is not active. So, the claims made about the dataset can’t be verified. I would request the authors to look into it.\n3. No details about the human annotation process used for comparison with ChatGPT.", "questions_to_authors": "Give more details about the human annotation process used for the dataset.", "paper_topic_and_main_contributions": "The paper proposes a dataset (POSQA) to probe the real-world understanding of LLMs using size comparison questions. Authors show that LLMs struggle in zero-shot scenarios. They further investigate whether this understanding relies upon the context in the prompt or the internal weights of the model. They also study the impact of prompt formats and biases toward different objects. Their results indicate that LLMs struggle with the real-world understanding of different object sizes. Moreover, they have a preference for using the information in context over the knowledge stored in the weights. The results also suggest that LLMs' real-world understanding can be underestimated or overestimated depending on the prompt structure, affecting alignment with human behavior.", "excitement": 4, "reproducibility": 2, "ethical_concerns": "No", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"Typos_Grammar_Style_and_Presentation_Improvements": "Line 392 - \"It is surprised\"\nLine 89 - \"our contributes can\""}, "extra_text": {}}, {"review_id": "rGWB7fkvvT", "reviewer": "Reviewer_GsC7", "summary": "", "questions": "A: I am mostly interested in missing analysis I mention in the Reasons to Reject. Shedding light on these topics would be appreciated.\n\nB: Does other human response data exist on this topic (i.e., from other studies)? Judging by the ample psycholinguistic backing for size comparison studies that is provided, one might expect there are existing statistics that may be informative to the reader.", "limitations": "", "rating": 4, "confidence": 3, "soundness": 4, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We extend our sincere gratitude for the dedicated time and meticulous evaluation provided by the reviewer. The insightful feedback is greatly valued, and we are eager to address the reviewer's points while elaborating on our approach and intentions:\n\n**Addressing the Concerns:**\n\n- **Sensitivity Analysis:** We agree with you that temperature is an important factor in the generation of LLMs. To alleviate the randomness effect of temperature, in all our experiments, we set the temperature as 0, so that the LLMs become deterministic. We agree that conducting the prompt and parameter tuning on a smaller subset will be beneficial, and we would like to provide the details about our existing tuning results here:\n\n1. **GPT3-Davinci using Different Temperature Parameters** To investigate the effect of different temperature parameters, we conducted experiments on three different temperatures (0, 0.5, 1) on 100 randomly sampled test cases (50 questions for each type). The results show that setting the temperature to 0 can give the best results.\n\n| **Question** | **Prompt**            | **Accuracy** | **Macro-F1** | **Accuracy** | **Macro-F1** | **Accuracy** | **Macro-F1** |\n|--------------|-----------------------|--------------|--------------|--------------|--------------|--------------|--------------|\n| Temperature  |                       | **0**        |              | **0.5**      |              | **1**        |              |\n| General      | Plain Question        | 0.52         | 0.38         | 0.52         | 0.38         | 0.52         | 0.38         |\n|              | + Gold Size           | 0.78         | 0.78         | 0.80         | 0.80         | 0.78         | 0.78         |\n|              | + General Knowledge   | 0.50         | 0.33         | 0.50         | 0.33         | 0.50         | 0.33         |\n|              | + Size Knowledge      | 0.62         | 0.56         | 0.62         | 0.56         | 0.60         | 0.52         |\n| Special      | Plain Question        | 0.84         | 0.83         | 0.82         | 0.80         | 0.82         | 0.81         |\n|              | + Gold Size           | 0.94         | 0.93         | 0.90         | 0.90         | 0.92         | 0.92         |\n|              | + General Knowledge   | 0.84         | 0.81         | 0.86         | 0.83         | 0.82         | 0.78         |\n|              | + Size Knowledge      | 0.88         | 0.86         | 0.82         | 0.78         | 0.80         | 0.75         |\n\n**Table:** The results of GPT3-Davinci using different temperatures in different prompts on 100 randomly chosen test cases (50 questions for each type).\n\n2. **GPT3-Davinci using Gold Size Information Prompt Templates** Since LLMs are sensitive to the format of the prompt, we use different prompt templates to probe their influence. We mainly test the gold size information prompt template and we list all templates in the Table.\n| **Template 1:** | The size of *Entity A* is *Exact Size of Entity A*. The size of *Entity B* is *Exact Size of Entity B*. + *Question* |\n|-----------------|--------------------------------------------------------|\n| **Template 2:** | *Entity A*: *Exact Size of Entity A*; *Entity B*: *Exact Size of Entity B*. + *Question*.  |\n| **Template 3:** | *Entity A* is *Exact Size of Entity A* and *Entity B* is *Exact Size of Entity B*. + *Question* |\n| **Template 4:** | *Question* + The size of *Entity A* is *Exact Size of Entity A*. The size of *Entity B* is *Exact Size of Entity B*. |\n\nWe conduct experiments on the same 100 randomly sampled test cases (50 questions for each type). The effect of using different templates is notable on general questions and using Template 3 in the Table above has the best performance. Interestingly, Template 4 decreases the performance on both general and special questions, which means the order of the question and context is significant when choosing the prompt.\n\n| **Question** | **Prompt** | **Accuracy** | **Macro-F1** |\n|--------------|------------|--------------|--------------|\n| General      | Plain Question | 0.52 | 0.38 |\n|              | + Gold Size Information using Template 1 | 0.78 | 0.78 |\n|              | + Gold Size Information using Template 2 | 0.82 | 0.82 |\n|              | + Gold Size Information using Template 3 | 0.88 | 0.88 |\n|              | + Gold Size Information using Template 4 | 0.50 | 0.37 |\n| Special      | Plain Question | 0.84 | 0.83 |\n|              | + Gold Size Information using Template 1 | 0.94 | 0.93 |\n|              | + Gold Size Information using Template 2 | 0.94 | 0.95 |\n|              | + Gold Size Information using Template 3 | 0.90 | 0.89 |\n|              | + Gold Size Information using Template 4 | 0.86 | 0.84 |\n\n- **Answer Mapping Sensitivity:** We appreciate the reviewer's recommendation to explore the sensitivity of answer mapping, specifically in the context of AI safety mechanisms. We also find the intersection of AI safety and the physical-world understanding of LLMs to be a promising direction for our subsequent research.\n\n- **Statistical Significance and Overstatement:** We acknowledge the reviewer's valid concern regarding the necessity of conducting an extensive statistical significance analysis to reinforce the robustness of our findings. We will present the comprehensive significance test results for all of our key claims in the appendix of the final version.\n\n\n**Variability of Human Annotators:** While we acknowledge that variability of <20% could arise due to varying educational backgrounds among human annotators, our primary aim is to emphasise the comparison of object sizes as a foundational and direct task that mirrors LLMs’ grasp of the tangible world. Discrepancies in this aspect can potentially have significant implications in future interactions between Embodied LLMs and the physical reality. Thus, we emphasise the importance of recognizing the existing gap between human and LLM comprehension of the physical world, urging a thorough assessment of the present state of LLMs.\n\n**Responding to Questions:**\n\n- **Question A:** We deeply value the reviewer's attention to the analysis gap highlighted in the \"Reasons to Reject'' section. We will incorporate the provided clarifications into the final version of the paper. \n- **Question B:** The reviewer's inquiry into the availability of human response data is well-noted. We also recognize the potential value in incorporating statistics from relevant studies.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "The dataset is interesting, new, and well-designed. On the last point, its unique topic fills an apparent gap in the literature, allowing easily interpretable experimental design for the paper's main research questions on in-context learning. The takeaways are mostly clear, not well-known, and interesting. Writing is mostly clear.", "reasons_to_reject": "**Main Point**\nSensitivity analysis on prompts and parameters is not provided. For instance, temperature can be an important factor in generating factual responses (as in the studied case). Similarly, LLMs can be sensitive to prompts in a way that is decidedly not human-like. Experimenting with a variety of different prompt formats / parameters and testing for consistency is an important, missing analysis. Doing so on the entire dataset, with all LLMs, and all prompts would be challenging - but it is reasonable to check these things on a smaller subset.\n\n**Similar Minor Points**\n - Similarly to the above, but a more minor point, some sensitivity analysis on the answer mapping would be nice. For instance, your error analysis in the appendix indicates trouble with the \"AIDS virus\" - this may be caused by AI safety features. \n\n - There appears to be a general lack of analysis on statistical significance, which can be important when interpreting many results. This may cause the authors/readers to draw false positive inferences. I do consider this a minor point, since most takeaways are not inferred by \"close calls\" - most discussed differences in accuracy, for instance, are substantial.\n\n - Some results may be \"over stated\", for instance, the gap between ChatGPT performance and average human performance is < 20%. Arguably, this is the type of variability one might expect from humans of varying experiences/education levels.\n\n*Post-rebuttal Revisions*\nThe main points are addressed by authors during the rebuttal, and authors have agreed to include these in the paper (Appendix with pointers in the main text is fine).", "questions_to_authors": "A: I am mostly interested in missing analysis I mention in the Reasons to Reject. Shedding light on these topics would be appreciated.\n\nB: Does other human response data exist on this topic (i.e., from other studies)? Judging by the ample psycholinguistic backing for size comparison studies that is provided, one might expect there are existing statistics that may be informative to the reader.", "paper_topic_and_main_contributions": "The authors provide a new dataset for measuring LLM world knowledge, particularly about the comparative size of objects. The new dataset caters to important research questions surrounding in-context learning of LLMs. In particular, the controlled topic facilities provision of *no, partial, full*, and *incorrect* information in the LLM prompt. Results on a wide variety of LLMs (small, large, and commercial) provide interesting insights such as on gaps between human and LLM performance and alignment of human and LLM reasoning processes.", "excitement": 4, "reproducibility": 3, "ethical_concerns": "No", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"Missing_References": "None notable.", "Typos_Grammar_Style_and_Presentation_Improvements": "Typo in opening line 042.\n\nSpecific examples of prompts should be provided, this is not available in all cases."}, "extra_text": {}}, {"review_id": "rq93pabyw8", "reviewer": "Reviewer_266w", "summary": "", "questions": "Question A: The linked website for the data (Footnote 1, www.posqa.data) is unreachable. Is this on purpose / will it be available after acceptance?\n\nQuestion B: In line 214, you describe the entity features as \"scale, size[, magnitude, and text]\". I find it difficult to truly distinguish between the first two, as you mention that \"the scale of the Solar System is 9 billion km\", which I would intuitively label as its size rather than simply scale. Could you perhaps add an example entry of the entities dataset that showcases all four features for that entity?\n\nQuestion C: Which GPT3 model are you using to generate the knowledge information in Section 4.1.2 (lines 246/269/274)? And similarly, when referring to InstructGPT and ChatGPT (lines 326-328), it would be nice to mention the exact name of each model in the API for further reproducibility.\n\nQuestion D: What does it mean for models to have (near) 1 CER & CMR in Figures 1 and 3? I find these data points abnormal without being addressed in the text.\n\nQuestion E: In line 410 you mention the use of \"first-word sampling\" techniques during decoding. What is this referring to? I don't see how this differs between InstructGPT and ChatGPT.\n\nQuestion F: In lines 622-626, the finding that annotators with access to online resources perform worse on the task is remarkably counter-intuitive. Do you have any explanation on why this may be the case?", "limitations": "", "rating": 3, "confidence": 3, "soundness": 3, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We extend our heartfelt appreciation for the insightful assessment of our paper. We value your valuable feedback and insights, and would like to address your observations and queries while providing further clarification on our research:\n\n**Addressing the Concerns:**\n\n- **Alignment of LLM Knowledge with Human Comprehension:** Exploring a large-scale alignment study between humans and LLMs is indeed an intriguing avenue, contingent upon the availability of resources within an academic context. Our current work, as showcased here, contributes valuable insights into comprehending the limitations of LLMs, delving into diverse analytical facets. It serves as a valuable resource that can potentially pave the way in this particular direction. \n\n- **Grammatical Errors and Clarity:** We are grateful for your suggestions and will fix the mentioned presentation issues and proofread the final version.\n\n**Responding to Questions:**\n\n- **Question A:** We express our gratitude for bringing the data link issue to our attention. In response, we are pleased to provide access to the [preview version of POSQA](https://anonymous.4open.science/r/POSQA-EF54/datasets). Upon acceptance, we commit to releasing the dataset on the Hugging Face Datasets platform, thereby facilitating extensive usage and validation.\n\n- **Question B:** We value your inquiry about distinguishing between \"scale\" and \"size\" in entity features. Below is an illustrative example of the entity \"Solar System.\" We will also incorporate this example within our paper:\n\n```json\n{\n  \"scale\": \"9 billion km\",\n  \"size\": \"9×10^12\",\n  \"magnitude\": \"12\",\n  \"name\": \"Solar System\",\n  \"text\": \"The Solar System consists of the central Sun, eight planets (Mercury, Venus, Earth, Mars, Jupiter, Saturn, Uranus and Neptune), dwarf planets including Pluto and smaller celestial bodies. The distance from the Sun to Neptune, the farthest planet from the Sun, is about 4.5 billion km. The Solar System was formed about 4.6 billion years ago when interstellar clouds, which were floating in space, were pulled together by gravity and began to rotate, pulling mass into the centre to form a primordial solar nebula. The other planets are believed to have formed from the gas and dust which remained in the surrounding area.\"\n}\n```\n\n- **Question C:** Thank you for your suggestion. The InstructGPT(GPT3) we employed is text-davinci-003, and the ChatGPT variant we utilized is gpt-3.5-turbo. We will provide clarification on this matter within our paper.\n\n- **Question D:** In essence, CER evaluates how many incorrectly answered questions can be corrected after adding context to the prompt. CMR, on the other hand, assesses how many correctly answered questions can turn incorrect upon incorporating context. When CER & CMR values approach 1, it signifies that all accurately answered questions are susceptible to being incorrectly answered due to prompt context, and vice versa. These measures expose the model's susceptibility to contextual changes and its limited robustness. We will elaborate on this concept in the final version of the paper.\n\n- **Question E:** When referring to \"first-word sampling,\" we intend to convey that the model tends to generate initial Yes/No responses, followed by elaboration through the reasoning process. This strategic approach prevents autoregressive models from influencing their yes/no predictions based on the reasoning process, thus preserving performance. We will rephrase this concept more explicitly in the upcoming version. Additionally, we would like to highlight the paper titled, [How Language Model Hallucinations Can Snowball\n](https://arxiv.org/abs/2305.13534), which extensively delves into this phenomenon, offering an in-depth exploration of its intricacies. \n\n- **Question F:**  Drawing from our annotation process, we noted that even when provided with the capability to perform web searches, most human annotators refrain from extensive searches for specific volume parameters of objects. Instead, they depend on fragmented information to build foundational understanding. For instance, they might deduce that 'pollen' is a type of cell without necessarily looking up its exact size. Alternatively, they may search for images of the object, which may lack information about relative sizes. Hence, while external information enhances comprehension, it does not necessarily enable accurate mental comparison of object sizes. \n\n**Missing References:**\n\nWe value your attention to related work and acknowledge the prior contribution by Hessam et al. We will certainly incorporate the reference and provide a discussion of their study within our revised submission to ensure due recognition of similar research in the domain.\n\n**Improvements and Corrections:**\n\nWe extend our sincere gratitude for your meticulous review of grammatical, typographical, and presentational aspects.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "The paper presents a novel dataset aimed at evaluating the parametric knowledge of Large Language Models on a specific type of questions that has not been extensively explored in previous research. The probing setup utilized in the study is well-structured and methodologically sound. Furthermore, the conclusions drawn from the experimental results are well-supported and credible.", "reasons_to_reject": "The paper mentions the study of the alignment of LLM knowledge with human comprehension of the world as one of its primary contributions. However, this aspect is only briefly addressed in the discussion section, where a small-scale experiment is conducted. Unfortunately, the analysis of this experiment is limited, as it solely compares ChatGPT's performance with that of human annotators.\n\nFurthermore, the paper contains several grammatical errors that may impede comprehension, particularly for non-native English speakers. To enhance the paper's clarity, I suggest addressing the listed grammatical errors in the Presentation Improvements field and conducting a more thorough analysis of the experiment comparing LLMs' knowledge to human understanding. This could provide deeper insights into the alignment or disparities between the two.", "questions_to_authors": "Question A: The linked website for the data (Footnote 1, www.posqa.data) is unreachable. Is this on purpose / will it be available after acceptance?\n\nQuestion B: In line 214, you describe the entity features as \"scale, size[, magnitude, and text]\". I find it difficult to truly distinguish between the first two, as you mention that \"the scale of the Solar System is 9 billion km\", which I would intuitively label as its size rather than simply scale. Could you perhaps add an example entry of the entities dataset that showcases all four features for that entity?\n\nQuestion C: Which GPT3 model are you using to generate the knowledge information in Section 4.1.2 (lines 246/269/274)? And similarly, when referring to InstructGPT and ChatGPT (lines 326-328), it would be nice to mention the exact name of each model in the API for further reproducibility.\n\nQuestion D: What does it mean for models to have (near) 1 CER & CMR in Figures 1 and 3? I find these data points abnormal without being addressed in the text.\n\nQuestion E: In line 410 you mention the use of \"first-word sampling\" techniques during decoding. What is this referring to? I don't see how this differs between InstructGPT and ChatGPT.\n\nQuestion F: In lines 622-626, the finding that annotators with access to online resources perform worse on the task is remarkably counter-intuitive. Do you have any explanation on why this may be the case?", "paper_topic_and_main_contributions": "This paper introduces a new dataset designed to assess the knowledge of Large Language Models (LLMs) concerning the physical sizes of different real-world objects. The authors gathered data on 92 entities and constructed an extensive set of pairwise questions that compare the sizes of these entities.\n\nThe study involves two types of questions: \"General Questions,\" which require yes/no answers, and \"Specific Questions,\" which demand identifying the entity that is bigger/smaller. To enhance the prompts, the authors incorporated in-context demonstrations using either manually curated size information or knowledge pieces generated by an LLM.\n\nThe authors also created variations of the dataset by introducing perturbations in the quality of the information provided in the demonstrations. These perturbations involve withholding information about certain entities, masking spans of information, or including counterfactual statements.\n\nThe experimental results revealed that even though powerful models like GPT3.5-Turbo achieved up to 90% accuracy when given the gold size information (and 94% with CoT prompts), their parametric knowledge falls short compared to human knowledge, despite their extensive pretraining on general domain knowledge.", "excitement": 3, "reproducibility": 4, "ethical_concerns": "No", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {}, "extra_text": {"Missing_References": "Previous research has curated a similar dataset consisting of 486 pairs involving 41 objects. While this study presents a more extensive collection, it is essential to acknowledge and cite the preceding work in a related domain.\n\nHessam et al. \"Are elephants bigger than butterflies? Reasoning about sizes of objects.\" In Proceedings of the Thirtieth AAAI Conference on Artificial Intelligence (AAAI'16). https://arxiv.org/abs/1602.00753", "Typos_Grammar_Style_and_Presentation_Improvements": "I would strongly suggest the use of a grammar/spell checker for the camera-ready version if the paper gets accepted, as most these issues should be easy to spot/fix.\n\nLine 67: \\citep instead of \\citet for \"de Koning et al.\"\n\nLine 89: contributes -> contributions\n\nLine 101: debates -> debate\n\nLine 104: are aligned -> is aligned\n\nLine 168: question -> questions\n\nLine 349: 'yes' or 'yes' -> 'yes' or 'YES'\n\nTables 2/3: The Macro-F1 column should be center aligned to match the rest.\n\nTables 2/3: \"+ Gold Size Information\" is originally introduced as \"Exact Size Information\" in the Methodology (line 251), so it might be preferable to be consistent for ease of understanding.\n\nTables 2/3: Since you often compare the metrics between the two tables, would it not make more sense to combine them into one table? Currently, it is tedious to compare the performance of General vs Special questions since you need to check both tables to see the difference between the two formats. You could, for example, present the results as XX.XX / YY.YY, with XX corresponding to General and YY to Special (and adding this information to the caption).\n\nFigures 1/2: I feel like they are misplaced since they are only discussed on Page 7 and the reader has to scroll back and forth to see what you are referring to.\n\nFigures 1/2: It would be nice to have different shapes to indicate which experiment each data point corresponds to since now we can only discern between models. I am curious to see if, for example, the CER/CMR are higher when providing masked information vs gold sizes.\n\nLine 388: reaches -> surpasses (since the performance starts increasing from FlanT5-Large onwards, while FlanT5-Base shows scores that are rather close to those of FlanT5-Small)\n\nThroughout Section 5: Mirco-F1 -> Accuracy (You don't list Micro-F1 anywhere in Section 4.3.2?)\n\nLine 392: surprised -> surprising\n\nLines 396/397/401/456: 0.6 -> 0.06, 0.3 -> 0.03, 0.4 -> 0.04, 0.6 -> 0.06, 0.3 -> 0.03\n\nLine 499: examples -> example\n\nLine 505: Even **when** the key information is masked [...]\n\nLine 509 Figure 5 -> Figure 2 (?)\n\nLine 522: \"Other LMs still have the same tendency\" (which LMs is this referring to? No model was mentioned before this sentence.)\n\nLines 546-547: They demonstrate that ~~while~~ providing extra information [...]\n\nLine 943: Figure 5 -> Figure 4 (?)"}}]

Author rebuttal:

Meta-review: The mojority decision of the reviewers does a fair job of weighting the pros and cons of the work as presented. The updated results and cleanup of the material should be reflected in further edits of the paper inorder to clarify the discussion and highlight the contributions.
Decision: Accept-Findings