ROME: Evaluating Pre-trained Vision-Language Models on Reasoning beyond Visual Common Sense
Authors: Kankan Zhou
Eason Lai
Wei Bin Au Yeong
Kyriakos Mouratidis
Jing Jiang
Venue: EMNLP 2023 Findings (2023)
Keywords: commonsense reasoning
multimodality
pre-trained vision-language models

Abstract: Humans possess a strong capability for reasoning beyond common sense. For example, given an unconventional image of a goldfish laying on the table next to an empty fishbowl, a human would effortlessly determine that the fish is not inside the fishbowl. The case, however, may be different for a vision-language model, whose reasoning could gravitate towards the common scenario that the fish is inside the bowl, despite the visual input. In this paper, we introduce a novel probing dataset named ROME (reasoning beyond commonsense knowledge) to evaluate whether the state-of-the-art pre-trained vision-language models have the reasoning capability to correctly interpret counter-intuitive content. ROME contains images that defy commonsense knowledge with regards to color, shape, material, size and positional relation. Experiments on the state-of-the-art pre-trained vision-language models reveal that most of these models are still largely incapable of interpreting counter-intuitive scenarios. We hope that ROME will spur further investigations on reasoning beyond commonsense knowledge in vision-language research.

ROME: Evaluating Pre-trained Vision-Language Models on Reasoning beyond Visual Common Sense
Kankan Zhou, Eason Lai, Wei Bin Au Yeong, Kyriakos Mouratidis, Jing Jiang School of Computing and Information Systems, Singapore Management University (kkzhou.2020, yblai, wb.auyeong. 2021, kyriakos, jingjiang @ @smu.edu.sg
Abstract
Humans possess strong capability for reason- ing beyond common sense. For example, given unconventional image of goldfish laying on the table next to an empty fishbowl, a human would effortlessly determine that the fish is inside the fishbowl. The case, however, may be different for a vision-language model, whose reasoning could gravitate towards the common scenario that the fish is inside the bowl, de- spite the visual input. this paper, we intro- duce novel probing dataset named ROME (reasoning beyond commonsense knowledge) evaluate whether the state-of-the-art pre- trained vision-language models have the rea- soning capability to correctly interpret counter- intuitive content. ROME contains images that defy commonsense knowledge with regards to color, shape, material, size positional rela- tion. Experiments the state-of-the- -he pre- trained vision-lane language models reveal that most these models are still largely incapable in- terpreting counter-intuitive scenarios. We hope that ROME will spur further investigations on reasoning beyond commonsense knowledge in vision- language research.
Figure 1: Sample counter-intuitive images from our ROME dataset. Left: an image showing an uncommon positional relation between fish and fishbowl. Right: an image showing a chair with an uncommon relative size with respect a pizza.
new probing dataset introduce, called ROME (reasoning beyond commonsense knowl- edge). Such empirical evaluation is important be- cause humans' ability to "think outside the box" is a critical source of creativity and, thus, also a desirable ability for AI models. Unfortunately, most existing benchmarks to evaluate VLMs do not challenge them with unconventional or counter- intuitive situations.
Our ROME dataset consists of 1,563 images gen- erated by DALL-E-2 depicting counter-intuitive scenarios. Instead of relying of people's notion of counter-intuitive scenarios, we largely leverage a recent visual common sense dataset, ViComTe, that was systematically derived from Visual Genome (Zhang et al., 2022). We automatically convert commonsense assertions re- lated to objects' color, shape. material and size in ViComTe into descriptions of counter-intuttive scenarios, and use DALL-E-2 to generate images accordingly. In addition, manually use Bing Image Creator generate counter-intuitive images related to objects' positional relations to supple- ment the data above.
1 Introduction
small group
Humans possess not only commonsense knowledge such fish are usually found inside (rather than outside) fishbowl, but also the ability to handle counter-intuitive scenarios and reason beyond com- mon sense. For example, when presented with the image on the left Figure 1 and asked "Is the fish inside the fishbowl?" humans would have no problem answering "no." However, it is not clear whether recent pre-tree pre-trained vision- laigunl mod- els (VLMs) such as LLaVA (Liu et al., 2023) and InstructBLIP (Dai et al., 2023a), which exhibit im- pressive vision-language understanding abilities, have the capacity reason beyond visual common sense.
To assess both VLM's commonsense knowl- edge and its abilities reason beyond common sense, we design two groups of probing tasks the form of binary (yes/no) visual questions: (1)
In this paper, we aim to empirically answer this research question by evaluating recent VLMs

---

Commonsense questions, which are related to the common attribute values of various objects, or the typical relative size or positional relationship be- tween two objects. (2) Counter-intuitive questions, which ask about the objects, their attributes, and their relative size or positional relation in counter- intuitive images. We then define four evaluation metrics based on these probing questions.
2 Related Work
Visual common sense. With the rapid develop- ment of foundation models, increasing attention is paid to their commonsense reasoning capabilities. Recent studies have investigated the existence of commonsense knowledge in pre-trained language models (e.g.. Bouraoui et al., 2020; Lin al., 2020).
Others have focused on visual common sense (e.g.- Vedantam et al. (2015); Zellers et al. (2021); Liu et al. (2022)). In particular, Zhang et al. (2022) studied commonsense knowledge pertaining to typ- ical visual attributes of different objects. We follow their notion of visual common sense and use their dataset our main source of commonsense knowl- edge. Note that this notion of visual common sense different from the notion of visual commonsense reasoning in the well-known VCR dataset (Zellers et al., 2019).
We use the ROME dataset and the probing ques- tions to evaluate six pre-trained vision-language models: ALBEF (Li et al., 2021), BLIP-2 (Li et al., 2023a), LLaVA (Liu et al., 2023), MiniGPT-4 (Zhu et al., 2023), mPLUG-Owl (Ye al., 2023), and InstructBLIP (Dai al., 2023a). Among these models, BLIP-2, LLaVA, MiniGPT-4, mPLUG- Owl, and InstructBLIP are the latest models that are built on top of pre-trained large language models (LLMs), e.g., I.LMA. They can be directly used by end-users through natural language conversations such as to perform zero-shot visual question an- swering (VQA), making them readily available for wide adoption in end-user applications. Therefore, we focus on the evaluation of these state-of-the-an models. In contrast, although earlier VLMs such as CLIP (Radford et al., 2021) and ALBEF also demonstrated strong zero-shot capabilities on tasks including object detection (Radford et al., 2021) and VQA (Song et al., 2022), they usually require careful prompt engineering to work properly, and thus we do not focus these earlier models. Nev- ertheless, we still include one of them, namely, ALBEF pre-trained with VQAv2), in order have a comprehensive representation of VLMs.
While most previous work has focused on com- monsense reasoning, we focus on evaluating rea- soning capabilities beyond common sense. Bitton- Guetta et al. (2023) introduced WHOOPS, dataset of weirc/unconventional images, where the selec- tion of "weird" scenarios is based on a (relatively) small group of 30 designers' impression of com- mon sense and weirdness, In contrast, our selection of counter-intuitivg scenarios is based on the visual commonsense dataset by Zhang et al. (2022) that was systematically constructed. WHOOPS covers wide range of weird scenarios, many of which involve non-visual common sense, such as social norms, cultural knowledge, and celebrities. On the other hand, our dataset is centered on five cat- egories of common sense that are purely visual. These types of common sense also more primi- tive (compared to WHOOPS, where the reasons for weirdness are often complex, involving multiple steps of reasoning). Also, WHOOPS focuses on explanation, while our dataset is framed around the models' ability to overcome common sense and correctly recognitze objects, attributes and spatial relations in counter-intuitive scenarios. Another dataset that contains some counter-intuitive images is the Winoground dataset (Thrush et al., 2022), but their focus is compositional reasoning and not all images in Winoground are counter-intuitive.
Our experiments demonstrate that, although these state-of-the-art VLMs are generally effec- tive at counter-intuitive object recognition, most of them perform poorly in terms of counter-intuitive attribute and relation recognition. Mcanwhile, in- terestingly, the models perform poorly on our com- monsense questions, often giving inconsistent an- swers to pair of questions asking about the same commonsense knowledge. This suggests that many of the latest trained VLMs still lack strong com- monsense reasoning capabilities and capabilities reason beyond common sense, which, in turn, indicates that further improvements are needed in both directions.
Pre-trained vision-language models. After the success of BERT (Kenton and Toutanova, 2019), the community started developing pre-trained vision-language models, such as ALBEF (Li

---

al., 2021), X2- (Zeng al., 2023b) and CLIP (Radford al., 2021), which are trained on web-scale image-text pairs and exhibit power- ful zero-shot and fhw-shot transfer capabilities for downstream tasks. Recently, given the impressive power of LLMs (such as ChatGPT) to perform var- ious tasks through natural language conversations, a new line of work leverages LLMs to build VLMs with parameter-efficient fine-tuning (e.g., BLIIP- 2 (Li et al., 2023a), LLaVA (Liu et al., 2023) and InstructBLIP (Dai et al., 2023a)). These models usually train an alignment module to convert visual inputs into embeddings acceptable by a pre-trained LLM.
diverse coverage. We use commonsense knowl- edge related color, shape, material, and size from ViComTe. We supplement with dataset by Liu et al. (2022) (which we call ThingsNorWritten) that contains positional commonsense knowledge.
Color, shape, and material are attributes of phys- ical objects. The commonsense knowledge con- tained ViComiTe on these attributes is repre- sented as frequencies of each possible attribute value, given an object and an attribute. For ex- ample, given the object knob and the attribute shape, ViComTe provides the frequencies of differ- ent shapes such as round, square, and triangle that are associated with knobs. We use only the sub- of ViCom'Te where the most frequent attribute value has at least 80% frequency, i.e., cases where common sense for the object to have a domi- nating attribute value. To create counter-intuine intuiiive descriptions, given an object and an attribute (color, shape or material), we choose the attribute value that has the lowest frequency (including 0). For ex- ample, given the object tire and the attribute color, the value with the lowest frequency in ViComTTe is blue. Based on this, we form the counter-intuitive description a fire in the color of blue.
Evaluation of VLMs. Recently, there have been various attempts to evaluate VLMs on different aspects. Thrush al. (2022) constructed the Winoground dataset to assess the ability of VLMs to conduct uisio-linguisti compositional reasoning. Zhou et al. (2022) studied the existence of social bias in VLMs. Li et al. (2023b) considered the hallucination problem, i.e., the inclusion of objects that are inconsistent with the given images in the generated text descriptions. Zeng al. (2023a) compared different LLM backbones and model de- signs of VLMs to explore the influence of diversi- fied prompts on the cntructiio-followiin ability of these models. In contrast, we evaluate statc-of-the art VLMs for reasoning beyond common sense.
ViComTTe represents size-related commonsense knowledge as a pair of objects and their relative size, e.g., deer is larger than a plate. We sim- ply reverse the relation form counter-intuitive descriptions, e.g.. a deer that smaller than a plate. Another important type of visual common sense is the positional relation between two objects, which however is not covered in ViConTe. The Wriiten dataset by Liu al. (2022) provides commonsense assertions of this type, e.g., a fish inside a fishbowl. Because the size of TringsNotWritten is relatively small, we in- clude additional positional commonsense asser- tions based on our own judgment. To convert these positional commonsense assertions into counter- intuitive descriptions, we replace their positional preposition with a random choice from pre- defined list in ToingsNotWritten.
3 Creation of Counter-intuitive Images
In this section, we explain how we create counter- intuitive images.
3.1 Counter-intuitive Descriptions
Counter-intuitive scenarios are rare in real life pic- tures. Therefore, similar to Bitton-Guetta et al. (2023), we opt to generate images using recent text- to-image models. To use these models, we need come up with textual descriptions of counter- intuitive scenarios, such as a fish outside a fishbowl, which needs to be derived from commonsense as- sertions such as a fish inside a fishbowl. Therefore, our first step is to identify a large set of visual commonsense assertions.
For all counter-intuitive descriptions, we experi- ment with different sentence templates and choose those that work the best with DALL-E-2. More details can be found in Appendix A.
We use ViComTe (Zhang et al., 2022) our main source of visual commonsense knowledge, because it is systematically derived from Visual Genome (Krishna et al., 2017), which contains 10K images from MSCOCO and therefore has
3.2 Image Generation
Our preliminary experiments show that for color, shape, and material, automatically generating the

---

images using DALL-E-2 followed by two rounds of filtering works well (details in $3.3), but for size and positional relation, automatic generation produces too many low-quality images that do not match the given object or attribute descriptions. Hence, for size and positional relation we adopt manual method.
Color
Shape
Material
Size
Positional
Count
562
310
391
200
100
Table 1: Dataset statistics
is further reduced from about 3.5K to 1,263. To prevent any bias from the human annotators, during the recognition step of filtering, we inten- tionally introduce data samples where the images do not match the given object descriptions, as a sanity check of the annotators. We calculate Fleiss' kappa for both the object recognition and the at- tribute confirmation steps. The Fleiss' kappa for the object recognition step is 0.67, indicating sub- stantial agreement. For the attribute confirmation step, the Fleiss' kappa 0,41, suggesting moder- ate level of agreement.
For automatic generation of counter-intuitive im- ages on color, shape, and material, we feed the counter-intuitive descriptions to DALL-E-2 and generate 20 images per description. We then em- ploy CLIP (Radford et al., 2021) for first round of filtering remove low-quality images. Specif- ically, for each generated image, calculate the image's cosine similarity with the counter-intuitive description used to generate it, and its cosine similarity with the corresponding commonsense assertion. We then normalize the two similarity scores into probabilities. A thresh- old of 0.8 is used to discard images whose proba- bility of being counter-intuitive is low. This step reduces the number of images from more than 21K around 3.5K.
CLIP to
3.4 Statisties of ROME
In total, we obtain 1,563 counter-intuitive im- ages. Table provides the breakdown of our dataset into different categories. Some exam- ples of our generated images can be found in Ap- pendix B. ROME will be made publicly available at http://ptbbbbcooo//////querrr//o
For manual creation of counter-intuitive images on size and positional relation, two researchers ex- periment with different description templates using Bing Image Creator (which is powered by DALL- E-2) until they achieve the desired high-quality counter-intuitive images.
Probing Questions and Evaluation Metrics
Our main research question is whether state-of-the the art pre-trained VLMs can handle counter-intuitive images well, i.e., whether they are able to reason beyond common sense. However, we should first address a more fundamental question, which is whether these pre-trained VLMs possess visual common sense in the first place. We therefore cre- ate two groups of probing questions: (1) questions testing pre-trained VLMs' visual common sense, and (2) questions testing pre-trained VLMs' rea- soning abilities beyond visual common sense.
3.3 Human Annotation
For automatically generated images, even after the filtering by CLIP, find that there are still images where the objects are not recognizable by humans. We therefore conduct a second round of filtering based on human annotations. We split this sec- ond round into two steps: object recognition, and attribute confirmation.
In the object recognition step, given an image, we ask three human annotators to identify the de- picted object by picking one of three options. The purpose is to make sure that humans can recog- nize the object, despite its uncommon or weird attributes. For those images where the human an- notator is able to recognize the objects, we proceed to the attribute confirmation step, where three hu- man annotators judge whether the object indeed matches the counter-intuitive description used to generate the image. Only images where all three an- notators agree on the objects and the attributes are kept. After human filtering, the number of images
Our preliminary investigations showed that the latest pre-trained VLMs, such as LLaVA, tend to generate verbose answers to open-ended questions. (See Appendix C for examples.) This makes it hard to automatically process the generated an- swers. We hence design only binary questions, whose answers be automatically processed by checking whether they start with "yes" or "no."
4.1 Probing Questions
Commonsense questions. This set of questions is to test whether a VLM is equipped with common-

---

Image
Template
Sample Questions
Qla: In general, is the color of a/an [object]
general, is the color of clementine normally orange?
normally [common attribute]
Q1b: general, is the color a/an [object]
general, is the color of clementine normally black?
normally [uncommon attribute]
Commonsense question(s)
QZa: general, is the color of a/an [object]
general, is the color of clementine more likely to be orange than to be black?
more likely to be [common attribute] than to be [uncommon attributs]
Q2b: In general, is the color of a/an [object]
general, is the color of clementine more likely to be black than to be orange?
more likely be [uncommon attribute] than to be [common attribute] ?
Object
Q3.: Does this image contain a/an [object]
Does this image contain clementina?
recognition
Counter-
question(s)
intuitive
Q4a: In this image, is the color of the [object]
In this image, is the color of the deeeentie orange?
question(s)
Attribute
[common attribute]
recognition
Q4b: In this image, is the color the (object] [uncommon attributo] ?
this image, is the color of the clementine black?
question(s)
Figure 2: Probing question templates and examples of instantiated questions
sense knowledge. We design two pairs of yes/no question templates, as shown in the upper section Figure 2. The first pair of questions each asks a VLM to judge the truthfulness of an assertion that is either commonsense or counter-intuitive. the model answers "yes" to Qla and "no" to Qlb, we deem it to have the corresponding commonsense knowledge. The reason introduce Qlb is to ensure that models do not score high just by swering "yes" to all questions. The second pair of questions each asks a VLM to compare the likeli- hood of the commonsense attribute value versus an uncommon attribute value. If the model answers "yes" to Q2a and "no" to Q2b, we deem it to have the respective commonsense knowledge.
to reason beyond visual common sense. Q4a and Q4b ask about either the color, shape, material of the object in the image, or the relative size or positional relation between two depicted objects. Similar to previous probing questions, if the VLM answers "no" to Q4a and "yes" to Q4b, it can cor- rectly identify the counter-intuitive attribute value of the object (or the counter-intuitive relation be- tween two depicted objects), thus demonstrating reasoning abilities beyond visual common sense. The question templates used are slightly differ- ent for different commonsense types. E.g., for material, the template Qlb is "In general, is a/an [object] normally made of [uncommon attribute]?" The full list of question templates can be found in Appendix D.
Counter-intuitive questions. This set of ques- tions based on the generated counter-intuitive images. Given an image showing counter- intuitive scenario, we test whether VLM can overcome visual common sense and answer the questions correctly. Specifically, we design two subgroups of questions: (1) object recognition, and (2) attribute/relation recognition. These question templates are shown in the lower section of Fig- ure 2. Q3 asks about the existence of an object. If VLM answers "yes" to Q3, the VLM can recog- nize the object despite its unusual color, shape. or material, or its co-occurrence with another object unusual relative size or position. This object recognition ability can be regarded as an ability
4.2 Evaluation Metrics
To facilitate the formal definition of the metries below, we use denote the set of images in the ROME dataset. For an image I R and question template Q, we use Q(J) to denote the question instantiated from template Q with the objects, at- tributes and/or relations found in image / as well as the underlying visual commonsense knowledge associated with this image. For example, given image showing blue banana and the knowledge that the commonsense color of bananas yellow, Qla(T) would be "In general, the color of ba- nana normally yellow?" and QIb(I) would be "In general, is the color of a banana normally blue?"

---

Let M denote VLM. We use (I,, Q(2)) to denote the answer given by M for image I and question (I2), which is instantiated from template Q with information from image /2- We expect M(1 ((2)) to be either "yes" or "no""
that contains is objects (where l is 1 for images related color, shape, and material, and ko is 2 for images related to size and positional relation), (3()) is a set of k questions, each corresponding to an object in J. We define CI-Obj(M, 7) to be if and only if model M can recognize all objects in I. In turn, CI-Obj M) is the average of C1-Obj I) over all images.
4.2.1 Metries for Commonsense Knowledge We introduce two metrics for testing VLMs' com- monsense knowledge.
Counter-intuitive score based on at- tribute/relation recognition (CI-AurRel):: If a VLM cannot recognize counter-intuitive at- tributes or relations, its reasoning beyond common sense is deemed poor. Given a counter-intuitive image I, we define CI-AttrRel I) to be 1 if and only if model M answers both Q4a(I) and Q4b(I) correctly, according to the actual content of image I. (I-Attree(MM) is the average of CI-AtuRee(M,1) over all R.
Commonsense score based on language (CS-L): When asking VLM the commonsense probing questions instantiated from templates Qla, Qlb, Q2a, and Q2b, the model does not have to refer to any image, because the questions are asking about general cases rather than a specific case depicted by any image. Therefore, it is natural to give the VLM a non -on-informative image, such as a blank image in white arguably random image. Let Auull denote this non-informative image. Given model M, define the first metric follows:
Experiments
CS-L1(M)
Σ FRR
(M,, (1)
5.1
VLMs for Evaluation
R
We focus on six of the latest VLMs, summarized in Table 2. The first five leverage LLMs with parameter-efficient fine-tuning and demonstrate zero-shot capabilities on downstream tasks. For completeness, we also consider ALBEF fine-tuned on VQAv2. Some of these models tend to give verbose answers, even to binary questions. We hence treat an answer as "yes" (or "no") if it starts with "yes" (or "no"). In the penultimate column of Table 2, we show the percentage of such answers. Except for LLaVA, the percentage is high.
where (M,,) is 1 if M (fmull QIa(I)) is
"yes" and M (Tmll,, QIb(T)) is "no". In other words, (M,, is if and only if model M answers both Qla(/) and Qlb(/) correctly according to commonsense knowledge, given non-informative image. CS-L, (M) is the aver- age of CS-L, (M, over all R. i.e. the fre- quency which the model demonstrates common- sense knowledge. We calculate CS- sim- ilarly, based on the model's answers to Q2a(1) and Q2b(/). The final CS-L(M) is set to be max (A)) Commonsense score based on vision and lan- guage (CS-VL): To test whether a VLM might be "distracted" by counter-intuitive image when given a commonsense question, we define CS-VL just like CS-L but, instead of non-informative image, we use counter-intuitive image.
5.2 Main Results
Table 3 presents the performance of the models over the entire ROME dataset, according to the four metrics in $4.2. We use blank images as non- informative images to obtain CS-L. We can answer the following research questions based on the sults.
(1) When given non-informative image, do state-of-the-art VLMMs demonstrate visual com- monsense knowledge?
4.2.2 Metrics for Reasoning beyond Common Sense
We define two metrics for reasoning beyond visual common sense. One corresponds to object recogni- tion and the other to attributc/relation recognition. Counter-intuitive score based on object recogni- tion (CI-Obj): This metric is designed based on the motivation that if VLM cannot recognize a counter-ing counter-intuitive object, then has limited ability to reason beyond common sense. Given an image
The relevant metric for this question is CS-L. We can see that the CS-L scores of all models are rela- tively low, ranging from 10.89% to 32.31%. This indicates that they do not exhibit strong visual com- monsense knowledge. This is some extent unex- pected, because previous work reported that CLIP and Oscar, two earlier pre-trained VLMs, achieved between 60% and 80% of accuracy when predict-

---

Model
Visual Encoder
Alignment Network
LLM
Yes/No (%)
Hallucination (%)
BLIP-2
VIT-L/14
Q-Former
99.81
30
InstructB1IIP
VIT-1/14
Q-Former
Vicunara
95.91
5
LLaVA
ViT-L/14
Linear
LLaMArg
39.22
100
MiniGPT-4
ViT-G/14
Linear
Vicunara
99.81
40
mPLUG-OWl
ViT-G/14
Attention
LLaMArB
98.78
65
ALBEF
VIT-B/16
Attention
BERT
100.00
Table The VLMMs included in the evaluation
Commonsense CS-L
Counter-intuitiv Cl-AttrRel
difference), implying that these models are prone
Model
CS-VL
CI-Obj
to visual distractions caused by the random images not related to the questions.
BLIP-2
5.82
2.48
91.88
27.80
InstructBLIP
32.31
9.66
94.75
63.72
LLaVA
31.41
28.09
98.34
0.13
MiniGPT-4
10.89
4.41
94.56
5.31
general, normally smaller towel?
mPLUG-Owl
28.53
18.11
97.38
35.83
ALBEF
14.01
0.51
90.79
44.53
No, general, generally larger than lowel. mPLUG-0R Owl
Table 3: Overall evaluation results (scores in %)
general, normally larger towel?
No, generally a car is smaller than towel.
mPLUG-Owl
ing the commonsense color, shape, and material of objects (Zhang et al., 2022). We therefore further inspect the results and find that the mistakes made by the VLMs we are evaluating are usually due to inconsistent responses. For example, in Figure 3, when asked to compare the relative size of car and a towel, m mPLUG-OWl gives contradictory answers to the two probing questions. Similar inconsisten- cies are commonly found with all models.
Figure 3: An example inconsistent answers questions asking about the same commonsense knowl- edge. A blank image input to mPLUG-Ow:
Queetion
general, normally smaller than towel?
Although blank images non-informative, one may argue that blank images are disttiibbioiooo data considering that these pre-trained VLMs have rarely encountered blank images their training data. Therefore, choosing blank images might af- feet these models' performance. To address this concern, we also conduct additional experiments with randomly selected non- bonnblank images. Specifi- cally, for each instance in our dataset, repeat the probing experiments for the CS-L scores, but in- stead of blank image, we use randomly selected image from the Visual Genome dataset. repeat the process three times on all models. The results are shown in Table 5. The results indicate that using randomly selected images produces similar CS-L scores using blank images.
Yes, a car usually smaller than a towel.
mPLUG-Ow
Question
general, normally larger than towel?
No. generally smaller than towel.
mPLUG Owl
Figure 4: An example of how counter- innuterrintuitive image weakens commonsense reasoning
(2) When presented with counter-intuitive im- ages and asked about commonsense knowledge, do the models behave differently? Comparing CS-L with CS-VL. it is clear that once the counter-intuitive images are presented, the com- monsense reasoning of all models is further im- paired; the CS-VL scores are lower, ranging be- tween 0.51 % and 28.09%. That is not surprising because the counter-intuitive images may confuse VLMs during commonsense reasoning. For ex- ample, in Figure 4, when mPLUG-OWl is given counter-intuitive image, it answers incorrectly even the first commonsense question (although did get right Figure with a blank image input). This problem widely observed with all models.
Overall, we find that InstructBLIP and LLaVA answer commonsense questions correctly about 30% and 35% of the time, regardless of whether blank image or random image is given. For AL- BEF, BLIP-2, MiniGPT-4 and mPLUG-Owl, using randomly selected images produces slightly lower CS-L scores than using blank images (around 5%

---

Model
Color
Shape
Material
Size
Position
BLIP-2
46,26
2.00
32.23
3.00
8.00
InstructBI.IP
84.34
28.39
57.03
88.00
35.00
VLaVA
0.18
0.00
0.00
0.00
1.00
MiniGFT-4
8.01
0.00
7.93
3.50
0.00
mPLUG-Owl
50.89
30.00
33.76
17.50
14.00
ALBEF
50.00
1.94
60.36
70.00
33.00
Table 4: Cl-AurRel (in %. for different commonsense types
Commonsense CS-L
image, our conjecture above may not hold. To get sense of how common the hallucination problem is, first randomly retrieve 20 images containing different objects from ROME. Next, for each image we pick a non-exist-object label random from the exist-object labels of the other 19 images. Once all images are assigned a non-exist-objeet label, we ask "Does the image contain a/an [non-exist- object]?" The hallucination rate is presented in Ta- ble 2. All models suffer from object hallucination to some extent, but LLa VA and mPLUG-Ow suf- fer the most. On the other hand, the hallucination rate of InstructBLIP and ALBEF is only 5%. The results suggest that for InstructBLIP and ALBEF, we can safely claim that they can recognize counter- intuitive objects well, for BLIP-2 and MiniGPT-4, they probably can recognize counter-intuitive ob- jects well, but for LLaVA and mPLUG-OWI, we cannot draw conclusions.
Model
(Blank Image)
(Random Image)
BLIP-2
5.82
0.30
InstructBLIP
32.31
35,03
LLaVA
31.41
31.01
MiniGPT-4
10.89
6.07
mPLJG-OWl
28.53
25.39
ALBEF
14.01
8.79
Table 5: Blank image random image (scores in %)
image, the chair larger than the pizza?
BLIP-2:
InstruciBLIP: No, the chair is smaller than the pizza.
LLaVA: the image, chair larger
pizza.
MiniGPT-4; Yes, the chair larger than the pizza.
mPLUG-OWt: Yes, the chair is bigger than the pizza the image.
image,
than pizza?
BLIP-2:
InstructBLIP: Yes, the chair is smaller than the pizza.
LLaVA: In this image, the chair is not smaller than the pizza.
MiniGPT-4: Yes. the chair is amaller than the pizza.
(4) Can the models recognize counter-intuitie attributes and spatial relations?
mPLUG-OW:: Yes, the image, the chair is smaller than the pizza.
Figure 5: Examples of models' responses to counter-
The relevant metric here is CI-Attrel.. The mod- els' scores are low, except maybe for IntructBL.IP. which achieves the highest Cl-Attrel of 63.72%. We infer that, despite recognizing the objects suc- cessfully, VLMs struggle with recognizing counter- intuitive attributes and spatial relations. Figure 5 presents all models' responses when the image de- picts a pizza that larger than a chair. BLIP-2 answers both questions wrongly. MiniGPT-4 and mPLUG-OWl answer "yes" to both questions and are thus inconsistent. LLaVA's answers do not start with "yes" or "no" but judging from the complete responses, it contradicts itself. This behaviour sug- gests that the models have room for improvement in terms of both visual language understanding and logical consistency. On the other hand, in this example InstructBLII answers both questions cor- rectly.
intuitive questions
(3) When given counter-intuitive images, can the models correctly recognize the depicted ob- jects, even though these objects are shown with counter-intuitive attributes or have a counter- intuitive relative size or counter-intuitive spatial position with respect to other objects?
Surprisingly, all models perform very well in counter-intuitive object recognition. They achieve CI-Obj scores from 90.79% 98.34%, which sug- gests a positive answer to this research question. That said, recent work has shown that some VLMs may suffer from the hallucination prob- lem (Li et al., 2023b; Dai et al., 2023b), i.e., they may determine that object exists in image while in reality it does not. This may affect the in- VLM tends
terpretation our results, because to answer "yes" when asked about the existence of any object, even when that object is not in the
In Table 4, we break down the models' CI- AttrRel scores for each commonsense type. Al- though the models perform very differently, recog-

---

Model
CI-AttrRel
CM-AttrRel
when the models are presented with commonsense images and asked about commonsense attributes (e.g., "In this image, the color of the banana yel- low?"), they clearly perform better (i.e., give consis- tent commonsense answers) than when presented with counter- inntter-nttuitve images. This is not surpris- ing, because the commonsense images present vi- sual commonsense knowledge, which presumably aligns with the commonsense knowledge implicitly containd in the VLMs.
InstructBLIP
63.72
77.00
LLaVA
0.13
4.00
Table 6: Cl-Attrell CM-AttReel (in %)
nizing counter-intuitive colors turns out to be the casiest for most of them. For BLIP-2, MiniGPT-4 and mPLUG-OWl, recognizing counter-intuitive material is the second easiest. Interestingly, for In- structBLIP and ALBEF, size is the easiest, whereas for the other models it tends to be very hard.
Conclusion
In this work, we construct the ROME dataset and propose probing framework for evaluating the capability of pre-trained vision-language models (VLMss to reason beyond common sense. Using ROME and set of metrics we define, we show that state-of of-the-art VLMs are lacking this aspect. We hope ROME will spur further research the important direction of reasoning beyond common sense in the NLP and vision communities.
5.3 Further Investigation Common Sense The finding that the VLMs do not demonstrate strong visual commonsense knowledge appears contradictory previous reports with CLIP and Oscar (Zhang et al., 2022). We have pointed out in $5.2 that a main reason we find is that the VLMs we evaluate often give inconsistent answers. We therefore suspect that the low CS-L scores may not be due to VLMs lacking commonsense knowledge per but maybe the phrasing of our questions does not "activate" that knowledge. To investi- gate this, we pick one of the VLMs, LLaVA, and further test with the following open-ended ques- tion: "In general, what is the common [attribute] a/an [object]?" where [attributc] is set to color, shape or material. We experiment with 20 ques- tions. Overall, LLaVA is able to give meaningful commonsense answers 75% of the time. This ob- servation confirms suspicion that some VLMs' commonsense reasoning ability may be sensitive the way the question is phrased. We believe that this may general weakness of the latest VLMs.
7 Limitations
Generation of counter-intuitive images requires sig- nificant human involvement, as current state-of- the-art text-to-image generation models, such as DALL-E-2, have a strong bias towards common- sense images. Such human involvement may in- evitably introduce human bias to our dataset, in- cluding possible cultural bias because all our an- notators come from Asia. Additionally, common sense may be binary. There are degrees which one might consider something to be of com- mon sense, hence, selecting an unequivocally non- commonsense characteristic for an object is also a challenge.
Our experiments earlier on Cl-AttrRel focus on attribute recognition in counter-intuitive images. It would be helpful to check how well the VLMs per- form attribute recognition in commonsense images. We therefore also conduct another set of additional experiments to test whether attribute recognition is easier for these VLMs in commonsense images. In this set of experiments, we randomly sample 100 images from ROME from the color, shape, and material categories and create 100 corresponding commonsense images (c.g., yellow bananas) via Bing Image Creator. We choose LLaVA and In- structBLIP and perform the same experiments as for CI-AttrRel, albeit with commonsense images as visual input. We report the respective score, de- noted as CM-Attreel, in Table 6. We observe that
We designed only binary questions to probe VLMs, which may not "activate" the full knowl- edge and capabilities of VLMs. For example, discussed earlier, some VLMs may show com- monsense knowledge when prompted in differ- ent way. We leave the investigation of how more effectively prompt VLMs to understand their commonsense-relann capabilities, and why VLMs behave in inconsistent ways when they are given the same questions phrased differently, as our fu- ture work.
A Acknowledgements
This research was supported by the Ministry Education, Singapore, under its Academic Re-

---

search Fund Tier 2 (Proposal ID: 12IP20222-0047 Project ID: MOE-000440-00), Eason Lai and Kyr- iakos Mouratidis were supported by the Ministry of Education, Singapore, under its Academic Re- search Fund Tier 2 (Award No. MOE-T2EP20121- 0002). Any opinions, findings and conclusions or recommendations expressed in this material are those of the authors and do not reflect the views of the Ministry of Education, Singapore.
Yifan Li, Yifan Du, Kun Zhou, Jinpeng Wang. Wayne Xin Zhao, and Ji-Rong Wen. 2023b. Eval- uating object hallucination in large vision-language models. arXiv preprint Xriv:23005 10355
Bill Yuchen Lin, Seyeon Lec, Rahul Khanna, and Xiang Ren. 2020. Birds have four legs?! NumerSense: Probing Numerical Commonsense Knowledge of Pre-Trained Language Models. In Proceedings of the 2020 Conference on Empirical Methods in Natu- ral Language Processing (EMNLP). Association for Computational Linguistics.
References
Haotian Liu, Chumyuan Li, Qingyang Wu, and Yong Jac Lee. 2023. Visual instruction tuning, arXiv preprint arXiv:2304.08885.
Nitzan Catton Yonatan Bitton, Jack Hessel, Ludwig Schmidt, Yuval Elovici, Gabriel Stanovsky, and Roy Schwartz. 2023, Breaking common sense: Whoops! vision-and ioooo--nd-lagguee language benchmark of syn- thetic and compositional images. arXiv preprint 0rXiv:2303.07274.
Xiao Liu, Da Yin, Yansong Feng, and Dongyan Zhao. 2022. Things written in text: Exploring spatial Proceedings
commonsense from visual signals. of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). Association for Computational Linguistics.
Zied Bouraoui, Jose Camacho-Collados, and Steven Schockaert. 2020. Inducing relational knowledge from bert. Proceedings of the AAAI Conference on Artificial Intelligence, volume 34, pages 1463.
Alee Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sas- try, Amanda Askell, Pamela Mishkin, Jack Clark, et al. 2021. Learning transfcrable visual models from natural language supervision. In International Conference Machine Learning, pages 8748-8763. PMLR.
Wenliang Dai, Junnan Li, Dongxu Li, Anthony Meng Huat Tiong, Junqi Zhao, Weisheng Wang, Boyang Li, Pascale Fung, and Steven Hoi. 2023a. In- structblip: Towards general-purpose vision-language models with instruction tuning. arXiv preprint arXiv:2305.06500
Haoyu Song, Li Dong, Wei-Nan Zhang, Ting Liu, and Furu Wei. 2022. Clip models are few-shot learners: Empirical studies vqa and visual entailment. arXiv preprint 2rXiv:2003.07190.
Wenliang Dai, Zihan Liu, Ziwei Ji, Dan Su, and Pascale Fung. 20236. Plausible may not be faithful: Probing object ballucination
pre-trame pre-training. In Proceedings of the 17th Conference of the Euro- pean Chapter of the Association for Computational Linguistics. Association for Computational Linguis- tics.
uision-langual
Tristan Thrush, Ryan Jiang, Max Bartolo, Amanpreet Singh. Adina Williams, Douwe Kiela, and Candace Ross. 2022. Winoground: Probing vision and lan- guage models for lisquliin compositionality In Proceedings of the IEEE/CVF Conference on Com- puter Vision and Pattern Recognition, pages 5238- 5248.
Jacob Devlin Ming-Wei Wei Chang Kenton and Lee Kristina Toutanova. 2019. Bert: Pre-training of deep hidirec- tional transformers for language understanding. In Proceedings of NAACL-HLT, pages
Ramakrishna Vedantam, Xiao Lin, Tanmay Batra, C Lawrence Zitnick, and Devi Parikh. 2015. Learn- ing common sense through visual abstraction. In Proceedings of the IEEE international conference on computer vision, pages 2542-2550.
Ranjay Krishma, Yuke Zhu, Oliver Groth, Justin John- son, Kenji Hata, Joshua Kravitz, Stephanie Chen, Yannis Kalantidis, Li-Jia Li, David A Shamma, et al. 2017. Visual genome: Connecting language and vi- sion using crowdsourced dense image annotations. International journal of computer vision, 123:32-73.
Qinghao Ye, Haiyang Xu, Guohai Xu, Jiabo Ye, Ming Yan. Yiyang Zhou, Junyang Wang, An- wen Hu, Pengeheng Shi, Yaya Shi, 2023, mplug-owl: Modularization empowers large lan- guage models with multimodality. arXiv preprint urXiv:2304.14177
Jumnan Li, Dongxu Li. Silvio Savarese, and Steven Hoi. 2023a. Blip-2: Bootstrapping language-image pre- training with frozen image encoders and large lan- guage models. arXiv preprint arXiv:2301. /2597.
Jumnan Li, Ramprasaath Selvaraju, Akhilesh Gotmare, Shafiq Joty, Caiming Xiong, and Steven Chu Hong Hoi. 2021. Align before fuse: Vision and language representation learning with momentum distillation. Advances in neural information processing systems, 34:9694-9705.
Rowan Zellers, Yonatan Bisk, Ali Farhadi, and Yejin Choi. 2019. From recognition to cognition: Vi- sual commonsense reasoning. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pat- lern Recognition (CVPR).

---

Rowan Zellers, Ari Holtzman, Matthew Peters, Roozbeh Mottaghi, Aniruddha Kembhavi, Ali Farhadi, and Yejin Choi. 2021. PIGLeT: Language grounding through neuro-symbolic interaction in 3D world. In Proceedings of the 59th Annual Meeting of the Asso- ciation for Computational Linguistics and the 11th International Joint Conference Natural Language Processing (Volume I: Long Papers), pages 2040- 2050. Association for Computational Linguistics.
Yan Zeng, Hanbo Zhang, Jiani Zheng, Jiangnan Xia, Guoqiang Wei, Yang Wei, Yuchen Zhang, and Tao Kong. 2023a. What matters training gpt4-style language model with multimodal inputs?
Yan Zeng, Xinsong Zhang, Hang Li, Jiawei Wang, Jipeng Zhang, and Wangchunshu Zhou. 2023b. X--vim: All-in-one pre-trained model for vision- language tasks.
Chenyu Zhang, Benjamin Van Durme, Zhuowan Li, and Elias Stengel-Esking 2022. Visual commonsense in pretrained unimodal and multimodal models. In Proceedings of the 2022 Conference of the North American Chapter of the Association for Computa- tional Linguistics: Human Language Technologies. Association for Computational Linguistics.
Kankan Zhou, Eason Lai, and Jing Jiang. 2022. VL- StereoSet: study stereotypical bias pre-trained vision-language models. In Proceedings of the 2nd Conference of the Asia-Pacific Chapter of the Asso- ciation for Computational Linguistics and the 12th International Joint Conference on Natural Language Processing (Volume 1: Long Papers). Association for Computational Linguistics.
Deyao Zhu, Jun Chen, Xiaoqian Shen, Xiang Li, and Mohamed Elhoseiny. 2023. Minigpt-4; Enhancing vision-lange languag understanding with advanced large language models. arXiv preprint 2304 10592.

---

A Examples of Templates
Figure 6, we lists some examples of templates that we use to instruct DALL-E-2 to generate im- ages.
Color
A full photo of a/an fattribute value) (object).
A photo of (object) color of (attribute value).
Material
A/an (object) made of/with (attribute value).
A photo of a/an (attribute value) (object).
A photo of (object) material of (attribute value).
A/an (object) made of/with (attribute value), digital art. photo of (object) material of (attribute value), digital art.
Shape
photo of (attribute value) (object).
A photo of a/an (object) in the shape of (attribute value),
tiny deer beside large plate
tny bicycle on huge hand
A full photo of a/an (attribute value) (object), digital art.
A photo of fattribute value) (object).
A photo of (object) the shape of (attribute value), digital
position
Figure 6: Examples of templates
bear
bloyde
beside sofa
B Sample Images in ROME
Figure 7, we show examples the generated im- ages that have been validated by human annotators and included in the final ROME dataset.
color
Open-ended Questions
orange
cigar
Figure 8, we show the responses from LLaVA some open-ended questions. These responses suggest that if we were to raise open-ended ques- tions instead of binary (i.e., yes/no) questions, would be very difficult to automatically extract the answer to these questions. Therefore, in our design of probing questions, we opt for binary questions.
material
celling light made
bethtub
shape
D Probing Questions
Figure shows the full list of probing questions we have designed based on the different types of commonsense knowledge we intend to probe. For some pre-trained VLMS (specifically, InstructBIIIP LLaVA, and mPLUG-Owl), the following extra instruction helps: "Please swer yes no." Therefore, in our experiments, include this extra instruction for these models.
doughnut in shape of polygon
rhombus broccoli
Figure ROME Sample Images
find that adding

---

Open-ended commonsense question
LLaVA response
In general, what the shape of a door knob?
he shape door knob typically round oval, with flat slightly curved surface and hole in the center fit doorknob handle.
The color of a/an chair arm depends its design and materials. Chair arms can be made from various malerials such wood, metal, plastic, and Uhey comre in veriely of colors.
In general, what is the color of a/an chair arm?
color knite handle can vary depending on specilic type knite its design. Hiwwees, general, knili handles found in a range coloes, inclutiing black, silver, gold, other metallic shades. Some knives may also have wooden or plastic handles.
In general, what is the color of a/an knile handle?
A burger typically has combination of ingredients that include a cooked patty of ground breel, a bum, and various loppings such as letiure, lomalo, theese, and rondiments like ketchup, mustard, or mayonnaisc.
in general, what is the color of afan hurger?
The waler drain is lypically made of malerials like slainless sirel, casl irun, or pladic, which are designed to allow water to flow through while preventing clogging.
in general, what water drain made
Hard sciap is lypirallyy made from combination of ingredients sucti as water, soap base (usually sodium or potassium soap), and various optional ingredients like fragrances, moisturizers, and coloring agents. The soap base is mixed with water and other ingredientss and then it undergoes a chemical reaction to create the soap bubbles that clean hands and provide refreshing feel. The fragrances and moisturizers added to the soap can enhance the user's experience, making the handwashing, process more enjoyable and providing additional benefits like moisturizing and deodorizing the skin.
In general, what is the hand soap made of?
Figure 8: Sample responses from LLaVA for open-ended questions.
Type of commonsense knowledge
generol, Is the color of a/on [objez] normally [common sttribute]
In general, is the shope of a/on (object] normilly (cammon sttribute]
In goneral, is a/an [object)
In goneral, is a/an (otject nuimally (common relasion) a/an [object 2j2
normally made [common attribute]
zeneral a/an (object normally langer than w/an jobject 2]
general, the color of alan lebject] normally (uncommon Atribule]
in gereral is the shape of a/an (ooject) normally juncommon allribauul
In guneral, is a/an (object) normally made (uncommon aitriblle
general & afan lobject at normally (uncommon reletion)
w/an [ubjact 217
In general, is a/an (object_1)
genseal, the coler of afien [object] more
In genwnol, is the shape of s/len
In gensrod wdenn (object) mone [common
likely |common
Quastinn))
(ooject] likell be [commoo attribute] than to be [uncommon sttribute)
Ikeey to attribute] than to be made of [uncommon attr strbbute
relation] s/an (object 2] then to be juncommon relation] w/en
[common antribuuee than
be (uncommon attri butel
genweel, acaeen [object normally smallertha than a/an Tokject 2]
generall the color of alan [ebjocs]]
in general, is the shape of afan be (uncommon intribute than to be (common attribune] ?
In general, is alan (obyect) more of [uncomman netribute] than to made of [common attribuec)
In general, is a/an [object_1]
mone loody be Juncommonn
[object]
(kely
(uncammon attribute] than be [common attr ibutt))
retation] [ooject 2]
to be [common relation] a/an (object
Does this image contain a/an jobject
Does this image contain a/an [objec: 1]
Object Recoprition Question))
Does this image contain a/an [object]
Does this image contain afann (onject)
Does this image contain a/an [object]
Does this image contain w/an jobject 2]?
Does this image contain a/an [object 2]
impee color the (object] [common attribtte? 7
image, the (object [common relation) the (object 217
in image, is shape the [object] [common attribute) ?
In imags, is the [objject] made of [common attr nurbbute
this imaga, the Jobject 1] larper than the (object 2|
Counter inuuto- Question)))
Ihe (séject] (umssmmman ottribute]
mmge,
In this image. is the shape the [object] [uncommon acur buto]
In this image. is the (object) made [llucommmon attrlbunee? ?
this image, the jobject 11
[umaummnn relatinn] the
image, is the (ooject 1)
smaller than the jobject 2]
[otjec:
Figure 9: The full list of probing questions by the types of commonsense knowledge.


Reviews:
[{"review_id": "IEtdw04rpo", "reviewer": "Reviewer_EaB5", "summary": "", "questions": "Question A: Can you justify the use of a blank image from a ML perspective as the model and its vision encoder hasn't been trained on this distribution. Does it contribute to the low score? \nQuestion B: Can you provide some preliminary results on the models performance when presented with a common image in attributes and spatial relations? And how does that contrast with the performance with counter-intuitive images?\nQuestion C: Will you open-source the dataset and its pipeline? I am currently marking a 3 on reproducibility and is happy to adjust it to 5 if so.", "limitations": "", "rating": 4, "confidence": 3, "soundness": 4, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "### Question A\n\nWe thank the reviewer for the question on the use of blank images. We agree that blank images are considered out-of-distribution data. Our rationale for using blank images is the following: (1) Blank images do not contain any visual information and presumably would not mislead the vision-language model in any way, thus allowing us to assess the model’s commonsense knowledge. (2) Ideally, pre-trained large vision-language models are expected to be robust and able to handle out-of-distribution data, including blank images. In fact, the counter-intuitive images we created are also out-of-distribution images.\n\nThat said, we also think the reviewer raised a valid and good question. To empirically analyze the implication of using blank images, we conducted additional experiments with randomly selected non-blank images. Specifically, for each instance in our dataset, we repeated the probing experiments for the CS-L scores, but instead of a blank image, we used a randomly selected image from the Visual Genome dataset, and we repeated the experiments three times. For this rebuttal, we conducted these experiments only on  LLaVA and InstructBLIP, which showed the highest commonsense CS-L scores when given blank images. We will conduct full experiments with other models when revising the paper. The results are shown below. We can see that for LLaVA, using randomly selected images gives similar CS-L scores as using blank images, showing that LLaVA answers commonsense questions correctly about 30% of the time, regardless of whether a blank image or a random image is given. For InstructBLIP, using randomly selected images gives lower CS-L scores (around 35%) than using blank images (around 40%), implying that a randomly selected image negatively affects InstructBLIP’s commonsense capability. However, we can also see that the CS-VL scores of InstructBLIP are anyway much lower (23.55%), indicating that when counter-intuitive images are given instead of random images, InstructBLIP still suffers a lot.\n\n|              | **Commonsense CS-L (blank-image)** | **Commonsense CS-L (random image, repeated 3 times)** | **Commonsense CS-VL (counter-intuitive image)** |\n|:------------:|:----------------------------------:|:--------------------------------------------------:|:-----------------------------------------------------:|\n|     LLaVa    |               30.06%               |                       30.95%                       |                        27.92%                         |\n|     LLaVa    |                                    |                       30.51%                       |                                                       |\n|     LLaVa    |                                    |                       31.58%                       |                                                       |\n| InstructBLIP |               39.96%               |                       34.92%                       |                        23.55%                         |\n| InstructBLIP |                                    |                       35.44%                       |                                                       |\n| InstructBLIP |                                    |                       34.73%                       |                                                       |\n\n\n### Question B\n\nWe thank the reviewer for this suggestion. In this work, we did not use common images because there has been other work evaluating the general object/attribute/relation recognition capabilities of VLMs [1]. Due to limited time for rebuttal, we only managed to create 50 common images (e.g., yellow bananas) using Bing Image Creator. We used LLaVA and InstructBLIP to perform some preliminary experiments. We found that when these models were presented with common images and asked about commonsense attributes/spatial relations (e.g., “in general, is the color of a banana normally yellow”), these two models performed better (i.e., giving consistent commonsense answers) compared with the performance when the models were presented with counter-intuitive images. This is not surprising because the common images given reinforce the commonsense knowledge. Meanwhile, when presented with common images and asked about the specific attributes/spatial relations shown in the specific images, the models often struggled to give consistent and correct answers. We believe more systematic investigation is needed, and we will include a thorough experiment and analysis on this when revising the paper.\n\n### Question C\n\nYes, we will make our code and dataset publicly available.\n\n\n[1] Li et al., “Evaluating Object Hallucination in Large Vision-Language Models.” https://arxiv.org/abs/2305.10355", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "1. The task to evaluate VLMs under counter-intuitive content is interesting and the dataset can be a helpful resource to the community to access VLM's capabilities.\n2. The conclusions this paper derives are valuable and raises many interesting questions. Eg: how to improve robustness of VLMs under prompt variation, how to improve their capabilities under counter-intuitive content.", "reasons_to_reject": "1. The experiment's methodology of feeding a blank image into the model is flawed, as this blank-image input is entirely out-of-distribution compared to the training data for all five models. This makes the argument that \"This indicates that they do not exhibit strong visual commonsense knowledge\" less convincing.\n2. The paper asserts limited performance under counter-intuitive scenarios with respect to attributes and spatial relations without providing a comparative analysis of the model's performance in typical environments. This lack of comparison makes it difficult to assess the true impact of counter-intuitive content on the model's reasoning capabilities.", "questions_to_authors": "Question A: Can you justify the use of a blank image from a ML perspective as the model and its vision encoder hasn't been trained on this distribution. Does it contribute to the low score? \nQuestion B: Can you provide some preliminary results on the models performance when presented with a common image in attributes and spatial relations? And how does that contrast with the performance with counter-intuitive images?\nQuestion C: Will you open-source the dataset and its pipeline? I am currently marking a 3 on reproducibility and is happy to adjust it to 5 if so.", "paper_topic_and_main_contributions": "The paper introduces a partially machine-generated dataset called ROME with over 1K samples to evaluate Vision-Language Models (VLMs)'s capability to interpret counter-intuitive content related to color, shape, material, size and positional relation.", "excitement": 4, "reproducibility": 5, "ethical_concerns": "No", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"Typos_Grammar_Style_and_Presentation_Improvements": "Line 563: \"except maybe for IntructBLIP\" - should be \"InstructBLIP\""}, "extra_text": {}}, {"review_id": "TdDi9DbHCo", "reviewer": "Reviewer_2QPL", "summary": "", "questions": "n/a", "limitations": "", "rating": 3, "confidence": 4, "soundness": 3, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "### Lack of evaluation on other VLMs\n\nWe thank the reviewer for this valuable comment and we acknowledge that this is a good suggestion. We have conducted additional experiments with ALBEF fine-tuned on VQAv2 [1], and our experiments show that fine-tuned ALBEF has limited commonsense knowledge but is comparable to multimodal LLMs for counter-intuitive reasoning. Details are given below. We will include evaluation on other VLMs such as X-VLM when revising the paper.\n\nFor the experiments with fine-tuned ALBEF, the results can be found in the table below in the last row. The other rows are taken from the original submission and are shown for comparison purposes. We can make the following observations: (1) When fine-tuned ALBEF is presented with counter-intuitive images, it shows strong object recognition abilities (90.63%), at a level close to BLIP-2 and InstructBLIP. (2) When fine-tuned ALBEF is presented with counter-intuitive images, its abilities to recognize attributes and spatial relations (which are counter-intuitive) (47.01%) are weaker than InstructBLIP (67.26%) but stronger than the other multimodal LLMs we evaluated. As the reviewer suspected, ALBEF indeed has some advantage over many multimodal LLMs in this aspect, although it still loses out to InstructBLIP. (3) When fine-tuned ALBEF is presented with either a blank image or a counter-intuitive image and prompted with commonsense questions, its performance (12.67% on blank images and 0.36% on counter-intuitive images) is substantially lower than some of the multimodal LLMs such as InstructBLIP, LLaVA, and mPLUG-Owl. We suspect that this is because ALBEF does not leverage a large pre-trained LLM, whereas InstructBLIP, LLaVA and mPLUG-Owl make use of pre-trained LLMs such as LLaMA, and their commonsense knowledge is likely derived mostly from the LLMs.\n\n|              | Counter-intuitive | Counter-intuitive | Commonsense | Commonsense |\n|:------------:|:-----------------:|:-----------------:|:-----------:|:-----------:|\n|   **Model**  |     **CI-Obj**    |   **CI-AttrRel**  |   **CS-L**  |  **CS-VL**  |\n|    BLIP-2    |       88.76%      |       25.69%      |    14.45%   |    2.50%    |\n| InstructBLIP |       92.77%      |     **67.26%**    |  **39.96%** |    23.55%   |\n|     LLaVA    |     **98.22%**    |       0.18%       |    30.06%   |  **27.92%** |\n|   MiniGPT-4  |       93.93%      |       5.08%       |    14.63%   |    5.62%    |\n|   mPLUG-Owl  |       97.23%      |       35.33%      |    28.37%   |    17.40%   |\n|     ALBEF    |       90.63%      |       47.01%      |    12.67%   |    0.36%    |\n\n### No new techniques proposed to enhance VLMs\n\nWe would like to emphasize that the main contributions of our work are the benchmark probing dataset and the analysis of existing models for reasoning capabilities beyond visual common sense, which to the best of our knowledge have not been studied, especially on the latest multimodal LLMs. There is a line of similar work focusing on constructing benchmark datasets without trying to improve the models, such as [2] and [3].\n\nTo answer the reviewer’s question on possible techniques to enhance VLMs, an option could be to artificially introduce counter-intuitive images into the training data of these models, but this approach may be too demanding in terms of computing resources. Another approach would be to perform parameter-efficient fine-tuning of these models using counter-intuitive images. This would be our future work to explore.\n\n### Size of the dataset\nPlease note that VQA2.0 and GQA are commonly used for QA model training, whereas the purpose of our dataset is to evaluate pre-trained VLMs’ zero-shot performance on the tasks we defined. For our purposes (i.e., benchmarking, not training), it is not important for the probing dataset to be at the scale of VQA2.0 and GQA. Please also note that recent similar probing datasets have similar sizes to ours. For example, the Winoground dataset [2] has 1600 images. The WinoGAViL dataset [3] has 3500 images. Probing datasets cannot easily scale up because the data typically require human annotation.\n\nAlso, regarding the comment that “similar samples are likely to be included in larger dataset such as VQA2.0 or GQA”, we would like to emphasize that the images in VQA2.0 and GQA are mostly common images whereas our dataset contains counter-intuitive images, which are not easy to be found in VQA2.0 and GQA.\n\n### Missing references\n\nThank you for pointing out these missing references. We will include them when revising the paper.\n\n\n\n\n[1] https://github.com/salesforce/LAVIS\n\n[2] Thrush et al., “Winoground: Probing Vision and Language Models for Visio-Linguistic Compositionality,” in CVPR 2022.\n\n[3] Bitton et al., “WinoGAViL: Gamified Association Benchmark to Challenge Vision-and-Language Models,” in NeurIPS 2022.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "The proposed dataset is new and challenging, and can effectively evaluate VLMs' capability in image understanding.", "reasons_to_reject": "1) The assessment is exclusively conducted on multimodal LLMs, which may not be sufficient. Recent study[1] indicates that most of these models have limited capabilities in image comprehension. For example, these models generate inaccurate image descriptions. In contrast, general VLMs such as ALBEF as mentioned and X-VLM[2] if fine-tuned on VQA2.0 very likely perform better than these multi-modal LLMs on the proposed task. \n\n\n2) This paper introduces a dataset, however, without proposing any new techniques or tricks to enhance VLMs' efficacy in this task.\n\n3) The proposed dataset is tiny (only 1k images). Similar samples are likely to be included in larger dataset such as VQA2.0 or GQA, with evaluation on these large datasets yielding more comprehensive results.", "questions_to_authors": "n/a", "paper_topic_and_main_contributions": "This paper introduce a new dataset named ROME to evaluate the reasoning capability of VLMs to correctly interpret counter-intuitive content. This dataset contains images that defy commonsense knowledge with regards to color, shape, material, size and positional relation. Experiments on the state-of-the-art multimodal LLMs reveal that most of these models are still largely incapable of interpreting counter-intuitive scenarios.", "excitement": 3, "reproducibility": 3, "ethical_concerns": "No", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"Typos_Grammar_Style_and_Presentation_Improvements": "n/a"}, "extra_text": {"Missing_References": "[1] What Matters in Training a GPT4-Style Language Model with Multimodal Inputs?\nhttps://arxiv.org/abs/2307.02469\n\n[2]X2-VLM: All-In-One Pre-trained Model For Vision-Language Tasks\nhttps://arxiv.org/abs/2211.12402"}}, {"review_id": "4PVjJJVXEk", "reviewer": "Reviewer_8GUZ", "summary": "", "questions": "- What was the scheme used to decide whether to retain a generated image or not?\n- Have you considered using some simulated engines (such as Unity) and simply programmatically render the imageries, which can be even more scalable for certain reasoning types (e.g., sizes).\n- For non counter-intuitive commonsense probing, VCR [1] and relevant tasks should’ve already investigated quite a decent amount of effort. How does the probing in this work (on this matter) bring more insights?\n\n[1] Zellers, Rowan, et al. \"From recognition to cognition: Visual commonsense reasoning.\" CVPR-19.", "limitations": "", "rating": 4, "confidence": 4, "soundness": 3, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "### Inter-annotator agreements\n\nWe thank the reviewer for this suggestion and will report these when revising the paper.\n\n\n### The method / scheme of retaining the samples\n\nWe will briefly explain the scheme of retaining the samples here and will include a clear description of the procedure when revising the paper.\n\nWe have two kinds of images: (1) those automatically generated by DALL-E-2 API, and (2) those manually and iteratively created by Bing Image Creator. For images generated by DALL-E-2, we go through three sequential rounds of filtering: (a) an automatic filtering step using CLIP, (b) a manual filtering step to check whether the object in the image can be recognized by humans, and (c) another manual filtering step to check whether the counter-intuitive attribute in the image can be recognized by humans. An image is retained only after it passes all three rounds of filtering.\n\n### Why commonsense and counter-intuitive questions are different\n\nThe commonsense questions and counter-intuitive questions shown in Figure 2 in the paper are different because commonsense questions are testing the models’ understanding of the common attributes or spatial relations of some objects, rather than the attributes or relations shown in a specific image. Therefore, we use “in general” to start these questions and we do not mention “this image” in the question. Counter-intuitive questions, on the other hand, have to refer to the specific image given. Therefore, we use “in this image” in the question.\n\n### Simulated engines such as Unity\n\nWe thank the reviewer for introducing Unity. We were not aware of these simulated engines and we will explore the possibility of using them in our future work.\n\n### New insights compared with VCR and relevant tasks\n\nFor non-counter-intuitive probing, we agree that there has been much previous work. The main difference between our work and previous work in terms of non-counter-intuitive probing is that we perform analysis and offer insights on the latest VLMs, which are built on top of LLMs. Given that these models are very recent, to the best of our knowledge, there has not been any systematic study of their visual commonsense capabilities.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "- The proposed ROME dataset would be a useful testbed for evaluating the models’ capabilities of grasping the counter-intuitive reasoning on common objects.\n- The implementation of leveraging both ViComTe and image generative models is neat and should be well-extendable.\n- The conducted experiments are comprehensive and solid, on various models as well as different aspects of their systematic probing using the proposed testbed.", "reasons_to_reject": "- The inter-annotator agreements on the human annotations need to be reported.\n- Following the previous point, the method of retaining the samples needs to be clearly written, i.e., it is unclear to me whether a majority vote is conducted, or a consensus vote is.\n- From Section 5.3, if there are indeed better prompts or ways of asking the same questions with different query formats to the VLMs, why did the authors not just follow similar schemes to design both the commonsense and the counter-intuitive questions?", "questions_to_authors": "- What was the scheme used to decide whether to retain a generated image or not?\n- Have you considered using some simulated engines (such as Unity) and simply programmatically render the imageries, which can be even more scalable for certain reasoning types (e.g., sizes).\n- For non counter-intuitive commonsense probing, VCR [1] and relevant tasks should’ve already investigated quite a decent amount of effort. How does the probing in this work (on this matter) bring more insights?\n\n[1] Zellers, Rowan, et al. \"From recognition to cognition: Visual commonsense reasoning.\" CVPR-19.", "paper_topic_and_main_contributions": "This paper collects a sizable dataset that focuses on counter-intuitive commonsense reasoning. Specifically, they perform an algorithmic (heuristics-based) approach to generate counter-intuitive descriptions (mostly among objects) and leverage generative models such as DALLE-2 (with manual and algorithmic refinements) to generate imageries based on these descriptions. They evaluate numerous state-of-the-art vision-language models on their benchmarks, suggesting significant rooms of improvement, especially in the counter-intuitive attribute and relation recognition of objects.", "excitement": 4, "reproducibility": 3, "ethical_concerns": "No", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {}, "extra_text": {"Typos_Grammar_Style_and_Presentation_Improvements": "- L440-L450 is not really necessary. Simply describing that the metric requires the model to be correct in both directions is sufficient and well-justified.\n- Same as above, Section 4.2.2 could’ve simply just described the terms of the metrics, without those over-specified notations."}}]

Author rebuttal:

Meta-review: This paper introduce a partially machine-generated new dataset, named ROME, to evaluate the reasoning capability of VLMs to correctly interpret counter-intuitive content. All reviewers agreed that evaluating VLMs under counter-intuitive content is interesting. However, the reviewers also pointed out that the scale of the dataset is relatively small and raised some concerns on the annotation process.
Decision: Accept-Findings