Abstract

Introduction
11
Large Language Models (LLMs) have demonstrated re- markable performance in natural language generation and understanding [1, 2]. To advance the capabilities of intelli- gent systems, several works [3, 5, 6] have extended LLMs to Multimodal Large Language Models (MLLMs) by in- corporating visual elements.  Although these MLLMs have achieved significant advancements in established bench- marks for image captioning and Visual Question Answer- ing (VQA), their proficiency in recognizing and under- standing novel entities in the open world remains unex- plored.  Moreover, existing benchmarks are not suitable anymore since MLLMs are trained on large amounts of data that include known objects.  Consequently, develop- ing more challenging benchmarks featuring novel objects is crucial for a thorough evaluation of MLLMs’ capabilities. In everyday life, we often come across unfamiliar ob- jects.  When describing these new objects, we typically refer to characteristics such as shape and color.  To bet- ter understand and remember these novel objects, we in- stinctively draw comparisons to known objects, using our existing knowledge base instead of relying solely on the ob- ject’s attributes. This necessitates the ability to understand known objects comprehensively. Inspired by the aforementioned ideas, we construct a new image dataset representing fantasy objects that do not exist in reality.  These novel objects are concocted by blending attributes of two or three known objects, utilizing image generation models, in order to evaluate whether current MLLM can identify that the presented objects are not real and can associate them to known concepts.  Figure 1 pro- vides an example. Our contributions are as follows: • We introduce a challenging new benchmark for MLLMs, comprised of novel objects that do not exist in the real world. • We evaluate state-of-the-art MLLMs using our new benchmark and observe that these models struggle with novel objects, highlighting the importance of further de- veloping MLLMs for open-world understanding.
Related Work
2
Multimodal Large Language Models Due to the dif- ficulty in obtaining image-text pairs, most MLLMs are developed using existing visual models, such as CLIP [7], and language foundation models, like LLaMA [2] and Vi- cuna [8], rather than being trained from scratch.  BLIP-2 [3] proposes a Querying Transformer (Q-former) to fuse vision and language module, which has been adopted by other works, including MiniGPT-4 [6].  For the development of instruction-following MLLMs, LLaVA [5] develops a mul- timodal instruction-following data, which is widely used by subsequent research [6, 4].  For our experiments, we test three well-known instruction-tuned MLLMs: LLaVA-1.5, MiniGPT-4, and InstructBLIP. Traditional Benchmark Traditional Image caption- ing and VQA benchmarks contains large amounts of di- verse images, providing robust evaluation for developing MLLMs.  Image captioning requires models to recog- nize the information in an image and describe it accu- rately.  One of the most famous image captioning dataset is MSCOCO [9].  VQA tasks require models to well un- derstand the information in the image and sometimes need reasoning on the visual cues.  A lot of VQA datasets are constructed to cover diverse scenarios, including COCO- QA [10], VQAv2 [11] and Visual Genome [12]. However, a limitation of these traditional benchmarks is that most images within these datasets contain existing knowledge, such as known objects and common sense, which may has already been included in the training data for MLLMs. Benchmark with Novel Elements The WHOOPS benchmark introduces images that defy common sense, with the objective of evaluating whether the generation by MLLMs exhibits bias towards common sense and assesses their reasoning ability to identify unusual parts.  Addi- tionally, the ISEKAI dataset [13], featuring images with fantasy objects, aims to assess the few-shot learning capa- bilities of MLLMs.  To assess the ability of LLMs when meeting new knowledge, KnowGen [14] is proposed.  It is a knowledge generation method by altering entity at- tributes and relationships to create entities not existing in the world.  A Question Answering (QA) benchmark named ALCUNA is constructed using KnowGen. Our work is in- spired by KnowGen.  The main distinction lies in our focus on MLLMs.  Compared to the ISEKAI dataset, we aim to synthesize novel entities with a more complex mixture of objects.
3
Image Synthesis
In this section, we introduce how to design new objects and synthesize the corresponding images.
3.1
Prompt construction
In our preliminary study, we employed a straightforward procedure to synthesize images.  Our approach involved prompting ChatGPT to design novel objects through the combination of existing entities, with a specific focus on animals and fruits.  It is instructed to provide detailed descriptions of combining these objects to generate novel ones.  This process yielded 100 novel animal descriptions and 50 fruit descriptions. ###User:  Generate visually rich prompts for Text-to-Image models, envisioning novel fruits by amalgamating physical attributes of diverse fruits. Ensure the following: 1. Keep the prompts non-threatening. 2. Avoid duplicating existing examples. 3. Cultivate creativity in the examples you generate. 4. PLEASE eliminate redundancy and background details unrelated to amalgamating appearance features, like taste and impression. The content should include shape, surface, and flesh. 5. Steer clear of repetition within 6. Aim for maximum diversity in the generated creature designs. 7. Each created fruit combines the characteristics of 2 or 3 kinds of fruits. Example: Melonkiwipear: a pear-shaped fruit that combines the fuzzy skin of a kiwi with the juicy, red, transparent flesh of a watermelon. ###ChatGPT:
3.2
Data construction
In the second step, we prompt the text-to-image model to synthesize images, utilizing OpenDalleV1.1 [15].  Due to the potential mismatch between the descriptions generated by ChatGPT and the requirements of the image generation model, we employed Promptist [16] to generate refined prompts.  Both the original and refined descriptions were fed into the model, resulting in the generation of 9 images for each description. Finally, to ensure the quality of images for our prelim- inary experiment, we manually selected 84 satisfactory images based on our judgment of high quality.
4
Experiment
4.1
Implementation
We employed beam search for inference in our evalua- tions.  For each model, the beam width is set to 5, and the length penalty is set to 1. The implementation is based on the HuggingFace Transformers library.
4.2
Compared methods
We conducted tests on three well-known MLLMs: LLaVA-1.5-7b [5], InstructBLIP [4], and MiniGPT-4 [6], using our selected images.
4.3
Evaluation metrics
The evaluation focused on two main aspects: Entity Identification.  The first test aimed to assess the models' capability to identify whether the entities within the synthesized images were artificially generated. Recognition of Unreal Features.  We e evaluated whether the models could recognize the source or origin of the features if we inform the models that the entities within the images were unreal. To achieve  these  goals, we design 3 ques- tions/instructions: 1) What is the {ENT} in this image? 2)Describe this image. 3) The {ENT} in the image is a combination of multiple {ENT}s. Please respond with only the names of the {ENT}s, without additional information. {ENT} will be replaced to be either "animal" or "fruit" depending on the image being evaluated.
4.4
Results
What is the {ENT} in this image?  The objective of this question was to assess whether the models could identify that the entities in the images did not exist in the world. The responses are checked manually.  We found that all models provided accurate but incomplete answers by cor- rectly naming the main entity.  For instance, if presented with an image featuring an elephant with eagle wings, the models will answer "'elephant'' without expressing doubt about the synthetic nature of the entity. It is observed that the responses were limited to naming the primary entity and did not explicitly acknowledge the artificial or unreal characteristics of the image.  One plausible explanation for this limitation is the lack of diversity in the current instruction-tuning dataset, leading the models to overlook expressing doubts about the authenticity of the entities, as discussed in a recent work [17]. Among the analyzed images, responses for 52 out of 64 animal images were consistently aligned across the chosen models, whereas responses for 7 out of 20 fruit images were consistent.  One of the inconsistent examples is shown in Table 1 (col 2).  The result suggests that our synthesized images for fruits are more confusing for the models. Image Captioning We prompt models with "'Describe this image" for image captioning, aiming to assess their ability to comprehend synthesized entities and recognize unusual parts within them.  Due to the lack of ground truth captions for synthesized images, automatic evaluation of this task is unfeasible. Upon observation, we noted a significant issue of hallucination in this test.  Most responses do not mention the unusual parts present in the synthesized entities. Even in cases where a model recognized these distinctive features from different entities, it often generated incorrect captions that include hallucinations, shown in Table 1 (col 3). This
Table 1     Col 2 shows a case where models have varying opinions on the fruit name. Col 3 demonstrates that the models successfully recognize elements of a parrot and a leopard, while they do not not notice that these elements constitute a single entity in the captions. Col 4 showcases a component analysis. The features of the cheetah are obvious, while the features of a kangaroo are less distinct, with its characteristics mainly manifested in its movements and paws. highlights a challenge in the models' ability to accurately capture and describe the unique elements of the synthesized objects. Component The final test aims to assess whether models could correctly identify the components of synthesized en- tities when explicitly informed that the entities depicted in the images were synthesized.  The results for prediction accuracy are presented in Table 2.  Notably, InstructBLIP displayed the highest recall and F1 score, while LLaVA- 1.5-7b achieved the best precision.  It is noteworthy that recalls for all models were close to 50%, indicating that one of the components could be correctly recognized in most cases. Table 1 (col 4) shows an example.
Table 2  Average precision, recall and f1 score on the prediction of components
5
Conclusion
This work is a preliminary study on the ability of three well-known MLLMs on images with synthesized objects. We conducted experiments to assess the ability of MLLMs in understanding novel objects combined with familiar en- tities.  Through the experiments, we found that 1) these MLLMs failed to express doubt about the presented ques- tions.  2) The observed prevalence of hallucination indi- cates that these MLLMs can identify known elements but often lack a comprehensive understanding of the overall entity. Limitation The dataset size in our experiments is rela- tively small, leading to a less convincing conclusion.  In our future work, we aim to devise methods to expand both the size and diversity, enabling a more comprehensive analy- sis.  Additionally, most evaluation processes are conducted manually, incurring both time costs and subjective judg- ments.  To enhance the evaluation process, we are con- sidering the utilization of well-developed LLMs, such as ChatGPT, which will be explored in our future experiments.