Behind the Magic, MERLIM: Multi-modal Evaluation Benchmark for Large Image-Language Models

## Abstract

Abstract Large Vision and Language Models have enabled significant advances in fully supervised and zero-shot visual tasks. These large architectures serve as the baseline to what is currently known as Instruction Tuning Large Vision and Language models (IT-LVLMs). IT-LVLMs are general-purpose multi-modal assistants whose responses are modulated by natural language instructions and visual data. Despite this versatility, IT-LVLM effectiveness in fundamental computer vision problems remains unclear, primarily due to the absence of a standardized evaluation benchmark. This paper introduces a Multi-modal Evaluation Benchmark named MERLIM , a scalable test-bed to assess the capabilities of IT-LVLMs on fundamental computer vision tasks. MERLIM contains over 300K image-question pairs and has a strong focus on detecting cross-modal “hallucination” events in IT-LVLMs. Our results bring important insights on the performance of state-of-the-art IT-LVMLs including limitations at identifying fine-grained visual concepts, object hallucinations across tasks, and biases towards the language query. Our findings also suggest that these models have weak visual grounding, but manage to make adequate guesses from global visual patterns or language biases contained in the LLM component.

## 1 Introduction

Figure: Figure 1: IT-LVLM ”hidden” Hallucinations. On the left sub-figure we show the prediction of InstructBLIP (Liu et al., ), despite the small size of the tennis racquet and the relatively cluttered scene, we get an (apparently) correct response including the racquet object (we highlight in green font true positives and in red font false positives). After a subtle image edit (right sub-figure), the response remains the same although there is no visual grounding for the tennis racquet. The box around the racquet object was drawn for easier visualization, but it is not part of the original visual input.
Refer to caption: extracted/5654323/images/racquet_org.png

Large-scale transformer networks have significantly advanced the field of Natural Language Processing (NLP) (Brown et al., ; Devlin et al., ; Liu et al., ; Raffel et al., ). Large Language Models (LLMs) have demonstrated remarkable zero-shot performance in a wide range of NLP tasks (Lin et al., ; Min et al., ), and their success has resulted in a fundamental shift for NLP research, moving from task-specific pre-training to task-agnostic representation learning. Similarly, in the image domain, vision transformers trained on large-scale image collections have achieved state-of-the-art performance on multiple fully supervised tasks (Dosovitskiy et al., ; Fan et al., ; Liang et al., ). The integration of a language stream has also demonstrated empirical benefits in fundamental computer vision tasks such as recognition (Alayrac et al., ; Jia et al., ; Radford et al., ) and detection (Chen et al., ; Du et al., ). Ultimately, these advances in computer vision and NLP fields have converged in the development of Large Vision and Language models (LVLMs) (Alayrac et al., ; Kim et al., ; Li et al., ; Nichol et al., ; Radford et al., ; Yu et al., ), which have achieved state-of-the-art performance on several downstream visual tasks while enabling significant improvements in the zero-shot setups.

Following the success of these large-scale multi-modal transformers, a new trend has emerged which involves integrating frozen LLMs and vision transformers by means of a smaller neural network (Bai et al., ; Dai et al., ; Dong et al., ; Huang et al., ; Li et al., ; Liu et al., ; Peng et al., ; Research, ; Tsimpoukelli et al., ; Zhai et al., ; Zhu et al., ). This straightforward modification only requires the training of a fusion module to interface between the frozen modality encoders. The resulting model requires significantly less computational effort to train, and enables querying image data with natural language, thus providing a contextualized natural language response through the frozen language decoder (Tsimpoukelli et al., ).

Such models are known as Instruction Tuning Large Vision and Language models (IT-LVLM), as their zero-shot capabilities are mediated by a natural language input that instructs the model about the expected output (Huang et al., ). Although IT-LVLMs are intended to be general-purpose instructional assistants, there is still no standard test-bed to assess their zero-shot effectiveness in fundamental computer vision tasks. The presence of an LLM introduces additional challenges, among them the inherited tendency to Hallucinate (Huang et al., ; Zhang et al., ), resulting in the emergence of language outputs that can not be grounded to the visual input.

In this paper we take an in depth look at the capabilities of IT-LVLMs for fundamental computer vision tasks, while assessing the impact of hallucination events in the model’s performance. Our proposal has two key elements: First, instead of presenting the model with a fixed set of possible answers, we allow the IT-LVLM to reply in natural language and then parse the open-set IT-LVLM prediction. Second, to better analyze the hallucination issue, we propose an inpainting procedure that subtly manipulates both the image and its corresponding ground-truth. This technique enables us to identify two types of hallucinations: (1) Regular Hallucinations, which are outputs that do not correspond to the image, and (2) Hidden Hallucinations, which appear to be true positives but lack visual grounding. We illustrate these two phenomena in Figure 1, where an apparently correct prediction, such as ”Tennis Racquet,” lacks visual grounding in the edited image, thereby categorizing it as a ”hidden hallucination”.

We compile our image edits, corresponding ground-truths, and parsing strategy into MERLIM (Multi-modal Evaluation benchmaRk for Large Image-language Models), a novel test-bed aimed at empirically evaluating IT-LVLMs on core computer vision tasks including object recognition, instance counting and identifying object-to-object relationships. Our benchmark serves as a straightforward and extensible evaluation strategy of core computer vision tasks on well-established image datasets like MS-COCO (Lin et al., ), LVIS (Gupta et al., ), and Visual Genome.

Our paper brings the following contributions: (i) A standardized test-bed to assess the zero-shot effectiveness of IT-LVLMs on fundamental computer vision tasks, including Object Recognition, Object Counting, and Inter-object Relationship Understanding. (ii) Through extensive empirical evaluation, we identify and quantify ”hidden” hallucination errors which are out of the scope of other benchmarks, we observe this hallucination unfairly increase the accuracy of IT-LVMLMs. (iii) We conducted a thorough analysis of language biases through extensive experiments using a diverse set of carefully selected question prompts. Our observations reveal that IT-LVLMs exhibit a strong bias toward the language tokens of both the question and answer.

## 2 Related Work

The widespread adoption of LLMs and their zero-shot capabilities (Brown et al., ; Chiang et al., ; Devlin et al., ; Liu et al., ; Raffel et al., ) has led to an increased interest in the development of LVLMs (Jia et al., ; Pham et al., ; Radford et al., ; Yu et al., ; Zhai et al., ). LVLMs typically build upon metric learning, estimating a similarity measure across modalities, making these architectures effective in inherently multi-modal tasks such as VQA and text-image retrieval.

#### Visual Instruction Tuning

The effectiveness of LVLMs, coupled with the unprecedented availability of large-scale data in the visual domain, has led to the development of IT-LVLMs. Unlike LVLMs, IT-LVLMs allow to prompt a joint language and visual representation and then generate results with a frozen language decoder. This procedure preserves the multi-modal capabilities of the LVLM, and enables the use of natural language queries as an instruction set that modulates the expected output(Alayrac et al., ; Tsimpoukelli et al., ).

A pioneer work exploring the few-shot learning abilities of frozen language models and vision was presented by Tsimpoukelli et al. (Tsimpoukelli et al., ) where the image input is encoded into a feature set and inserted as a prefix in the text stream, thereby conditioning the responses of the language network. Flamingo (Alayrac et al., ) introduced an auto-regressive text model conditioned on visual input, by using gated cross-attention. The tokens from the perceiver architecture (Jaegle et al., ) are fed into a frozen language model. This Gated cross-attention was extended by BLIP-2 (Li et al., ), linking the T5 language model and its visual stream (Raffel et al., ) with the Q-Former module, thus improving the modality alignment at training time. Recent work has continued to explore the visual-to-language alignments on IT-LVLMS, either by improving the training and annotation data (Liu et al., ; Zhu et al., ), or by introducing large-scale data on the training of the IT-LVLM (Bai et al., ; Dai et al., ).

#### VQA Benchmarks & Datasets

Many existing benchmarks and datasets for IT-LVLMs are designed for the standard VQA tasks, including a pre-defined set of questions that can be answered by looking at image or video data (Antol et al., ; Chen et al., ; Fu et al., ; Goyal et al., ; Gurari et al., ; Hudson & Manning, ; Zhao et al., ). Although these benchmarks provide a direct evaluation of the multi-modal capabilities of LVLMs, they do not assess if the reply originated from the global visual context of the image or from the local visual patterns that effectively ground the prompted question. In contrast, MERLIM focuses on querying and attributing the response of the IT-LVLM to a specific set of object instances in the image by means of small image alterations and direct verification of the same query on the modified visual data. Moreover, we explore the connection between these replies (original image and edited image) and the potential hallucinations from the LLMs component, as some responses can only be attributed to the LLM hallucinating objects.

Perhaps the most similar benchmarks to ours are POPE (Li et al., ) and NExT-GQA (Xiao et al., ), as both these benchmarks aim at finding direct visual groundings related to the language query. While POPE proposes to prompt for the presence/absence of a single instance to evaluate hallucinations (in the form of a false positive), we directly modify the visual ground-truth and can verify if the predictions have an effective visual grounding. NExT-GQA approximates the grounding of a prediction by the accurate temporal localization of the event. Despite the straightforward proposal, contemporary research (Otani et al., ) has shown that temporal action localization commonly suffers from biased predictions, in particular, this bias emerges directly from the language stream.

## 3 MERLIM

MERLIM’s main goal is to assess the effectiveness of Instruction-Tuning Large Visual Language Models (IT-LVLM) in fundamental computer vision tasks, while simultaneously evaluating if the responses have an effective visual grounding. In practice, we design an evaluation scheme that queries the IT-LVLM with a set of predefined language questions. Then, we parse the text responses, transforming the natural language output into a suitable prediction format for each task. This scheme enables us to measure the performance of the IT-LVLM using standard image evaluation metrics, such as accuracy, precision, recall, and F1 metrics. Across all tasks, MERLIM contains a total of 300.664 unique image-question pairs. Table 1 outlines the sizes of the query set and the image data for all the selected tasks.

#### Notation & Definitions.

A pre-trained IT-LVLM ($h$) takes as input an image ($i$) and a language query ($l$), producing a language response ($r$). Formally: $r=h(f(i,\theta_{f}),g(l,\theta_{g}),\theta_{h})$, where $f$ is an image encoder with parameter set $\theta_{f}$, $g$ is a language encoder with parameter set $\theta_{g}$, and $\theta_{h}$ are IT-LVLM specific parameters that integrate the two modalities. In MERLIM, we define a fixed language query set ($L$), where $l\in L$, and transform the open set output $r$ into a set $r^{\prime}\in K$ where $K$ can be directly matched to the closed-set ground truth of fundamental computer vision tasks.

**Table 1: MERLIM Benchmark. We incorporate three core visual tasks to benchmark IT-LVLMs: Object Recognition, Object Counting, and Relationship Understanding. We leverage images from the validation set of MS-COCO (Lin et al., ) and create edited versions of these images (”Edited” column). Moreover, we extend the MS-COCO ground-truth classes by incorporating the class labels present in LVIS (Gupta et al., ) and include the relationships proposed in the overlapping set of Visual Genome (Krishna et al., ).**
| Task | Dataset |  |  |  |  |  |  |
| --- | --- | --- | --- | --- | --- | --- | --- |
| Response<br>Type | Prompts<br>Per Image | Images | Labels |  |  |  |  |
| Original | Edited | MS-COCO | LVIS | V. Gnome |  |  |  |
| Object Recognition | Set/Enumeration | 5 | 4.183 | 31.373 | ✓ | ✓ | ✗ |
| Object Counting | Numeric | 2 | 4.183 | 31.373 | ✓ | ✗ | ✗ |
| Object Relationship Underst. | True / False | 4 | 1.240 | 5.630 | ✓ | ✗ | ✓ |

#### Datasets and Tasks.

We focus our study on three core computer vision tasks: (i) Object Recognition, (ii) Object Relationship Understanding, and (iii) Object Counting. The primary source of data for our Benchmark is the validation set of MS-COCO, where we can directly evaluate the Object recognition and instance counting. For the Reasoning Task, we resort to the labels provided on the validation set of Visual Genome (Krishna et al., ).

Considering the open nature of the vocabulary on IT-LVLMs responses, some correctly grounded visual objects could be labeled as false predictions if their class name is not included in the $80$ Categories of MS-COCO ground truth labels. To alleviate this undesired penalty, we complement the MS-COCO ground-truths with the labels provided by the LVIS dataset (Gupta et al., ). This extended ground-truth allows us to compare against a richer vocabulary.

#### IT-LVLM Hallucinations.

In MERLIM, we assess the effective visual grounding of the IT-LVLM responses through two separate tests. First, we directly compare the responses against the dataset ground-truth to identify Regular Hallucinations, which occur when the predicted noun is not present in the image ground truth. Second, we edit images such that one of the object instances disappears and then compare the IT-LVLM responses when presented with the original ($i$) and edited ($\hat{i}$) versions. This allows us to identify Hidden Hallucinations, which correspond to correct predictions that lack any visual grounding. These errors are beyond the scope of any other multi-modal hallucination benchmarks like (Li et al., ; Xiao et al., ). Figure 1 illustrates this process: the original image (left) includes a tennis racquet (near the leg of the child located on the far left), while in the edited version (right), the tennis racket was inpainted. Despite the absence of the racquet, InstructBLIP with Vicuna7B continues to infer its presence. To minimize the alterations to the original image, we remove only one object instance at a time. We use the inpainting method proposed by Li et al. (Li et al., ) to fill in the ground-truth segmentation mask according to the image context. The edited validation set expands to a total of $31373$ inpainted images obtained from the $4183$ original images in MS-COCO val. We empirically verify that our edits do not significantly alter the global or local appearance of the image ($\hat{i}$), we include a detailed analysis of this evaluation in the supplementary material.

### 3.1 Object Recognition

We evaluate the object recognition capabilities of IT-LVLMs on the MS-COCO dataset by adapting its ground-truth for the multi-label classification task. If at least one instance of class $c$ is grounded in image $i$, that indicates the image contains the class $c$, and is therefore expected in $r$. In this task, we design a query set for the IT-LVLM requesting a list of all the visible objects in the image.

#### Prompt Construction.

To further analyze the bias towards a particular language prompt, we formulate 5 similar prompts that instruct the IT-LVLM to list the objects present in the images. Our set $P$ for the object recognition task is detailed in the following listing:

[backgroundcolor=blue!7]
1. \sayItemize the things seen in the image.
2. \sayIdentify the objects within the image.
3. \sayDetail the items shown in the picture.
4. \sayList the objects that appear in the image.
5. \sayEnumerate the items visible in the picture.

#### Evaluation.

We identify all the nouns in $r$ and compare them against the ground truth labels of MS-COCO. To do this, we split the sentence by words and determine if it is a noun or not using the spaCy library (Honnibal & Montani, ). We discard words that refer to places or areas like room, street, square, etc. We consider compound nouns as one, for instance, ”the back of the car” and ”ice cream”. We designate a prediction as a True Positive if the predicted word matches or is a synonym of a ground truth element. Likewise, a prediction that does not match any ground=truth label or any of its synonyms is labeled as a False Positive. Finally, any ground truth class that can not be found in the prediction (including their synonyms) is regarded as a False Negative. These definitions enable us to calculate precision, recall, and F1 metrics for all IT-LVLMs.

To better manage the extensive vocabulary output of IT-LVLMs, we also utilize the WordNet hierarchy (Miller, ), and the API provided by ChatGPT (OpenAI, ) to identify potential synonyms for elements in $r$. We pre-compute a corpus of noun-to-noun relations totaling 334608 tuples and stored them, future changes in ChatGPT model/API will not affect our benchmark predictions. On occasions, we discover unsuitable outputs, e.g., \sayThere are many objects or \sayIt is a beautiful scene, the recall and precision for such responses are defined as 0. For completeness, we also report the amount of such instances in the supplementary material.

### 3.2 Inter-object Relationship Understanding

After evaluating the object recognition capabilities, we proceed with a finer-grained task, and evaluate the capability of these models to infer simple inter-object relationships. To this end, we use the subset of COCO images that overlaps with the Visual Gnome labels. On this subset, we query the model about object relationships with visual grounding in the original ($i$) and edited images ($\hat{i}$).

Figure: Figure 2: Results for the description task. We report the F1 Score on the original images with the set of five prompts. We highlight that BLIP-2 and Kosmos-2 are the only IT-LVLMs that obtain a similar performance across the five prompts. The best performance for any prompt is attained by InternLM-XComposer2-VL and xGen-MM, critically these models exhibit 14.73% and 26.50% variability between their best score and the worst, respectively.
Refer to caption: extracted/5654323/images/description_f1_ORIG_LVIS.png

#### Prompt Construction.

Since the IT-LVLM outputs are much harder to constrain and parse in this task, we only formulate questions with a yes or no response. We also verify if the described inter-object relationship has a visual grounding or not. For instance, for the images in Figure 1, we could query the model about the relationship between the tennis racquet and the child: \sayDoes the child have a tennis racquet?. Since Visual Genome does not provide, questions with this exact structure, we use the ChatGPT API (OpenAI, ) to build a natural language question with a Yes/No answer:

[backgroundcolor=blue!7]
Authors: \sayCorrect the mistakes in the following question: ’skateboard has wheels?’ Please, just write the correct question in present tense and consider it is a yes or no question.
ChatGPT API: \sayDoes the skateboard have wheels?

#### Relationship Ambiguity.

It is challenging to direct the IT-LVLM attention to a specific instance when there are multiple objects of the same class. This makes the visual grounding assessment ambiguous, as the target relationship might randomly exist among two other entities with the same semantic classes. We mitigate this ambiguity by substituting nouns that have two or more instances in the image with another class noun. We follow two strategies to select the new object categories: (1) We randomly sample another object category from the ground truth classes of MS-COCO which are not included in the image ground truth. We name this set the Random Set. (2) We ask the ChatGPT API (OpenAI, ) to select an object category to generate a meaningful relationship (in terms of language). This set of relationships is named the Curated Set. The expected answer to the queries in any of these sets are always negative, as the relationship no longer has a valid visual grounding.

The main difference between the random and curated sets is that, the first one can generate relationships that do not make sense and can be trivially evaluated using an LLM, for example, \sayDoes a clock have wheels?. Meanwhile, the curated set has more plausible relationships, which are better suited for multi-modal data, for example: \sayDoes the water bottle have a sticker?. We provide the results of both sets to better analyze the influence of language biases, and possible uni-modal (language only) shortcuts taken by the IT-LVLM. Finally, we also evaluate the response for a negative version of the query. Again, we use the ChatGPT API to generate negative questions based on the original queries. For instance, \sayDoes a bicycle have wheels? is the negative form of \sayDoes a bicycle not have wheels?.

Figure: Figure 3: Results on the original and edited sets. We compare the precision of IT-LVLMs on the edited and original image sets across two prompts. To analyze the hidden hallucination problem, we focus on the subset where the image inpainting removes an entire category. Notably, all methods lose performance on the edited image set.
Refer to caption: extracted/5654323/images/description_prec_in_orig.png

#### Evaluation.

Since the model’s answers are restricted to Yes / No questions, we compute the accuracy (Acc) of the IT-LVLMs responses. Furthermore, we compare the accuracy between the original and edited image sets to assert how much the model’s answers rely on robust visual grounding.

### 3.3 Object Counting

Finally, we also evaluate the counting capabilities of IT-LVLMs. Although this task is common in VQA benchmarks, even state-of-the-art methods struggle to provide accurate results. In particular, the counting task requires the models to have strong object recognition and instance detection capabilities that enable them to count in a discrete fashion (Trott et al., ; Zhang et al., ).

#### Prompt Construction.

We instruct the model to count the objects in the original and the edited images and require its response to be strictly a number. Unlike other tasks, we share a single prompt for the entire set \sayHow many ¡object_category¿ are there? Just answer the number.. As a second step, we check the consistency of the answers. We formulate yes or no questions based on previous answers over the very same image. For instance, if the model responds \sayThere are five books, on the second step we prompt: \sayAre there five books? using the very same image.

Although the task is very challenging, we focus on the consistency exhibited by the IT-LVLM on the follow-up questions, and the responses obtained from the edited images (which should have exactly 1 less object), even if the actual number of objects does not match the ground-truth, it would be expected for the secondary query to be consistent with the initial one. This query scheme helps us understand if the IT-LVLM leverages the same visual patterns despite being queried in syntactically different ways.

#### Evaluation Scheme.

As we instruct the model to deliver a numeric answer, we compute the mean and standard deviation of the absolute error from the original and edited sets. Additionally, we calculate the accuracy metric for secondary questions designed to return Yes/No responses.

## 4 MERLIM Results

We proceed with the experimental assessment of several representative state-of-the-art Instruction Tuning Large Visual Language Models, following the protocol outlined in MERLIM. Specifically, we evaluate BLIP-2 (with FlanT5xl LLM) (Li et al., ), InstructBLIP (with Vicuna-7B v1.1, Vicuna-13B v1.1 and FlanT5xl LLMs) (Dai et al., ), MiniGPT-4 (with Vicuna-7B v0 and Vicuna-13B v0 LLMs) (Zhu et al., ), LLaVA-1.5 (with Vicuna-7B v1.5 LLM) (Liu et al., ), xGen-MM (with Phi-3 Mini 3.8B LLM) (Research, ), Kosmos-2 (with MAGNETO LLM) (Peng et al., ), InternLM-XComposer2-VL (with InternLM-7B LLM) (Dong et al., ), and Qwen-VL-Chat (with Qwen-7B LLM) (Bai et al., ). We outline and analyze the results across the three evaluation tasks in MERLIM.

### 4.1 Object Recognition

IT-LVLMs aim to deliver coherent, detailed, and visually grounded language responses based on the provided images and questions. To assess their effectiveness, we first evaluate the performance of all IT-LVLMs reporting the F1 Score in MERLIM, for each of the five proposed questions. Then, we take a closer look at the precision metric, as it is closely linked to hallucinations events, a lower precision indicates a higher rate of false positives due to object hallucinations. We also perform a detailed analysis for a subset of MERLIM, to quantify Hidden Hallucination events. In this subset, the inpainting strategy has fully removed the visual grounding of an entire category (i.e. there was a single instance of the category). As outlined in section 3, we compare the response on the original image and the edited data to discover hidden hallucinations.

#### Instruction Bias.

Figure 2 outlines the performance (F1 score) of the IT-LVLMs in MERLIM for each prompt. Interestingly, while ChatGPT (OpenAI, ) and Vicuna-33B (Chiang et al., ) consider the prompts semantically identical, only BLIP-2 (Li et al., ), and Kosmos-2 (Peng et al., ) exhibit consistent performance across all five instructions. These models exhibit low standard deviations of 0.59 and 0.67, respectively. Notably, BLIP-2 achieves the best average performance with 40.55%, even outperforming its successor, InstructBLIP. This superior consistency of BLIP-2 over InstructBLIP might stem from it lacking an instruction-aware Q-former module.

The best performance for a single question is attained by InternLM-XComposer2-VL on question 1 with an F1 Score of 47.79%. In contrast, MiniGPT-4 (Vicuna-13B) exhibits the lowest average performance with 25.02%, which might be attributed to the version (v0) of Vicuna and the quality of its multimodal instructional training data.

**Table 2: Results for Inter-object Relationship Understanding. We report the Accuracy (Acc) of the IT-LVLMs in the two sets: one with randomly sampled relationships (Random Set) and another where the relationships are curated by an LLM (Curated Set). We compute the Accuracy with the affirmative ($\bf Acc$) and negative ($\bf Acc^{neg}$) versions of the relationships to validate the models’ answers. We also present the Acc in both sets of COCO images, the edited ($\bf Acc_{ed}$) and Orginal ($\bf Acc_{org}$), and the absolute accuracy difference $\bf\Delta Acc=Abs(Acc_{org}-Acc_{ed})$. The Curated set constitutes a more challenging set for all methods because. No IT-LVLM is superior across all evaluated scenarios.**
| Model | LLM | Random Set | Curated Set |  |  |  |  |  |  |  |  |
| --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- |
| $\bf Acc_{org}\uparrow$ | $\bf Acc_{ed}\uparrow$ | $\bf\Delta Acc\downarrow$ | $\bf Acc^{neg}_{org}\uparrow$ | $\bf Acc^{neg}_{ed}\uparrow$ | $\bf Acc_{org}\uparrow$ | $\bf Acc_{ed}\uparrow$ | $\bf\Delta Acc\downarrow$ | $\bf Acc^{neg}_{org}\uparrow$ | $\bf Acc^{neg}_{ed}\uparrow$ |  |  |
| MiniGPT-4 | Vicuna-7B v0 | 41.13% | 37.35% | 3.78% | 23.77% | 20.67% | 37.38% | 36.77% | 0.61% | 29.29% | 22.36% |
| MiniGPT-4 | Vicuna-13B v0 | 45.35% | 44.81% | 0.54% | 34.30% | 31.87% | 41.01% | 41.81% | 0.80% | 41.06% | 34.28% |
| LLaVA-1.5 | Vicuna-7B v1.5 | 55.59% | 40.37% | 15.22% | 68.85% | 69.00% | 62.06% | 24.87% | 37.19% | 83.26% | 80.18% |
| BLIP-2 | FlanT5xl | 83.62% | 78.61% | 5.00% | 48.80% | 49.38% | 67.84% | 62.04% | 5.80% | 49.77% | 45.88% |
| InstructBLIP | Vicuna-7B v1.1 | 91.17% | 86.27% | 4.90% | 19.60% | 9.91% | 76.75% | 67.87% | 8.88% | 45.31% | 16.34% |
| InstructBLIP | Vicuna-13B v1.1 | 90.32% | 86.98% | 3.34% | 16.24% | 5.58% | 75.99% | 71.49% | 4.49% | 43.52% | 12.17% |
| InstructBLIP | FlanT5xl | 80.69% | 75.10% | 5.59% | 77.16% | 79.49% | 70.15% | 57.19% | 12.95% | 63.90% | 70.98% |
| xGen-MM | Phi-3 Mini 3.8B | 61.31% | 46.45% | 14.86% | 49.61% | 49.06% | 66.36% | 32.89% | 33.46% | 64.41% | 54.48% |
| Kosmos-2 | MAGNETO | 19.34% | 5.33% | 14.02% | 90.47% | 92.73% | 45.98% | 4.42% | 41.56% | 85.05% | 93.98% |
| XComposer2-VL | InternLM-7B | 71.47% | 60.32% | 11.15% | 43.91% | 51.60% | 72.96% | 45.67% | 27.30% | 48.69% | 63.37% |
| Qwen-VL-Chat | Qwen-7B | 92.11% | 88.17% | 3.94% | 15.81% | 3.34% | 81.41% | 76.79% | 4.63% | 42.29% | 4.65% |

#### Object Hallucination.

MERLIM can break down Hallucination events into regular and hidden Hallucinations. To this end, we compare the precision of IT-LVLMs on both the original and edited image sets. Figure 3 presents results for two of the five designed prompts(^1^11We chose Questions 1 and 2 as they are, on average, the best-performing prompts by F1 Score). Regarding regular hallucinations, 5 of the 11 models we tested (Dai et al., ; Dong et al., ; Li et al., ; Liu et al., ; Research, ) achieved a precision near or above 50% on the original image set in at least one of the selected questions. Surprisingly, these models exhibit the largest performance gap on the edited image set, with an average drop of 8.05%. This empirical evidence indicates that the best performing methods are also the ones which suffer the most from hidden hallucinations.

Some replies lack direct visual grounding (as the corresponding visual instance was removed from the image in the inpainting process), however these objects persist in the language prediction. This result suggests that the best performing IT-LVLMs are obtaining an advantage by prioritizing the global visual context or previously predicted language tokens over the effective visual grounding. We provide further evidence to support this observation in Table 5 (Supplementary Material). In this table, we show that the gradient contributions of the previously predicted language tokens are, on average, larger than those of the image. This suggest that top-performing models are improving their performance by hallucinating objects which are largely determined by the initial answer tokens.

Additional results for the Precision and Recall metrics are reported in the supplementary material.

Figure: Figure 4: Results on Object Counting task. Subfigure a) reports the mean absolute error of IT-LVLMs in the original and edited sets. LlaVa v1.5 (Liu et al., ) demonstrates better performance across both sets. On average, BLIP2 and MiniGPT make the worst estimations. Every model reduces its performance in the edited set, indicating a weak correlation between visual grounding and model response. Subfigure b plots the accuracy for the yes/no questions designed to validate answer consistency for the main counting task. LlaVa exhibits remarkable consistency across both sets, significantly outperforming other methods. Other IT-LVLMs perform close to random chance, suggesting a bias towards the question format in their answers.
Refer to caption: extracted/5654323/images/counting_mae.png

### 4.2 Inter-object Relationship Understanding

Following Section 3.2, we continue by evaluating the capabilities of IT-LVLMs to identify fine-grained inter-object relationship using the subset of MS-COCO that overlaps with Visual Gnome as described in Table 1. We query the IT-LVLMs about the relationships in the original and edited image sets. Table 2 summarizes the accuracy in both sets ($\bf Acc_{orig}$) and ($\bf Acc_{ed}$) respectively. Following the described protocols to mitigate the relation ambiguity in Section 3.2, we also report the accuracy of the models in the two sets of relationships. The Random Set (negative relationships are built from any two random objects) and the Curated Set (we verify the relationship is plausible according to an LLM). Finally, we also query the models with the negative version of the two sets of relationships ($\bf Acc^{neg}$).

#### Performance in the edited images set.

Table 2 shows that all tested IT-LVLMs exhibit an average performance drop of 7.49% when transitioning from the original to the edited set. This performance gap, is nearly the same gap in the object recognition task, suggesting that IT-LVLMs may also be hallucinating certain visual relationships due to the global visual context and the inherent biases within the language models. MiniGPT-4 (Zhu et al., ) (with Vicuna-7B or 13B) attains the lowest accuracy overall, although it compensates with the smallest variability between the original and edited sets ($\bf\Delta Acc$). With MiniGPT-4’s LLM and Visual stream being comparable in size to other models, the reduced performance could be attributed to a different Vicuna version (V0 for MiniGPT-4) and a relatively small fusion module ($\theta_{h}$), consisting of a single linear layer. It is noteworthy that no IT-LVLM consistently outperforms the others in all the evaluated scenarios. However, Qwen-VL-Chat (Bai et al., ) exhibits the best performance for affirmative relationships in both original and edited images, regardless of the set (random or curated).

#### Querying with more realistic relationships.

The curated set requires the IT-LVLMs to build a better understanding of visual information to reply about the visual relationships. In short, relationships can not be trivially resolved from the language query (see Section 3.2). As summarized in Table 2, all the evaluated IT-LVLMs significantly reduce their performance in the edited set by 11.63% when compared to the curated set. Furthermore, the average performance gap between the original and edited sets ($\bf\Delta Acc$) has risen to 16.15% more than double the gap observed in the random set.

These results support our hypothesis that some visual relationships are entirely resolved by the language stream, thus artificially increasing the performance of the IT-LVLM. More importantly, visual object relations follow a similar hallucination pattern as the object recognition task, where most of IT-LVLMs favor shortcuts over the language modality, rather than building grounded multimodal representation where inter-object relationship can be modeled and queried.

#### LLMs vs IT-LVLMS.

To get a better perspective on the relevance of language shortcuts on the IT-LVLM input, we conduct a comparative analysis of the performance between IT-LVLMs and LLMs (no visual input) in the Inter-object Relationship Understanding task. We observe that LLMs lag behind IT-LVLMs, however, LLMs can outperform a random baseline despite having no visual data which further suggest language only biases constitute a significant party of the performance of the IT-LVLM. We include the detailed analysis in the Supplementary Material.

### 4.3 Object Counting

In Figure 4, we report the mean absolute error (MAE) and its standard deviation for all tested IT-LVLMs on both the edited (blue) and original (orange) in MERLIM. All IT-LVLMs exhibit a significant increase in error on the edited set. This indicates that the intrinsic error of the models in the counting task is not the only source of error but also a lack of visual grounding in their responses, which leads to hallucinations.

#### Answer Consistency.

To further investigate language biases, we also compute the accuracy metric for the yes/no questions, which validate the answers given by the IT-LVLM in the initial prediction (i.e. Are there N objects?). We highlight that all methods still perform better in the original image set, especially the largest IT-LVLMs, like InstructBLIP over Vicuna-13B. However, the drop in performance remains significant in the edited set. LLaVA (Liu et al., ) exhibits the highest consistency, with over 80% of the replies confirming the initial assessment (regardless if the count was accurate or not) for both the edited and original sets. Similarly, the LLaVA IT-LVLM also has the smallest average error and the smallest standard deviation on the initial counting, clearly outperforming every other IT-LVLM in this particular task.

## 5 Discussion & Conclusions

#### Hidden Hallucinations.

Unlike previous benchmarks, MERLIM stands out for its ability to define and study a novel type of hallucination: hidden hallucinations. These are more challenging to detect because they are seemingly correct predictions but in fact lack a meaningful visual grounding. Our findings suggest that top-performing models might achieve higher scores (in part) due to these hidden hallucinations. Moreover, our analysis also identifies potential causes for this phenomenon: language biases and a lack of robust visual grounding.

#### Prompt Selection.

Despite current advances in LLMs, slight syntactic changes of input prompt heavily condition the language output of the IT-LVLM. In our benchmark most models vary drastically their outputs and associated performance. Although our queries are semantically equal, all the questions had a different average length, and the average length for the same question would also differ across models. We conclude IT-LVLMs responses remain heavily biased towards the selected query set. To foster future research, we will release the queries and results in MERLIM, to encourage the community to further explore the strong discrepancies observed in this paper.

#### Limitations.

MERLIM comprises a total of 300664 unique image-question pairs across all tasks, which may limit its usability for evaluating and prototyping new models. Therefore, we recommend using the subset of MERLIM designed for the Object Recognition Task, where image inpainting removes an entire category. This subset, used in Figure 3, contains 5608 edited and 3037 original images.

#### Conclusion.

This paper introduces MERLIM, a novel multi-modal Benchmark for evaluating IT-LVLMs in fundamental computer vision tasks. Despite the versatility of current IT-LVLMs, we observe an overall poor performance in these fundamental tasks, with multiple failure scenarios associated with weak or non-existent visual grounding of the language predictions. We think MERLIM can be the initial effort to better diagnose Hallucination events, and to create stronger IT-LVLMs where ensuring a solid visual grounding could improve the overall quality of the predictions.

## Appendix A Supplementary Material

To complement the experimental assessments in Section 4, we provide a summary of comparisons across a subset of tasks and evaluation metrics in Figure 7. This is followed by an in-depth analysis of the effects of our inpainting rationale and a direct comparison between IT-LVLMs and their LLM-only counterparts in MERLIM. Additionally, we offer insights into language biases by examining the average number of nouns generated for each of the five proposed prompts in Section 3.1. Lastly, we delve into the object hallucination issues by presenting the full Precision and Recall metrics for the models across the original image set, using the same five prompts from Section 3.1. Additionally, we analyze the gradient percentage corresponding to each input (Image, Question, and Answer) to understand their contributions to the generated answers.

### A.1 Analysis of Inpainted Images

To analyze the visual changes in the images after the inpainting process, we first assess that the selected inpainting strategy induces a minimal change in the global image features and that the inpainted masks effectively hide the object instance from object detectors. We use YOLOv7 (Wang et al., ) to verify the absence of the object in the edited image and ResNet50 (He et al., ) to evaluate the global feature similarity with the original image. For each image in MERLIM, we enforce two essential criteria: (1) YOLOv7 must not generate a detection box with an Intersection over Union (IoU) greater than 0.7 relative to the ground-truth bounding box of the removed instance, and (2) the cosine similarity between the ResNet50 features of the edited image and the original image must exceed 0.7.

For a more in-depth assessment of the visual changes in the inpainted images, we use another box detector, Mask R-CNN (He et al., ), to verify if each predicted box in the inpainted image has a corresponding box with the same class and a similar spatial location (i.e., high Intersection over Union) in the original image.

We perform this analysis for bounding boxes with confidence scores above 0.7 and 0.5, summarizing the results in Table 3.

**(a)**
| IoU | Percentage | # of boxes |
| --- | --- | --- |
| above 0.9 | 85.37% | 259454 |
| 0.8-0.9 | 9.3% | 28250 |
| 0.7-0.8 | 3.1% | 9428 |
| 0.6-0.7 | 1.4% | 4260 |
| 0.5-0.6 | 0.44% | 1324 |
| bellow 0.5 | 0.38% | 1165 |

Out of a total of 762782 predicted boxes in the inpainted image set, we found that only 4455 boxes (about 0.59%) could not be mapped to the boxes detected in the original image with the same class and an overlap of at least 0.5 IoU. In comparison, 726969 boxes (96.8%) in the inpainted images have a matching box with the same class and a nearly identical spatial location (IoU ¿ 0.7) in the original image. Despite the inpainting strategy being relatively under-explored, we conclude that the observed performance gaps in MERLIM should not be attributed to the minimal discrepancies identified by representative image networks (ResNet50, YOLOv7, Mask R-CNN) but rather to hallucination events in the IT-LVLM.

Figure: Figure 5: Number of nouns predicted. We extract the nouns using the spaCy library from the answers of the models from all the formulated prompts on the original image set. It is worth noting that BLIP-2 predicts a consistent number of nouns across all the prompts, unlike the other methods.
Refer to caption: extracted/5654323/images/descrip_leng_ans.png

Figure: (a)
Refer to caption: extracted/5654323/images/description_PREC_ORIG_LVIS.png

### A.2 Object Hallucination

In Figure 5, we provide further evidence of instruction bias in IT-LVLMs. Despite the semantic equivalence of the proposed prompts, all methods, except for BLIP-2 (Li et al., ) and Kosmos-2 (Peng et al., ), generate varying numbers of nouns for each prompt. Figure 6 illustrates that while BLIP-2 produces the shortest answers (in terms of noun count), it consistently achieves higher precision compared to other methods, albeit with lower recall. This indicates that BLIP-2 is less prone to hallucination, likely because it was trained to generate concise captions rather than detailed descriptions. On the other hand, larger models trained with instructional data, such as InstructBLIP and MiniGPT-4 with Vicuna-13B (Dai et al., ; Zhu et al., ), LLaVa (Liu et al., ), and xGen-MM (Research, ), tend to generate longer answers on average. This increases the likelihood of hallucinations, particularly when visual grounding is insufficient, resulting in higher recall but lower precision compared to BLIP-2.

### A.3 LLMs vs IT-LVLMs

**Table 4: Textual Biases. We compare the accuracy of IT-LVLMs in the relationship understanding with the performance of LLMs. Since the LLMs lack visual context we allow the model to reply ’yes’, ’no’ or ’I Don’t know’. We report two accuracy values, one where we penalized every ’I don’t know’ as a false prediction (left side of /) and the other considering only the questions answered with yes or no (right side of /). FlanT5xl outperforms the other opensource LLMs.**
| Model | LLM | Random Set | Curated Set |  |  |
| --- | --- | --- | --- | --- | --- |
| $\bf Acc_{org}\uparrow$ | $\bf Acc^{neg}_{org}\uparrow$ | $\bf Acc_{org}\uparrow$ | $\bf Acc^{neg}_{org}\uparrow$ |  |  |
| Random Baseline | N/A | 33% / 50% | 33% / 50% | 33% / 50% | 33% / 50% |
| LLM Only | ChatGPT 3.5 | 21.27% / 80.01% | 8.05% / 28.83% | 13.06% / 55.80% | 11.57% / 47.78% |
| LLM Only | Vicuna-7B v1.1 | 6.59% / 44.72% | 27.53% / 66.86% | 11.11% / 43.05% | 26.42% / 55.54% |
| LLM Only | Vicuna-13B v1.1 | 7.92% / 75.22% | 2.07% / 25.23% | 7.63% / 55.60% | 4.92% / 47.76% |
| LLM Only | FlanT5xl | 41.82% / 69.47% | 40.06% / 55.84% | 36.10% / 54.65% | 50.74% / 65.80% |
| BLIP-2 | FlanT5xl | 83.62% | 48.80% | 67.84% | 49.77% |
| InstructBLIP | Vicuna-7B v1.1 | 91.17% | 19.60% | 76.75% | 45.31% |
| InstructBLIP | Vicuna-13B v1.1 | 90.32% | 16.24% | 75.99% | 43.52% |
| InstructBLIP | FlanT5xl | 80.69% | 77.16% | 70.15% | 63.90% |

In Table 4, we compare the performance of IT-LVLMs to their corresponding LLMs without visual input. Since the LLMs lack the capability to process visual information, we allow them to answer \saydo not know. Using the original questions from the visual relationship task (denoted as ${\bf Question}$), we prompt an LLM as follows: \say$p_{llm}$ = ${\bf Question}$. Please answer yes, no, or do not know. While LLMs sometimes respond with \sayI do not know, they typically base their answers on the knowledge acquired during language pre-training. With three response options, the random chance of selecting the correct answer is 33%.

For additional evaluation (shown on the right side of the table), we consider only the answers where the LLM responded with either \sayyes or \sayno. Although Vicuna performs below the random baseline, its performance improves significantly when focusing solely on yes/no responses, especially for Vicuna-13B. FlanT5xl exceeds random chance and further narrows the performance gap between LLMs and IT-LVLMs.

We conclude that the language model serves as a strong prior for resolving visual relations, even without image information. These textual priors can act as \sayshortcuts in MERLIM, contributing to the performance difference between the Random and Curated sets, where these biases are less effective. While these textual \sayshortcuts can occasionally align with visual grounding (Hidden Hallucinations), they are often revealed when visual grounding contradicts language-based intuitions.

### A.4 Gradient Analysis

To further investigate the visual grounding of IT-LVLMs, we take follow(Selvaraju et al., ) and compute the proportion of the total gradient attributable to each specific type of input (Image, Question, Answer) for each predicted token. We analyze the model’s output logits before token selection and propagate the gradient to the input by identifying the maximum activation value per output token. Specifically, for each output token, we determine the gradient contribution from each input type and then average these contributions across all answers. To simplify our analysis, we focus on architectures with autoregressive LLMs, such as Vicuna, which explicitly use previous output tokens as inputs for predicting subsequent tokens.

**Table 5: Study of the relevance of the inputs. We analyze the relevance of tokens from each kind of input (Image, Question, and Answer tokens) by computing the portion of the total gradient produced in the input that belongs to the image, question, and answer tokens relative to the outputs. Specifically, for each output, we calculate the gradient contribution from each input type and then average these contributions across all answers.**
| Model | Vis. Tokens | Que. Tokens | $\bf\nabla_{vis}\uparrow$ | $\bf\nabla_{que}\uparrow$ | $\bf\nabla_{ans}\uparrow$ |
| --- | --- | --- | --- | --- | --- |
| LLaVA-1.5 | 575 (All) | 52 (All) | 27.68% | 44.75% | 28.67% |
| LLaVA-1.5 | top 10 | top 10 | 11.47% | 53.28% | 36.68% |
| InstructBLIP | 32 (All) | 10 (All) | 24.89% | 47.45% | 29.04% |
| InstructBLIP | top 10 | top 10 | 19.98% | 52.01% | 29.41% |

As shown in Table 5, the gradient of the visual input is significantly lower than that of the language inputs (Question and Answer). This indicates that LLMs prioritize language tokens over visual ones when predicting answers, leading to hallucinations based on language biases. Additionally, we observe that only a few tokens account for most of the visual gradients. For instance, in InstructBLIP with Vicuna-7B, just 10 out of 30 tokens represent nearly 73% of the visual gradients, making it challenging for specific and strong visual information to be adequately considered. These findings support the results of our previous experiments.

Figure: Figure 7: Results on MERLIM. We showcase a subset of the performance evaluations on MERLIM. Our evaluation metrics include Precision (Prec), Recall (Rec), and F1 Score (F1) for the Object Recognition Task (OR). We also measure the Accuracy at identifying inter-object relationships (IOR) in two sets: one set generates negative examples through random sampling (Random - Sample RS), and the second set has curated relations (Curated Set - CS) where a commercial LLM discards impossible associations, thus forcing the IT-LVLM to use the visual data. We further verify the IT-LVLMs consistency, as we calculate the Accuracy for both affirmative (Acc) and negative (Acc Neg) versions of the instructions describing the object relations.
Refer to caption: extracted/5654323/images/polar_diagram.png

### A.5 Number of unsuitable outputs

On the Object Recognition task models will occasionally provide responses lacking any valid nouns, such as \sayThere are many objects or \sayIt is a beautiful scene. In such cases, we set both recall and precision metrics to 0. The results in Table 6 outline that MiniGPT-4 with Vicuna-7B v0 and BLIP-2 with FlanT5xl produce the largest number of unsuitable answers. Conversely, the InstructBLIP methods consistently generate valid responses with object lists.

**Table 6: Number of unsuitable outputs. The IT-LVLMs occasionally provide unsuitable answers to the five proposed questions for the Object Recognition Task. That is answers without valid nouns, for instance, “There are many objects” or “It is a beautiful scene”. For such instances, we establish both recall and precision metrics as 0.**
| Model | LLM | Num. unsuitable outputs |  |  |  |  |  |  |  |  |  |
| --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- |
| Question 1 | Question 2 | Question 3 | Question 4 | Question 5 |  |  |  |  |  |  |  |
| Original | Edited | Original | Edited | Original | Edited | Original | Edited | Original | Edited |  |  |
| MiniGPT-4 | Vicuna-7B v0 | 17 | 110 | 6 | 28 | 2 | 26 | 10 | 61 | 81 | 414 |
| MiniGPT-4 | Vicuna-13B v0 | 0 | 2 | 0 | 0 | 0 | 0 | 1 | 10 | 0 | 1 |
| LLaVA-1.5 | Vicuna-7B v1.5 | 0 | 1 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 |
| BLIP-2 | FlanT5xl | 11 | 73 | 6 | 58 | 5 | 65 | 21 | 209 | 15 | 96 |
| InstructBLIP | Vicuna-7B v1.1 | 0 | 1 | 0 | 2 | 0 | 0 | 0 | 0 | 0 | 0 |
| InstructBLIP | Vicuna-13B v1.1 | 0 | 2 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 |
| InstructBLIP | FlanT5xl | 0 | 0 | 0 | 1 | 0 | 1 | 0 | 0 | 0 | 0 |
| xGen-MM | Phi-3 Mini 3.8B | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 |
| Kosmos-2 | MAGNETO | 0 | 0 | 0 | 0 | 1 | 12 | 0 | 0 | 0 | 0 |
| XComposer2-VL | InternLM-7B | 4 | 36 | 12 | 180 | 0 | 1 | 0 | 1 | 4 | 2 |
| Qwen-VL-Chat | Qwen-7B | 6 | 38 | 2 | 4 | 0 | 1 | 0 | 4 | 3 | 22 |

## Peer Reviews

confidence: 4
contribution: 3
initial_rating: 6
presentation: 3
questions: Some details on the output processing could still be added to the supplementary: How exactly are the yes/no answers evaluated, e.g. lowercase, remove dots, then compare to the string “yes”? How are the number answers turned into integers?

Why use only one prompt in 3.3?

Figure 4: Why would models be worse on the edited images? If I understood correctly you mask one of the objects and reduce the count by one. It is not obvious to me why it would be more difficult to count 3 apples after one apple was removed, than to count 4 apples.
soundness: 3
strengths: The paper is well written and understandable, it shows all the relevant details. Missing or incorrect grounding of VLMs is a relevant research question and the approach to simply delete objects and check if the model notices the deletion is clever. Using open-ended answers fits to real use cases of VLMs (compared to testing with multi-choice).
summary: Authors propose a new benchmark for “IT-LVLMs”, where the main contribution is to remove objects from images and thereby detect “hidden” hallucinations, where the model answer is correct but not grounded in the image. The benchmark tests object recognition, counting, and object relationship. Authors test multiple VLMs, each on several prompts.
weaknesses: The work is missing examples. There is only one datapoint shown in figure 1. Please add some randomly picked samples for each of the tasks, to give a better qualitative overview of the dataset.

In figure 1, the tennis racket is still somewhat visible even after deletion, at least the edges of the shape can be detected. However authors verify that an object detector cannot easily find those instances in Appendix A1. Maybe looking at some more edits will show in more details how good this editing works. An alternative approach would be e.g. to use image inpainting but this might lead to other artifacts.

In chapter 3.2 the authors go back to yes/no binary choice questions. This is understandable due to difficulties of posing such questions in an open-ended manner, however asking open questions is more fitting to how a VLM would be used in practice. Maybe it would have been possible to evaluate in a similar manner to 3.1 with making the model list relations and parsing the nouns.

As noted by the authors in line 364 some of the models are already somewhat old, e.g. MiniGPT-4 is ~18 months old at the time of submission. Maybe they could have been replaced by newer models like PaliGemma, or LLaVA 1.6 instead of 1.5. However since the focus is the benchmark this is only a small weakness.

In line 369 consider citing the models with model name instead of author name for clarity.

The citations are messy, e.g. in line 116 they need brackets. In line 201 Li et al. is duplicate. This repeats through the entire work. In general use \cite if the authors are the subject of a sentence, otherwise use \citep to put them into brackets.

In line 443 reference the supplementary material directly by chapter as a clickable reference instead of just as “in the supplementary material”.

The total number of unedited images is somewhat low with less than 10k images, however it’s enough for a benchmark.

In summary I believe if the authors add more clarification and examples, the work is valuable for publication.

---

confidence: 4
contribution: 2
initial_rating: 5
presentation: 2
questions: 1. Some typos should be fixed, e.g. in line 023, "IT-LVMLs" -> "IT-LVLMs"; in line 156, "300.664" -> "300,664"; in line 301, delete "!" and "?". The submission should be carefully checked.
2. What is the version of Qwen-VL. Its performance in Figure 2 and 3 is much lower than BLIP-2, Instruct BLIP, which seems strange.
soundness: 2
strengths: 1. The proposed dataset focues on the fundamental object-level  vision tasks for LVLMs, which was seldom explored by previous works.
2. The paper intends to study two types of hallucinations and identify the potential reasons.
3. The authors comprehensively evaluate the popular LVLMs, e.g., BLIP-2, InstructBLIP, LLaVA, in the proposed dataset.
summary: This paper proposes a new dataset  MERLIM to evaluate the zero-shot ability of large vision-language models for fundamental vision tasks, i.e., object recognition, object relationship and object counting. The authors also introduce a inpainting procedure to study the hallucination issues. The authors evaluate the popular LVLMs in the proposed MERLIM.
weaknesses: 1. My biggest concern is the data sources all come from the COCO images, which would lead to the bias for LVLMs.
2. The title does not convey the necessary information of the paper. I suggest the authors to add the key motivations in the submission, such as "for evaluating the hallucination in visual objects".
3. The details of inpating process is not well presents. For example, how to choose the edited objects? And how many objects are edited in an image?
4. From Figure 1, it seems that the edited image lose the natural structual in the edited part. Could the authors provide more examples for the edited images?

---

confidence: 3
contribution: 3
initial_rating: 6
presentation: 3
questions: Do the authors have any plans to expand or adapt MERLIM to include new tasks or modalities as IT-LVLM capabilities evolve?
soundness: 3
strengths: - The newly introduced evaluation aspect *Hidden Hallucinations* can help better understand the Hallucination issue of IT-LVLM.
- The analyses on the performance of state-of-the-art IT-LVMLs are beneficial.
- This paper is clearly-written, with a well-structured presentation of MERLIM, including its detailed design and evaluation methodology.
summary: This paper presents a new evaluation benchmark for Instruction Tuning Large Vision and Language models (IT-LVLMs), dubbed MERLIM, containing over 300K image-question pairs. This benchmark focused on three types of tasks: Object Recognition, Object Counting, and Relationship Understanding. Besides, this benchmark enables to identify two types of hallucinations: Regular Hallucinations and Hidden Hallucinations, the latter of which is newly introduced in this paper. In experiments, the authors have evaluated many IT-LVLMs, obtaining several insights on the performance of state-of-the-art IT-LVMLs.
weaknesses: - The included tasks such as Object Recognition and Object Counting seem to have overlaps with previous benchmarks [1]. It would be better if the authors could include detailed comparisons between the proposed benchmark and existing ones. It may be helpful to more thoroughly analyze the dataset.
- The field of LVLMs is constantly developing. Many models evaluated in the paper may not be that strong nowadays. It would be better if the authors could include more recent and advanced models like LLAVA-NEXT, etc.






---
[cite 1] Ying, Kaining, et al. MMT-Bench: A Comprehensive Multimodal Benchmark for Evaluating Large Vision-Language Models Towards Multitask AGI. In ICML 2024.

---

confidence: 3
contribution: 3
initial_rating: 5
presentation: 2
questions: 1. The article seems to have two focal points, one is to conduct evaluations on fundamental computer vision problems, and the other is on hidden hallucinations. Which is the focus of this paper? From my perspective, you are trying to demonstrate the language bias and limitation of vision groundings in LVLM from the phenomenon called "hidden hallucinations" through several computer vision problems (Object Recognition, Object Counting, and Inter-object Relationship Understanding), is my understanding right?
2. What is the difference between IT-LVLM and LVLM? Why does the article emphasize IT-LVLM instead of directly using the term LVLM which is widely used?
soundness: 3
strengths: This paper propose the concept of hidden hallucinations in LVLM, a phenomenon  that has been overlooked in the community. This brings insights on the performance of state-of-the-art IT-LVMLs including limitations at identifying fine-grained visual concepts.
summary: This paper introduces MERLIM, a novel multi-modal Benchmark for evaluating IT-LVLMs for hidden hallucinations through several basic computer vision tasks.
weaknesses: 1. The writing of this article needs to be strengthened. Figure 1 highlights the finding of "hidden hallucinations", but this is not elaborated in detail in the abstract. I was wondering if "hidden hallucinations" is the key point of this paper?
2. The focus of the article is not clear. The core of the article seems to be the discovery of the phenomenon of hidden hallucinations. However, in the experiments, too much emphasis appears to have been placed on the mention of fundamental visual tasks. Nevertheless, there are already benchmarks for evaluating fundamental visual tasks nowadays.

---

confidence: 5
contribution: 2
initial_rating: 3
presentation: 2
questions: - Regarding the lack of open-sourcing the benchmark data, could the authors explain the rationale behind this decision?  


- How does MERLIM compare to other existing benchmarks in terms of its ability to detect and quantify hallucination events?     Could the authors provide a comparative analysis to highlight the unique aspects of MERLIM?
soundness: 2
strengths: - The paper try to address a pertinent issue in the field of MLLMs, which is the need for a standardized evaluation benchmark to assess their performance on computer vision tasks.

- The focus on identifying hallucination events in MLLMs is a valuable contribution, as it highlights a critical challenge in the reliability of these models.

- The paper provides a detailed analysis of several MLLMs, shedding light on their limitations and potential areas for enhancement.
summary: The paper introduces MERLIM, a multi-modal evaluation benchmark for assessing Instruction Tuning Large Vision and Language Models (IT-LVLMs) on fundamental computer vision tasks. While the paper aims to provide a scalable test-bed to evaluate IT-LVLMs, it relies heavily on reusing and editing images from the MS-COCO dataset, which significantly diminishes the novelty and contribution of the work as a new benchmark. The paper also fails to incorporate evaluations of newer models that have emerged in the past year, limiting its relevance and timeliness. Additionally, the lack of open-sourcing the benchmark data hinders its practical utility and community adoption.
weaknesses: - The paper's reuse of MS-COCO images and ground-truth annotations as the primary dataset for the benchmark is a significant concern.  This approach potentially introduces biases and data leakage, which can affect the evaluation's fairness and accuracy.  A new benchmark should ideally utilize a fresh dataset to provide an unbiased assessment of model performance.

- The paper does not include evaluations of newer MLLMs that have emerged since the initial submission, which is a critical oversight given the rapid advancements in the field.  This limits the benchmark's relevance and timeliness.

- The lack of open-sourcing the benchmark data is a substantial drawback.  Open-sourcing is a standard practice that fosters transparency, reproducibility, and collaborative improvement.  Benchmarks like POPE, NExT-GQA, and HallusionBench have demonstrated the value of open-sourcing by enabling broader community engagement.

Meta-review: The AC appreciates the authors' responses and acknowledges the potential significance of this work. However, the submission falls short in several critical areas. The benchmark relies entirely on the COCO dataset, an prior generation resource with inherent biases that limit its diversity and utility for evaluating fundamental vision tasks (AC agrees). Furthermore, this reliance raises concerns about data leakage, as COCO overlaps with newer datasets and training sets, undermining its relevance to now. The inpainting method is of question, as it is used to create the dataset is also unreliable, often producing artifacts or failing to erase objects, resulting in subpar image quality. Additionally, the evaluation lacks breadth, with a limited range of models tested despite the availability of numerous open-source options. These shortcomings significantly limits the contribution of this work, and the submission does not meet the standards for acceptance.

Decision: Reject