Mitigating Hallucination in Large Multi-Modal Models via Robust Instruction Tuning
Authors: Fuxiao Liu
Kevin Lin
Linjie Li
Jianfeng Wang
Yaser Yacoob
Lijuan Wang
Venue: ICLR 2024 poster (2024)
Keywords: instruction tuning
multimodal large language model
hallucination
datasets
Primary area: datasets and benchmarks

Abstract: Despite the promising progress in multi-modal tasks, current large multi-modal models (LMMs) are prone to hallucinating inconsistent descriptions with respect to the associated image and human instructions. This paper addresses this issue by introducing the first large and diverse visual instruction tuning dataset, named Large-scale Robust Visual (LRV)-Instruction. Our dataset comprises 400k visual
instructions generated by GPT4, covering 16 vision-and-language tasks with open-ended instructions and answers. Unlike existing studies that primarily focus on positive instruction samples, we design LRV-Instruction to include both positive and negative instructions for more robust visual instruction tuning. Our negative instructions are designed at three semantic levels: (i) Nonexistent Object Manipulation, (ii) Existent Object Manipulation and (iii) Knowledge Manipulation. To efficiently measure the hallucination generated by LMMs, we propose GPT4-Assisted Visual Instruction Evaluation (GAVIE), a stable approach to evaluate visual instruction tuning like human experts. GAVIE does not require human-annotated groundtruth answers and can adapt to diverse instruction formats. We conduct comprehensive experiments to investigate the hallucination of LMMs. Our results demonstrate existing LMMs exhibit significant hallucinations when presented with our negative instructions, particularly Existent Object and Knowledge Manipulation instructions. Moreover, we successfully mitigate hallucination by finetuning MiniGPT4 and mPLUG-Owl on LRV-Instruction while improving performance on several public
datasets compared to state-of-the-art methods. Additionally, we observed that a balanced ratio of positive and negative instances in the training data leads to a more robust model. Code and data will be released upon publication.

Published as a conference paper at ICLR 2024
MITIGATING HALLUCINATION IN LARGE MULTI- MODAL MODELS VIA ROBUST INSTRUCTION TUNING
Fuxiao Liu', Kevin Lin?, Linjie Li,, Jianfeng Wang?, Yaser Yacoob', Lijuan Wang²
University of Maryland, College Park Miccosooft Corporation (f13es, yaser))und edu, (keli, lindsey. .i, jianfw, 1ijuanw)mmicroooft.co
ABSTRACT
Despite the promising progress in multi- tasks, current large multi modal models (LMMs) are prone to hallucinating inconsistent descriptions with respect to the associated image and human instructions. This paper addresses this is- sue by introducing the first large and diverse visual instruction tuning dataset, named Large-scale Robust Visual Inst)--nnttucttin. Our dataset comprises 400k visual instructions generated by GPT4 covering 16 vision-ande iisio-and-langeee and-language tasks with open-ended instructions and answers. Unlike existing studies that primar- ily focus on positive instruction samples, we design LRV-Instruction to include both positive and negative instructions for more robust visual instruction tuning. Our negative instructions are designed at three semantic levels: (i) Nonexistent Object Manipulation, (ii) Existent Object Manipulation and (iii) Knowledge Ma- nipulation. To efficiently measure the hallucination generated by LMMs, we propose GPT4-Assisted Visual Instruction Evaluation (GAVIE), a stable approach to evaluate visual instruction tuning like human experts. GAVIE does not require annotatdd groundtruth answers and can adapt to diverse instruction formats. We conduct comprehensive experiments investigate the hallucination of LMMs. Our results demonstrate existing LMMs exhibit significant hallucinations when presented with our negative instructions, particularly Existent Object and Knowl- edge Manipulation instructions. Moreover, we successfully mitigate hallucination by finetuning MiniGPT4 and mPLUG-OWl LRV-Instructiin while improving performance on several public datasets compared to state-of-the-ard methods. Ad- ditionally, observed that balanced ratio of positive and negative instances in the training data leads to more robust model. Code and data are available at https://ggithbb. com/ Ltw.://Fwwi.aLLL//IIIIISiii....-
INTRODUCTION
Significant progress has been made in the field of natural language processing, leading the develop- ment of models that can comprehend and follow instructions given natural language inputs (Wang
et al. 2022b Gong et al. 2023 Ouyang et al. 2022 Brown al., 2020). These models harness the power of large language models (LLM) and rely on high-quality instruction data, Similarly, efforts
have been made to introduce similar capabilities multi-modal models. GPT4 (OpenAl (023) has demonstrated impressive performance in multi-modal conversations with humans, yet the techniques contributing its extraordinary capabilities remain opaque. As result, several large multi-modal
models (LMMs) have recently emerged (Zhu et al. 2023 Liu et al. 2023d Gong et al. 2023 Dai al., 2023). such as MiniGPT4 (Zhu al.) 2023) and LLa VA (Liu et al.) 2023d). both utilize the Vicuna (Chiang et al. (202)) as the language generator but with different vision encoders (Radford
al. 2021]Li et al. (022). InstructBI.IP (Dai et al. (2023) is initialized from a pre-trained BL.IP-2
(Li et al. 2023b) while Multimodal-GPT (MMGPT) (Gong et al. 2023) is built on Flamingo (Alayrac
ct al. 2022 Awadalla et al. 2023).
A recent study (Li al.) 2023a) revealed that the hallucination issue of LLM, although not desired,
inherited by these LMMs (Zhu et al., 2023 Liu et al.] 2023d Gong et al.) 2023 Dai et al. 2023). Hallucination, major ethical concern associated with LLMs (Bang al. (223)) can lead to harmful consequences, especially when users without adequate domain knowledge over-rely on
these increasingly convincing language models. In the context of LMM hallucinations, the model

---

Published as a conference paper at ICLR 2024
GAVIE
Relevancy Score(0-10)
Accuracy Score(o-10)
Score (mPLLNG-Owl)) to
Score (mPL DG-bil- F
Reason: fullowed the instruction and
Reesor Despie the desotled desripttio provied, there mention of dog in imoge content
directly described a dog playing with exact
the
nskol the listraztion
Score (MinissPT4)
Score (MimiGPPT.):
Reesoon provided on ovemieeu the
Reesoo proviles detuiled description the image, such the people surfing, the ocean, and the wet sulfs.
Eveluate
image but didn't direcity address the instruction about findinng dog ond bolli
you help find doy playing with shore?
Score (Ours): 20
with a ball on the Chose The day located near the center the
There o drn
Score (Qurs): so Reason: The onswer directly responds question.
Reeson: The unswer line with the image injormation pointing out
scens, with the ball close its pan, This udct
mentoone elements from
playful element to the otherwise serene
guestion Imagee desripiinssitionn
beach setting where people are enjoying water activifies like sarting and subtimiing.
image a photo of group back surfing on their light heads.
Finetune
They wearing
mention e dog or
given imope. introdlacing on confusion and
blue color with white clouds.
MinitPP7
LRV- Instrucciitnn (Ouss
find
cornect detoii..
Figure 1: Given an image and human instruction as the input, we introduce GPT4-Assisted Visual Instruction Evaluation (GAVIE) to assess the output from current LMMs, such as MiniGPT4 and mPLUG-OWl BLUE represents LMMs can not accurately follow human instructions while RED means they suffer from the hallucination problem. After finetuning current LMMs on our proposed LRV-Instruction dataset, we can generate a more robust answer.
can generate descriptions with conflicting information to the given image. For instance, as shown in
Fig. (highlighted in red), existing LMMs (Zhu et al. 2023 Liu et al. 2023d Dai et al. 2023 Gong
et al.
2023) tend to describe nonexistent objects such as a dog" engaging in a nonexisting activity
like "llaying with ball". Additionally, the model may generate long image descriptions without following human instructions (highlighted in blue).
What are the likely causes of these hallucinations? As current LMMs are built on strong LLMs, they may over-rely on language priors and generate words more likely to go together with the instruction text regardless of the image content. What's 's more. LMMs, such as MiniGPT4 (Zhu et al.] [0223] and
LLaVA (Liu et al., 2023d), employ synthetic instruction data for training, which are generally long and involve nonexistent objects, activities, or relationships in the image.
Why can't 't LMMs accurately follow human instructions? We conjecture it is due to the lack of diversity in their training data. For example, MiniGPT4 (Zhu et al.) (023) is only instruction tuning with four instruction templates designed for image captioning tasks. Though MMGPT (Gong et al., 2023) and InstructBLIP (Dai et al. 2023) combine several datasets as the instruction tuning data, their instructions and answers still based on a few templates.
To address these challenges, we present LRV-Instruction, a large and diverse visual instruction benchmark. Our benchmark consists of 400k visual instructions generated by GPT4, taking inspiration from the success of recent GPT models in text-annotation tasks (Liu et al. 2023e). Unlike previous studies that focused on limited tasks and pre-defined templates created by human experts (Zhu et al.) 2023] Dai et al.) 2023 Gong et al.] 2023). LRV-Instruction covers 16 vision-language tasks with open-ended instructions and answers, as shown in Fig. 2 and Fig. 4 As observed by (Li al. 2023c), current LMMs tend to answer "Yes" for any instructions presented to the model, even when the proper answer should be "No". Our investigation reveals that most LMMs are finetuned on unbalanced datasets containing only positive instructions (Tab. D To enable LMMs to respond to human instructions more faithfully, we design LRV-Instruction to include both negative and positive instructions for robust instruction tuning. Our negative instructions are generated at three semantic levels (Fig- 2): (i) Nonexistent Object Manipulation, (ii) Existent Object Manipulation and (iii) Knowledge Manipulation in two different formats, Declarative and Interrogative. To improve the robustness and flexibility of the evaluation on visual instruction tuning, we propose GPT4-Assisted Visual Instruction Evaluation (GAVIE) to assess the LMM output in two different aspects: Relevancy to evaluate the instruction- -oolooiinnn performance and Accuracy to measure the visual hallucination in the LMM output. GAVIE does not require human-annotated groundtruth answers (Rohrbach et al.
2

---

Published as a conference paper at ICLR 2024
(Pas) Multi Choice VQA
(Pod object Detection
STOP
thoose the cornect storement about the weother condition the imsae ca Cloudy and raing, Clear (cieoagy miny, Stawy
Si what objjccts ore on toltler's jent?? & White socis
sty
(Pos) Object Attribute Detection
(Pos) OCR
Describe the gir''s hair color und whather tie wearing ony orcervery on her wris.
Q What tie number writee the lead smoobarnder?
The glr''s color is Blrk and the she wearinng bracelee
& The sumber is
(Poc) Visual Petalimmnt
(Poc)
venjy if statemont cornect "There parking the image."
What appears the purpose
streetigge?
True
& Giving directions or providing street nome.
(Neg) Nonexistent Object Mlnnipulation
(Neg) Existent Object Manipulation
you balloen There hot balloon mentiuned
G the woman blue panes pink There bulloon mentioned dhe
floor?
(Neg) Knowledge Manipulation
(Neg) Knowledge Monipulation
@ Was the youth emplymmen yas in Uibralne gradislly denessing 2015?
@ Did Merkei orrive to the Los Angoles Get Out The Vous Rully?
youth employment Morocco gradually incressing
Aooo, Hillory Glinton orrived Los Angeles Get Holly image.
Figure 2: Examples of positive and negative instances in our LRV- IRV-Instruction dataset. RED means inconsistent elements in the negative instructions. More examples are in the Appendix.
2018) and can be easily adapted to different formats instead of specific designs in (Li et al. 2023c). From our experiments, we show that GAVIE only stable but also aligns with human evaluation.
We empirically evaluate five publicly available LMMs (Zhu et al., 2023 Liu et al. 2023d Dai
al. 2023 Gong al. 2023 Ye al. 2023) on our benchmark and found that existing LMMs seriously hallucinate when prompted with our negative instructions, especially with Existent Object Manipulation and Knowledge Manipulation instructions. We further verify the effectiveness of
our LRV-Instruction by finetuning MiniGPT4 (Zhu et al. 2023) and mPLUG-Ow/ (Ye et al. 2023)
on this more balanced data. Our instruct-tuner LMMs suffer much less from hallucination and achieve tate-of-theearr performance compared to the original MiniGPT4, LLaVA (Liu et al. 2023d).
InstructBI.P (Dai et al., 2023), mPLUG-OW (Ye et al. both our evaluation set and public benchmarks (Li al.
2023) and MMGPT (Gong et al. 2023) on
2023c Hudson & Manning 2019 Fu et al.) 2023). We also observe that Existent Object Manipulation and Knowledge Manipulation instructions more challenging than Nonexistent Object Manipulation instructions for LMMs. Furthermore, robust model performance requires balanced ratio between positive and negative instances. To sum up, our contributions are three-fold:
We build LRV- Instructinn, large and diverse dataset containing 400k visual instructions, with 16 vision and language tasks and negative instructions in different semantic levels and styles.
We propose GAVIE, a novel approach to evaluate visual instruction tuning without requiring groundtruth answers and pre-designed instruction formats.
We conduct comprehensive experiments to investigate the hallucination of current LMMs. The empirical study validates the effectiveness LRV-Instruction for robust visual instruction tuning.
2 RELATED WORKS
Early explorations WWan et al. 2022a Li et al. 2022
2019 Sun et al. 2019) of vision and language
pre-trained models tend to use Bert- Bert-based (Liu al.
2019 Koroteev [02I] models as the language
et al. 2023 Zhao et al. 2023) and datasets (Lin et al.) 2014 Changpinyo et al. 2021 Krishna al. 2017 Liu et al. 2020) Sharma et al. 22018) Srinivasan et al. 2021 Liu et al. (023b), many studies (Alayrac ct al. 2022 Li et al. 2023b a) have been focused on improving vision-language pre-trained models
decoder. Inspired by the recent success of large language models (Touvron et al. 2023]GGlaa
by integrating powerful LLMs with in-context or few-shot learning capability. More recently, some
visual instruction-tunee LMMs (Zhu et al., 2023 Liu et al., 2023d; Gong et al. 2023: Dai et al.
2023) have emerged, showing excellent generalization performance in unseen VL tasks. Specifically,
LLa VA (Liu et al.) 2023d) projects the output of a visual encoder as input to LLaMA (Touvron et al.
2023 and trains both the alignment network and the L.LM on synthetic data. MiniGPT4 (Zhu et al.
3

---

Published as a conference paper at ICLR 2024
2023) is built on BLIP-2 but uses Vicuma (Chiang al. 2023)
the language decoder. It only finetunes the cross cross-modee modal alignment network on longer image captions from ChatGPT. The research approaches (Gong et al.] [223] Dai et al. (023) are instruction-tuned on collection of VL datasets, but InstructBBIIP (Dai et al. (023) uses BLIP2 (Li et al. 2023b) as the backbone while (Gong et al.
2023) is initialized from Flamingo (Alayrac et al. 2022). mPLUG-owl (Ye et al.] 2023) finetunes LLaMA (Touvron et al. (023)) model using both text instruction data and vision-language instruction data from VL (Liu al. 2023d). In comparison, we propose large and diverse visual instruction dataset with 16 vision and language tasks and negative instructions in different semantic levels and
styles. This can help improve the robustness of current LMMs.
Although LMMs are powerful in solving VL tasks, they also suffer from the hallucination inherited from LLM. Popular image captioning metries like CIDEr (Vedantam et al. 2015) SPICE (Anderson)
et al. 2016) do not appropriately penalize hallucination. CHAIR, (Rohrbach et al., 2018), is unstable and needs complex human-crafted crafted parsing rules for exact matching. Alternatively, (Li et al. 2023c) converts the hallucination into binary classification problem. However, it requires the input questions to follow specific templates, such as "Is there a/an <object> in the image?". In comparison. our proposed GAVIE can evaluate model hallucination in an open-ended manner without needing human-annotated groundtruth answers.
LRV-INSTRUCTION
Annotating large-scale visual instruction data can be challenging and time-consuming (Wang et al. 2022b). It involves expertly written detailed instructions and specific labels for different tasks, Inspired by the success of GPT4 in text-annotation tasks (Gilardi et al. (023), we leverage GPT4, instead of human workers, to build LRV-Instruction. LRV InV-Iustruction is designed to cover variety of VL tasks, with open-ended positive and negative instructions (Fig. 2 in different linguistic styles.
the in-context few-shot learning ability of GPT4 to generate instruction data for various VI. tasks automatically. We filter the output tasks manually and select 16 tasks (Tab. [a) with text answers. In contrast with (Liu iet al. (023d) using a few scene captions to represent an image as input to the text-only GPT4, we take advantage of the Visual Genome dataset (Krishna et al., 2017). which has detailed visual information like image size, bounding boxes, and dense captions. Specifically, each image typically has 21 object regions and their corresponding captions. We leverage GPT4 to create the mnstruction-folloiinn data with the image size, bounding boxes, and dense captions as the "visual" input as if it can "see" the image. An example is shown in Fig. 3 For each image, we randomly select 10 tasks. To enrich the instructions, we ask GPT4 to generate instances in both declarative and interrogative formats. The limitation of (Liu et al., 2023d Zhu et al.) (023) is that synthetic visual instructions are generally longer and may involve unexpected descriptive information inconsistent with the image. Therefore, we explicitly instruct GPT4 with "The answers should be less than 30 words" to reduce the chance of generating extra unrelated information in the training data.
Positive Visual Instruction Generation. Inspired by (Wang al. 2022b);
To improve the diversity of images, we collect chart images from (Tang et al. 2023). which has human-annotated captions describing the construction and patterns of charts. We also select news
images from (Liu et al.) 2020) with many named entities in the captions. We ask GPT4 to generate question- question-answers answers pairs with captions as visual input. The last two images in Fig. 2 are examples. More examples and the general prompt we use are shown in the Appendix (Fig.
2023c), current LMMs tend to answer "Yes" by following any instruction presented to the model rather than predicting faithful answer. To teach LMMs (Zhu al., 2023 Liu et al., 2023d] Gong et al., [023 Dai al., 2023) to answer questions in instructions faithfully, we introduce three categories of negative instructions based on Visual Genome dataset: (1) Negl: "Nonexistent Object Manipulattion" " by introducing nonexistent objects, activities, attributes and interactions to the "visual" input described above. (2) Neg2: "Existent Object Manipulation" by manipulating existent objects with inconsistent attributes (Fig. (3) Neg3: "Knowledge Manipulation" by manipulating knowledge in instructions (Fig. 2 As for the detailed prompt of Negl, we leverage the same format of the "visual" input as shown in Fig. Additionally, we provide the following instructions to GPT4:
Negative Visual Instruction Generation. As shown in (Li et al.
"Come up with 6 misleading instructions with nonexistent elements (nonexistent objects, nonexistent activities, nonexistent attributes, nonexistent interactions) in the images with different language
4

---

Published as a conference paper at ICLR 2024
Ours MiniGPT4
LLaVA
InstructBLIP
MMGPT
mPLUG-OW/
Hard Negative Instructions?
Self Generated Instruction?
X
Address Hallucination?
x
X
X
X
X
NOT Template Instruction?
X
X
X
V
# of Genf-Generated Instances
400k
3k
150k
X
X
X
# Tasks
Table I: comparison of LRV-Instruction with datasets used by current LMMs.
Prompt:
Give an image with following information: bounding box, positions that are the object toft-top comerr coordinates(X, Y), object sizes(Width, Height). Highly overlapping bounding boxes may refer the object.
bounding box:
elephant heard on rocks X: 73 Y. 8o Width: 418 Height 418
woman wearing long dress 176 Y: 298 Width: 35 Height: 83
group green chairs
Y: 326 Width:
Height: 126
an orange bucket on the ground X: 92 Y: 342 Width: 38 Height: 36
group white umbrellas X: 99 Y: 82 Width: 112 Height: 28
a man in an orange shirt X: 204, Y: 265 Width: 31 Height: 47
a woman wearing yellow dress X: 169 298 Width: 47 Height: 76
Task: image captioning, Image Sentiment Analysis, Image Quality Assessment, Object Interaction Analysis, Object Attribute Detection, Mull-choice VQA
Come up with 20 diverse instructions for all the tasks above with different language styles and accurate answers, The instructions should contain interrogative sentence and declarative sentences, The should less than 30 words. Each task should have less than instructions.
GPT4 OUTPUT Example:
Instruction:
Craft brief harrative about the baby elephant and adult elephant.
Answer:
A baby elephant is depicted behind an adult elephant, possibly seeking protection.
One example illustrate the prompt We use the bounding box coordinates and dense captions to represent image content.
Figure 3:
generate the visual instruction data by GPT4.
styles. The instructions should contain interrogative and declarative sentences. Please also ex- plain the reason." "
We replace the underlined text with "existing objects but wrong attributes" for the prompt of Neg2. As for the Neg3: knowledge manipulation, we use GPT4 to manipulate the knowledge in the captions, including named entities, events keywords. After that, GPT4 is instructed to generate questions and answers indicating correct knowledge. More examples are shown in the Appendix (Fig.
Quality Control. We first remove instances with answers longer than 30 words. We remove the instances mentioning unneeded content like "bounding box description", "given caption". and "existing descriptions". Additionally, GPT4 will output the task name for each instruction. However, we found that GPT4 sometimes assigns inaccurate task names for the instructions. As a result, we exclude the task name our release data. Furthermore, we removed the instructions asking about facial expressions. This is because the Visual Genome dataset doesn't 't include facial expression attributes in the ground truth-dense captions. To examine the quality of our dataset, we randomly sample 500 instances and ask ten expert annotators to determine whether the output answers from GPT4 are correct not, with regard the instruction and the image content. We found 91% % of the instructions are appropriate for the image inputs. Furthermore, 85% of outputs are acceptable responses the instructions. Even though some responses may contain errors, most generations conform to the correct structure, serving applicable visual instruction tuning guidelines. We created a total over 400k visual instructions after filtering.
Evaluation Set. After the processing above, randomly select 1000 instances our evaluation set. Furthermore, we manually check the quality of all instances and see whether the instruction describes valid task. If it's not, edit the instruction make it clearer for LMMs. For example, instruction "Observe the beautiful colboww-olloed sign that says 'Le Louvre'. You won't 't miss it!' to "Are you able observe the beautiful rainbow-colored sign that says 'Le Louvre' in the image?"
edit the
5

---

Published as a conference paper at ICLR 2024
Tasks of LRV- LRV-ttttuuution
Statistic
Image Captoning, Object Detection, Image Sentiment Anelysis, Image Quality Asiessment, Object Interaction Analysis, Image Anomaly Detection, Raferential Expression Grounding, OCR, VCR, Object Attribute Detection, Mull choica VQA, Semaniic Segmentation, Dense Captioning, Visual Entallment, Styled Image Caption, Activity Recognition, Chart Understanding
of Instructions
400k
of Images
35k
of Interragative Instruction
250k
of Deciarative Instruction
150k
of Negative Instructions
180k
Ave # of Instruction per Image
17.7
Ave Answer Length
15.2
taslec in dataset.
Ave Instruction Length
12.1
Frequency of Non-Existent Objects
(c) Statistic -istuustutttoon
09
builiig
bay
9290
2000
3000
4000
(b) Statistic of Non-existent objects generated GPT4
(d) Distribution of different Manipulation
Figure 4: Comprehensive Statistic of LRV-Instruction. In (d), BLUE means existent object manipula-
tion. PINK means nonexistent object manipulation. GREEN means knowledge manipulation.
3.1 DATA STATISTICS
Tab. D shows a comparison of I.R-IInstrcction and other datasets used by current LMMs. LRV- Instruction covers much more VI. tasks than existing visual instruction tuning datasets. Instead of only using positive instructions, LRV-Instruction also includes negative instructions at different semantic levels. In addition, employing the GPT4-assisted generation, LRV- IRV-Instruction has more open-ended instructions instead of following a few templates. From Fig. (b), we observe that instructions with non non-existing objects generated by GPT4 are diverse and physically plausible in the image, including "birds in the sky" or replacing 'elephant' with 'zebra". Fig. [O in the appendix shows the diverse distribution of knowledge manipulation, including event, number, date, persons, place, and others.
VISUAL INSTRUCTION TUNING
We constructed two current LMMs: MiniGPT4 Zhu al. (2023) and mPLUG-Owl Ye et al. 2023
as the backbones for visual instruction tuning. MiniGPT4 consists of the Vision transformer Liu et al. 2023c) backbone the image encoder, Vieuna Chiang et al. (2023) the text decoder and a tre-tr Q-Former to connect them. Vicuna is built upon LLMMA Touvron et al. (2023) with stronger following ability. Following Zhu et al. (2023), the Q-Former is designed to extract visual features from the frozen image encoder. Before feeding into the frozen Vicuna as the visual prompt, learnable linear projection layer to narrow the gap between extracted visual features with Vicuna embeddings, mPLUG-Owl comprises pre-trained visual encoder, a visual abstractor, and Vicuna Chiang et al. (2023) the text decoder. The visual encoder responsible for extracting visual features from the input images, and the visual abstractor distills these features using set of learnable tokens. The resulting visual features are concatenated with the word embeddings of the input sentence and fed into Vicuna to generate the response. We freeze the visual abstractor and visual encoder. Instead, we adopt the low-rank adaptation Hu et al. (22T) to train the text decoder.
GPT4-ASSISTED VISUAL INSTRUCTION EVALUATION
CHAIR (Rohrbach et al. 2018) was introduced to evaluate object hallucination in image captioning tasks. However, it usually demands complex human-craftee rules. Alternatively, (Li et al., 2023c binary classification task that prompts LMM to output "Yes" or "No". However, is hard to evaluate the LMM output in an open-ended manner. In addition, both methods highly depend human-annotated groundtruth answers.
Fu et al. 2023) formulate the evaluation of hallucination

---

Published as a conference paper at ICLR 2024
Backbone
Perception
Cognition
Backbone
Acc(Pos)
Acc(Neg)
Original MiniGPT4
616.41
232.71
Original MiniGPT4
0.53
0.54
Finetuned MiniGPT4
895.96
296.43
Finetuned MiniGPT4
0.58
0.68
Original mPLUG- UG-Owl
967.34
276.07
Original mPLUG-OW:
0,62
0.55
Finetuned mPLUG-Owl
1298.78
328.21
Finetuned mPLUG-Owl
0.69
0.78
Table 2: Zero-shot multimodal evaluation MME (Fu et al.
2023) of MiniGPT4-7B, mPLUG-Owl
7B between original models and ARR-Inntruuctionfiie finetuned models. The left chart shows perception and cognition scores. The right chart shows the accuracy on the positive set and the negative set.
Model
Acc
F1
Model
Acc
F1
Model
Acc
F1
mPLUG-Owl-7B
0.52
0.68
mPLUG-OWI-7B
0.57
0.66
mPLUG-OWl-77
0.60
0.64
LLaVA-13B
0.50
0.66
LLaVA-138 133
0.50
0.66
LLaVA-13B
0.50
0.66
MiniGPT4-133 13B
0.73
0.71
MiniGPT4-13B
0.67
0.67
MiniGPT4-13B
0.62
0.63
InstructBLIP-139
0.86
0.87
InstructBLIP-13B
0.71
0.76
InstructBLIP-13B
0.63
0.72
Ours-7B
0.86
Ours-7B
0.73
0.79
Ours-7B
0.65
0.73
(a) Random Set.
(b) Popular Set.
(c) Adversarial Set.
Table Zero-shot object hallucination evaluation on POPE (Li al. 2023c Objects not existing in
the image are sampled with three different strategies. Random: random sampling, Popular: top-k k most frequent objects in MS-COCO, Adversial: objects are first ranked based on co-occurring frequencies, then top-k frequent ones are sampled. Ours-7B means Finetuned mPLUG-OWW-7B.
To this end, introduce GPT4-Assisted Visual Instruction Evaluation (GAVIE) and robust approach to evaluate object-level hallucination. The general prompt we use is shown in the Appendix. GPT4 takes the dense captions with bounding box coordinates as the image content and compares human instructions and model response. Then we ask GPT4 to work as a smart teacher criteria. (1) Accuracy: whether the response is
flexible
and score (0-10) students' answers based accurate concerning the image content. (2) Relevancy: whether the response directly follows the instruction. We use GPT4-32K-0314 in the experiments. Fig. [isccessssllllll points that "dog, ball" is inconsistent with the image, and the response from the MiniGPT4 did not address the instruction. Unlike previous evaluation methods (Li al., 2023c] Rohrbach et al. (018). GAVIE does not require human-annotated groundtruth answers and can freely adapt to diverse instruction formats. As for the knowledge level hallucination or images that are not from the Visual Genome dataset, we use the groundtruth answers as reference and compare them with predictions (Fig. 7 in the appendix).
EXPERIMENT
6.1 IMPLEMENTATION SETUP
Baselines. We evaluate the zero-shot performance of 5 recently released LMMs: (1) MiniGPT4; (2) MiniGPTv2; (3) InstructBLIP: (4) Multimodal-Gri (MMGPT); (5) mPLUG-Owl (6) LLaVA; (7)
LLaVA 1.5. All models above have been tuned on their collected visual instruction data.
Training Details. As for MiniGPT4, we initialize from its checkpoint of the first pretraining stage. Then we instruct-tune the model on LRV Instrucctnn with the linear projection layer as the only learnable module. As for mPLUG-OWl, we train the text encoder by LoRA training. Additionally, we only replace the LLaVA dataset in their finetuning data with LRV-Instruction to make a fair comparison with the original Mplug-Owl.
utilize MiniGPT4-7B and mPLUG-OW1-7B since we don't 't have the computing resources to finetune the 13B models. We trained our models on NVIDIA
Quadro RTX 8000. As for the hyper-parameters. please refer to (Zhu et al. 2023 Ye et al. 2023).
Evaluation Benchmarks. Apart from our proposed evaluation set, evaluate LMMs three public benchmarks, MME (Fu et al. 2023) is a human--nnotater annotated benchmark, measuring perception and cognition abilities 14 subtasks. POPE (Li et al.) (023c) and AMBER (Wang al. 2023)
recently released datasets to evaluate object hallucination. GQA dataset (Hudson & Manning 2019) is public visual question-ansen dataset with open-ended questions.
6.2
MAIN RESULTS
How do LMMs perform on public datasets? We compare our model against the baseline models on POPE in Tab and AMBER in Tab. 10 The results show that current L.MS may not work well
7

---

Published as a conference paper at ICLR 2024
GAVIE
Ours
MiniGPT4
LLaVA
InstructBL.IP
MMGPT
mPLUG-Owl
ACCURACY (0-10)
6.58
4.14
4.36
5.93
0.91
4.84
RELEVANCY (0-10)
8.46
5.81
6.11
7.34
1.79
6,35
Human Expertl (1-4)
3.48
2.61
2.87
3.00
1.90
2.90
Human Exper12 (1-4)
3.58
2.23
2.07
2.48
1.05
2.27
Human Expert3 (1-4)
3.33
2.58
2.89
2.94
1.38
2.91
Table 4: Comparison results on our evaluation set evaluated by GAVIE. Ours means Finetuned mPL.UG-OWL-7B. All the LMMs are 7B versions make fair comparison.
Model
InstructBLIP-I 13B LLaVA-13B MiniGPT4-13B mPLUG-Owl Ours-7B Ours-7B-PP -sus
Accuracy
0.62
0.47
0.42
0.41
0.64
0.60
Table 5: Zero-shot evaluation on GQA. Ours-7B means Finetuned WwwPLUG-OONe Oww--B. Ours-7B-Par means finetune mPLUG-Owl on pseudo instruction data by (Wu et al. 2022).
with open-ended negative instructions. In contrast, the highest scores of our model demonstrate that LRV-Instruction exhibits robustness to visual hallucination, matching or surpassing the performance of 13B counterparts. From Tab we found both finetuned LMMs on LRV-Instruction outperform original ones in the zero-shot evaluations. Additionally, Finetuned-MMlug-OO0 MiniGPT4 because Mplug-Ow, Owll can do the LoRA training improve the language ability. We also calculate the accuracy on positive and negative samples of MME in the right chart of Tab The improvement in the positive samples is because L.R--Instucction has more diverse tasks than mPLUG- Owl datasets and MiniGPT4 datasets. The improvement in the negative samples demonstrates the value LRV-Instruction dataset to equip the model with the ability to say 'no' and provide correct answers. The completed results on shown in Tab. 14 We further explore the LMMs performance in the common scenario of visual question- guestion-answering (VQA). Results in Tab. suggest our method (Finetuned mPLUG-Ow] achieves on-par performance with InstructBLIP in generic VQA setting.
exceeds Finetuned-
How do LMMs perform on LRV-Instruction? Tab. 4 and Tab. 12 show results on our evaluation set. Among the baselines, InstructBLIPP achieves better results than other LMM baselines because its visual instructions are collected from a wide variety of publicly available datasets. LLaVA (Liu al. 2023d) utilizes the GPT-assisted approach to generate visual instructions, but its performance is much worse. This is probably because its synthetic answers from GPT4 are generally longer and may involve irrelevant information. As a comparison, our model outperforms the existing LMM baselines by a large margin, benefiting from the rich composition of our dataset and better prompt design.
6.3 DETAILED ANALYSIS
Does GsS4tsse Visual Instruction Evaluation align with Human Evaluation? We select three human experts specializing in the field of NLP to evaluate the predictions from LMMs with four options for the scores (1) Very Poor, (2) Poor, (3) Good, (4) Excellent. To evaluate the results quantitatively, we assign different scores for the options: Very Poor=1, Poor=2, Good=3, Excellen==4. More implementation details are shown in the appendix. From Tab. 4 all experts agree that the output from our model is the best, followed by InstructBLIP in second place, and MMGPT performs the worst. The observation aligns with the GAVIE evaluation results.
GPT4-Assisted Evaluation Stable? We execute GAVIE 5 times on each instruction and evaluate the predictions from different LMMs. We leverage Standard Deviation (STD) to measure the stability of GAVIE. From Tab. 7 (left), we observe that STD ranges from 0.65 2.46. The ACCURACY and RELEVANCY scores of an instance from GPT4 may vary between different times, but they always belong to the same grade level. According to completed results from Tab. 9 RELEVANCY has four grade levels: (1) The response is completely relevant (9-10), (2) The response is mostly relevant (6-8), (3) The response is partly relevant (3-5), (4) The response is seldom relevant (0-2). ACCURACY has four grade levels: (1) The response is completely accurate (9-10), (2) The response has minor errors (6-8), (3) The response is partly accurate (3-5), (4) The response is mostly or completely wrong (0-2).
How do LMMs perform at the different semantic levels of hallucination? As shown in Tab 6 all baselines perform better on Negl (Nonexistent Object Manipulation) than Neg2 (Existent Object Manipulation) and Neg3 (Knowledge Manipulation). From the visual perspective, existent object
8

---

Published as a conference paper at ICLR 2024
Categories
Metric
Ours MiniGPT4
LLaVA
InstructBLIP
MMGPT
mPL.UGOWl
Negl
ACP))
3.72
2.09
5.50
1.13
4.20
Neg2
(CP)))
6.50
2.57
1.42
2.18
0.96
2.46
Neg3
ACCURACY(GPT4)
6.25
2.30
1.56
2,38
0.94
2.57
Neg1
RELEVANCY(GPTT)
8.96
5,94
4.83
7.22
2.24
5.35
Neg2
RELEVANCY(GPT4)
8.46
2.53
1,82
2.73
1.19
3.16
Neg3
RELEVANCY(GPT4)
8.21
2.40
1.78
2.39
0.98
2.87
Table 6: Completed evaluation results on Negl: Nonexistent Object Manipulation, Neg2: Existent
Object Manipulation and Neg3: Knowledge Manipulation by GAVIE.
Metrie
Accuracy-STD
Accuracy-Mean
Ratio
AcCpos
ACCneg
Ours
2.42
6.60
All Pos
0.97
0.05
MiniGPT4
2,46
3.76
Pos:Neg=2:1
0.95
0,50
InstructBLIP
2.42
5.29
Pos:Neg=1:1
0.92
0,85
mPLUG-Owl
1.96
0.87
Pos:Neg=1:2
0.87
0.86
LLaVA
2.37
3.80
All Neg
0.10
0.98
MMGPT
0.65
4.84
Table 7: (left): Evaluation of the stability of GAVIE. STD means standard deviation. Completed results are shown in Tab. (right): Results of different composition ratios in instruction tuning.
manipulations with wrong attributes in Neg2 are more challenging than adding nonexistent objects from images to instructions in Neg.. For example, in Fig. 2 it may be straightforward find that the "hot air balloon" does not appear in the image. However, "woman" does exist in the second example of Fig. 2 while she is not in the blue pants and pink shirts, which requires fine-grained understanding of the visual content. Therefore, a more powerful vision encoder is needed for future LMMs. Knowledge manipulation is challenging because current LMMs are finetuned on general images without specific knowledge. In contrast, our model greatly improves at all semantic levels, which benefits from our diverse instruction tuning data.
How do LMMs perform at the different composition ratios in training data? In Tab. 7 (right), we investigate how IRV-Instruction addresses hallucination issues with different ratios of positive and negative samples in the training set. Inspired by (Li et al. 2023c), we instruct the model to produce "Yes" or "No" and use classification accuracy on our evaluation set. Acc is the accuracy on the positive instruction set, while Accneg is the accuracy (right), we found that AcCneg increases with more negative samples, which verifies our hypothesis that the hallucination problem of current LMMs is due to the lack of negative instructions. Besides, with a balanced ratio (pos:neg=1:1),. the model performs the best in both positive and negative sets.
the negative instruction set. From Tab. 7
Use Pseudo Dense Captions instead of GT from Visual Genome to Generate Instructions. To demonstrate the scalability of our dataset, we use pseudo-dens captions generated by GRiT (Wu) al. 2022) replace the GT captions in the Visual Genome dataset. We remove the images, whose detected objects by GRiT are less than 15 to ensure GPT4 has enough visual information when generating visual instructions. From Tab. 5 we found finetuning pseudo captions also improve the performance compared to the original mPLUG-Owl. This demonstrates that our visual instruction generation method be further scaled up without groundtruth dense captions.
CONCLUSION
In this work, we constructed LRV-Instruction, a large and diverse dataset containing 400k visual instructions, covering 16 vision and language tasks with both positive and negative instructions in different semantic levels and styles. With LRV-Instruction, we comprehensively investigated the hallucination of existing LMMs and empirically validated its effectiveness in more robust visual instruction tuning. In addition, we propose GAVIE, a novel approach to evaluate visual instruction tuning without requiring human-labeled groundtruth answers and can be easily adapted to different instruction formats. We hope our work can help address the unexpected hallucination issues of LMMss. Future directions include replacing the vision encoders in current LMMs with more powerful visual models to match the capabilities of multimodal GPT4 and investigation other biases of LMMs to develop more robust models.

---

Published as a conference paper at ICLR 2024
8 ACKNOWLEDGEMENTS
This work is supported in part by the US Defense Advanced Research Projects Agency (DARPA) Semantic Forensics (SemaFor) Program under HIR001120C0122 Any opinions, findings, and conclusions recommendations expressed in this material are those of the authors and do not necessarily reflect the views of the DARPA. We also thank the anonymous reviewers for their constructive feedback.
REFERENCES
Lean-Baptiste Alayrac, Jeff Donahue, Pauline Luc, Antoine Miech, lain Barr, Yana Hasson, Karel Lenc, Arthur Mensch, Katherine Millican, Malcolm Reynolds, et al. Flamingo: visual language model for few-shot learning. Advances in Neural Information Processing Systems, 35:23716- 23736, 2022.
Peter Anderson, Basura Fernando, Mark Johnson, and Stephen Gould. Spice: Semantic propositional image caption evaluation. Computer Vision- 2016: 14th European Conference, Ams- terdam, The Netherlands, October 11-14, 2016, Proceedings, Part V 14, pp. 382-33 292399..0 Springer, 2016.
Anas Awadalla, Irena Gao, Joshua Gardner, Jack Hessel, Yusuf Hanafy, Wanrong Zhu, Kalyani Marathe, Yonatan Bitton, Samir Gadre, Jenia Jitsev, Simon Kornblith, Pang Wei Koh, Gabriel Ilharco, Mitchell Wortsman, and Ludwig Schmidt. Openflamingo, March 2023. URL https
//doi.org/10 2281/2enod..
Yejin Bang, Samuel Cahyawijaya, Nayeon Lee, Wenliang Dai, Dan Su, Bryan Wilie, Holy Lovenia, Ziwei Ji, Tiezheng Yu, Willy Chung, et al. A multitask, multilingual, multimodal evaluation of chatgpt on reasoning, hallucination, and interactivity. arXiv preprint arXiv:2302.04022. 2023.
Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared D Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, et al. Language models are few-shot leamners. Advances in neural information processing systems, 33:1877-1901. 2020.
Soravit Changpinyo, Piyush Sharma, Nan Ding, and Radu Soricut. Conceptual 12m: Pushing web-scale image-text pre-training to recognize long-tail visual concepts. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3558-3568, 2021.
Jun Chen, Deyao Zhu, Xiaoqian Shen, Xiang Li, Zechun Liu, Pengchuan Zhang, Raghuraman Krishnamoorthi, Vikas Chandra. Yunyang Xiong, and Mohamed Elhoseiny. Minigpt-v2: large language model as a unified interface for vision-language multi-task learning. arXiv preprint arXiv:2310.09478, 2023.
Wei-Lin Chiang, Zhuohan Li. Zi Lin, Ying Sheng, Zhanghao Wu, Hao Zhang, Lianmin Zheng, Siyuan Zhuang, Yonghao Zhuang, Joseph E Gonzalez, al. Vicuna: An open-source chatbot impressing gpt-4 with 90%* chatgpt quality, 2023.
Wenliang Dai, Junnan Li, Dongxu Li, Anthony Meng Huat Tiong, Junqi Zhao, Weisheng Wang. Boyang Li, Pascale Fung, and Steven Hoi. Instructblip: Towards seneral-purpose vision-language models with instruction tuning. arXiv preprint arXiv:2305.06500 2023.
Chaoyou Fu, Peixian Chen, Yunhang Shen, Yulei Qin, Mengdan Zhang, Xu Lin, Zhenyu Qiu, Wei Lin, Jinrui Yang, Xiawu Zheng, et al. Mme: A comprehensive evaluation benchmark for multimodal large language models. arXiv preprint arXiv:2306. 13394, 2023.
Fabrizio Gilardi, Meysam Alizadeh, and Maël Kubli. Chatgpt outperforms crowe-workers for text-annotation tasks. arXiv preprint arXiv:2303. 15056, 2023.
Tao Gong, Chenggi Lyu, Shilong Zhang, Yudong Wang, Miao Zheng, Qian Zhao. Kuikun Liu, Wenwei Zhang, Ping Luo, and Kai Chen. Multimodal-gpi A vision and language model for dialogue with humans. arXiv preprint arXiv:2305.04790 2023.
10

---

Published as a conference paper at ICLR 2024
Edward J Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang. Lu Wang, and Weizhu Chen. Lora: Low-rank adaptation of large language models. arXiv preprint 1xXfv:2106.0968... 2021.
Drew A Hudson and Christopher D Manning. Gqa: A new dataset for real-world visual reasoning and compositional question answering. In Proceedings of the IEEE/CVT conference on computer vision and pattern recognition, pp. 5700-6707 2019.
MV Koroteev. Bert: review of applications natural language processing and understanding. arXiv preprint 10.Xiv:2103..1 1943, 2021.
Ranjay Krishna, Yuke Zhu, Oliver Groth, Justin Johnson, Kenji Hata, Joshua Kravitz, Stephanic Chen, Yannis Kalantidis, Li-Jia Li, David Shamma, et al. Visual genome: Connecting language and vision using crowdsourced dense image annotations. International journal of computer vision, 123:32-73, 2017.
Bo Li, Yuanhan Zhang, Liangyu Chen, Jinghao Wang, Jingkang Yang, and Ziwei Liu. Otter: A multi-modal model with in-context instruction tuning. arXiv preprint arXiv:2305.03726. 2023a.
Junnan Li. Dongxu Li, Caiming Xiong, and Steven Hoi. Blip: Bootstrapping language-iman pre- training for unified vision-language understanding and generation. In International Conference on Machine Learning, pp. 17888-22900 PMLR, 2022.
Junnan Li, Dongxu Li, Silvio Savarese, and Steven Hoi. Blip-2: Bootstrapping language-image pre- training with frozen image encoders and large language models. arXiv preprint arXIv:2301.12559, 2023b.
Liunian Harold Li, Mark Yatskar, Da Yin, Cho-Jui Hsieh, and Kai-Wei Chang. Visualbert: A simple and performant baseline for vision and language. arXiv preprint arXiv: (xXiv:/90000 (aai:/9980.0755 2019.
Yifan Li, Yifan Du, Kun Zhou, Jinpeng Wang, Wayne Xin Zhao, and Ji-Rong Wen. Evaluating object hallucination in large vision-language models. arXiv preprint arXiv:2305. 10355, 2023c.
Tsung- Yi Lin, Michael Maire, Serge Belongie, James Hays, Pietro Perona, Deva Ramanan, Piotr Dollár, and Lawrence Zitnick. Microsoft coco: Common objects context. In Computer Vision- ECCV 2014: 13th European Conference, Zurich, Switzerland, September 6-12, 2014, Proceedings. Part V 13, pp. 740-755. Springer, 2014.
Fuxiao Liu, Yinghan Wang, Tianlu Wang, and Vicente Ordonez. Visual news: Benchmark and challenges in news image captioning. arXiv preprint arXiv:2010.03743, 2020.
Fuxiao Liu, Tianrui Guan, Zongxia Li, Lichang Chen, Yaser Yacoob, Dinesh Manocha, and Tianyi Zhou. Hallusionbench: You see what you think? or you think what you see? an image-context reasoning benchmark challenging for gpt-4v (ision), 1lava-1.5, and other multi-modality models. arXiv preprint arXiv:2310. 1...226600..66, 2023a.
Fuxiao Liu, Hao Tan, and Chris Tensmeyer. Documentclip: Linking figures and main body text in reflowed documents. arXiv preprint arXiv:2306.00 06306, 2023b.
Fuxiao Liu, Yaser Yacoob, and Abhinav Shrivastava. Covid-vts: Fact extraction and verification on short video platforms. In Proceedings of the 17th Conference the European Chapter the Association for Computational Linguistics, pp. 178-188, 2023c.
Haotian Liu, Chunyuan Li, Qingyang Wu, and Yong Jae Lee. Visual instruction tuning. arXiv preprint 2r04v:2304 200.0044440.4.... 2023d.
Yang Liu, Dan Iter, Yichong Xu, Shuohang Wang, Ruochen Xu, and Chenguang Zhu. Gpteval: Nlg evaluation using gpt-4 with better human alignment. arXiv preprint arXiv:2303. 16634, 2023e.
Yinhan Liu. Myle Ott. Naman Goyal, Jingfei Du, Mandar Joshi, Danqi Chen, Omer Levy, Mike Lewis, Luke Zettlemoyer, and Veselin Stoyanov. Roberta: A robustly optimized bert pretraining approach. arXiv preprint arXiv: 197. 2019.
OpenAI. Gpt-4 technical report. 2023.

---

Published as a conference paper at ICLR 2024
Long Ouyang, Jeffrey Wu, Xu Jiang. Diogo Almeida, Carroll Wainwright, Pamela Mishkin, Chong Zhang, Sandhini Agarwal, Katarina Slama, Alex Ray, et al. Training language models to follow instructions with human feedback. Advances in Neural Information Processing Systems, 35: 27730-27744, 2022.
Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et al. Learning transferable visual models from natural language supervision. International conference on machine learning, pp- $748-8763 PMLR, 2021.
Anna Rohrbach, Lisa Anne Hendricks, Kaylee Burns, Trevor Darrell, and Kate Saenko. Object hallucination image captioning. arXiv preprint arXiv:1809.02156 2018.
Piyush Sharma, Nan Ding, Sebastian Goodman, and Radu Soricut. Conceptual captions: A cleaned, hypernymed. image alt-text dataset for automatic image captioning. In Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pp. 9556-2565, 2018
Krishna Srinivasan, Karthik Raman, Jiecao Chen, Michael Bendersky, and Mare Najork. Wit: Wikipedia-basse image text dataset for multimodal multilingual machine learning. In Proceedings of the 44th International ACM SIGIR Conference on Research and Development in Information Retrieval, pp. 2443-2449, 2021.
Chen Sun, Austin Myers, Carl Vondrick, Kevin Murphy, and Cordelia Schmid. Videobert: A joint model for video and language representation learning. In Proceedings of the IEEE/CVF international conference on computer vision, pp. 7464-7473, 2019.
Benny J Tang, Angie Boggust, and Arvind Satyanarayan. Vistext: benchmark for semantically rich chart captioning. arXiv preprint arXiv:2307.05356, 2023.
Hugo Touvron, Thibaut Lavril, Gautier Izacard, Xavier Martinet, Marie-Anne Lachaux, Timothée Lacroix, Baptiste Rozière, Naman Goyal, Eric Hambro, Faisal Azhar, et al. Llama: Open and efficient foundation language models. arXiv preprint arXiv:2302. 1397,,
Ramakrishna Vedantam, C Lawrence Zitnick and Devi Parikh. Cider: Consensus-basee image description evaluation. In Proceedings of the IEEE conference computer vision and pattern recognition, pp. 4576, 2015.
Jianfeng Wang. Zhengyuan Yang, Xiaowei Hu, Linjie Li, Kevin Lin, Zhe Gan, Zicheng Liu, Ce Liu, and Lijuan Wang. Git: A generative image-to-text transformer for vision and language. arXiv preprint arxi::2205. 14100, 2022a.
Junyang Wang, Yuhang Wang, Guohai Xu, Jing Zhang, Yukai Gu, Haitao Jia, Ming Yan, Ji Zhang, and Jitao Sang. An lim-free culti-dimeensonal benchmark for milms hallucination evaluation. arXiv preprini arXi:22311.07397 2023.
Yizhong Wang, Yeganeh Kordi, Swaroop Mishra, Alisa Liu, Noah A Smith, Daniel Khashabi, and Hannaneh Hajishirzi. Self-instruct: Aligning language model with self generated instructions. arXiv preprint arXiv:2212. 2022b.
Jialian Wu, Jianfeng Wang, Zhengyuan Yang, Zhe Gan, Zicheng Liu, Junsong Yuan, and Lijuan Wang. Grit: A generative region-to-text transformer for object understanding. arXiv preprint arXiv:2212.00280, 2022.
Qinghao Ye, Haiyang Xu, Guohai Xu, Jiabo Ye, Ming Yan, Yiyang Zhou, Junyang Wang, Anwen Hu, Pengcheng Shi, Yaya Shi, et al. mplug-owl: Modularization empowers large language models with multimodality. arXiv preprint arXiv:2304. 14178, 2023.
Wayne Xin Zhao, Kun Zhou, Junyi Li, Tianyi Tang, Xiaolei Wang, Yupeng Hou, Yingqian Min, Beichen Zhang, Junjic Zhang, Zican Dong, et al. A survey of large language models. arXiv preprint Xrxv:22303. 0.200.18222.. 18223, 2023.
Deyao Zhu, Jun Chen, Xiaoqian Shen, Xiang Li, and Mohamed Elhoseiny. Minigpt-4: En- hancing laniun-l understanding with advanced large language models. arXiv preprint arXiv:2304. 10592, 2023.
12

---

Published as a conference paper at ICLR 2024
A
APPENDIX
A.1 GAVIE EVALUATION
We show two full examples of the text prompt for GAVIE in (i) Fig. 21 22 23 and (ii) Fig. 24 25
26 We first leverage the bounding boxes and dense captions as the visual input. We provide the human instructions and responses from different models in Fig. 22 and Fig. 25 Furthermore, we ask GPT4 to pretend as a smart teacher and score (0-10) the answers according to the image content and instructions. There are two criteria. (I) Accuracy: whether the response is accurate concerning the image content. (2) Relevancy: whether the response directly follows the instruction. After that, GPT4 required to generate a score and reason. Fig. 23 and Fig. 26 show the full evaluation output from GAVIE.
A.1.1 GPT4-Assisted Visual Instruction Evaluation (GAVIE) VS. HUMAN EVALUATION
This section provides insights into the GAVIE via human evaluation. Here, we randomly select 40 image-instruction instances from the evaluation set. The human assessment is carried out by three experts specializing in NLP. The questionnaire consists of 40 questions randomly shuffled for each expert. The questionnaire takes about 20 minutes to complete on average. Each question includes an instruction, an image, and responses from 4 different LMMs. We provide instructions for experts as follows:
"As for each question, there are an instruction, an image, and several answers. Suppose you are smart teacher, please score the answers according to the two criteria. (1) Accuracy: whether the response is accurate concerning the image content. (2) Relevancy: whether the response directly follows the instruction without unrelated answers. There are four options for the scores: A score of I (Very Poor): The response is seldom relevant to the instruction and is mostly wrong according to the image. A score of 2 (Poor): The response is partly relevant to the instruction and partly accurate according to the image. A score of 3 (Good): The response is mostly relevant to the instruction and has minor errors according to the image. A score of 4 (Excellent): The response is completely relevant to the instruction and completely accurate according to the image." "
Evaluator
Ours MiniGPT4
LLaVA InstructBLIP MMGPT mPLUG-Owl
Expert](1-4)
3.48
2.61
2.87
3.00
1.90
2.90
1xper12(1-4)
3.58
2.23
2.07
2.48
1.05
2.27
11))rt3(1-4)
3.33
2.58
2.89
2.94
1.38
2.91
GAVIE-Accuraa (0-10)
6.58
4.14
4.36
5,93
0.91
4.84
(0-10)
8.46
5,81
6.11
7.34
1.79
6.35
Table 8: GAVIE vs. Human Evaluation. GAVIE scores roughly align with the expert ratings. Numbers highlighted with red, orange, black, green, blue, and magenta indicate rank to 6.
To evaluate the results quantitatively, we assign different scores for the options: Very Poor=1, Poor=2, Good=3, Excellent=4. From Tab. 8 all experts agree that the output from followed by InstructBLIP in second place, and MMGPT performs the worst. The observation is similar that of GAVIE evaluation results. Although the ranking orders of MiniGPT4 and LLaVA from experts are not always the same as that of GAVIE, the scores assigned to them are fairly close. One possible reason is that the answers from MiniGPT4 and LLaVA tend to be longer, making them more challenging for humans to evaluate.
model is the best,
A.1.2 STABILITY OF GPT4-Assisted Visual Instruction Evaluation (GAVIE)
This section investigates the stability of GAVIE Precisely, we execute GAVIE 5 times on the model predictions. We leverage two metrics to measure the stability of GAVIE on each instance: Mean and Standard Deviation (STD). The average scores of the evaluation set are shown in the following table. From the perspective of the Mean, the ranking order of ACCURACY and RELEVANCY is the same as Tab. 8 As for the Standard Deviation in Tab. 9 it ranges from 0.65 to 2.46. From our observation, the ACCURACY and RELEVANCY scores of instance may vary between different times, but they belong to the same grade level. Specifically, RELEVANCY has four grade levels: (1) The response is completely relevant (9-10), (2) The response is mostly relevant (6-8), (3) The response is partly
13

---

Published as a conference paper at ICLR 2024
relevant (3-5), (4) The response is seldom relevant (0-2). ACCURACY has four grade levels: (1) The response completely accurate (9-10), (2) The response has minor errors (6-8), (3) The response is partly accurate (3-5), (4) The response is mostly completely wrong (0-2).
Metric
Ours MiniGPT4
InstructBLIP MMGPT mPLUG-Owl LLaVA
AcCURACY(GPPT4)-Meen 6.60
3.76
5.29
0.87
4.84
3.80
RELEVANCY((GPT4)-Memn
8.37
5.35
6.83
1.71
6.35
5.65
ACCURACY(GPP44--TD 2.42
2.46
2.42
0.65
1.96
2.37
RELEVANCY(GPT44:ST 1.30
1.99
1.88
0.81
1.48
2.18
Table 9: Evaluation of the stability of GAVIE. We run GAVIE 5 times on the randomly selected instances from the evaluation set. Mean and Standard Deviation(STD) are calculated to measure the stability. The metric scores of ACCURACY(GPTA)) and RELEVANCY(GPT4) are from 0 to 10.
A.2 MORE EXPERIMENTS
A.2.1 Do LMMs perform better on Positive or Negative Instructions?
Our evaluation set consists of positive and negative instances. We divide into two sets and analyze the model performance each. As shown in Fig. 8 baseline models, including MiniGPT4, LLaVa. and InSP perform better on positive instances than negative ones, as the training data adopted by these models do not contain negative instructions. MMGPT performance poorly on both sets due LLaVA
to many repetitive phrases in the response. In addition, we found that the degradation is the most severe. We hypothesize that the synthetic answers for instruction tuning in LLaVA generally longer and involve more unrelated information. contrast, our model performs the best in both sets. InstructBLIP performs with higher scores than other LMMs because of the effectiveness of its instruction-aware visual encoder to extract image information.
A.2.2 Do LMMs perform better different formats and lengths of instructions?
From Tab 13 LMMs perform with higher scores on interrogative instructions than declarative, but the difference is relatively small. Even though recent visual instruction tuning datasets lack diverse declarative instructions, the LMMs built on L.M are powerful enough to understand and follow the declarative instructions. From Fig. g current LMMs achieve better results in short instructions than long ones since longer instructions contain more information, making it more challenging.
A.2.3 How do LMMs perform on more benchmarks?
We analyze the discriminative task of AMBER (Wang et al. 2023 object attribute, and object relation hallucination. The evaluation score F1, and the groundtruth answer is yes or no. As shown in Tab. [0] although our model achieves slightly lower Fl score regarding object existence, we outperform LLaVAI.5 in both the object attribute and object relation hallucinations. We attribute our model's success the existent object (attributes and relationship) manipulation in the negative instructions. We also analyze Hallusionbench (Liu et al. 2023a), an magee-context reasoning benchmark covering various topics and image types. The groundtruth answer is yes no. From Tab. 11 we find that both LLLVVAI. ..5 and MiniGPT4-v2 (Chen et al. 2023) achieve high accuracy on the positive set but perform less favorably on the negative set. Our model achieve similar level of accuracy when the groundtruth answer yes and much higher accuracy when the groundtruth answer no. We attribute the success to the knowledge manipulation in the negative instructions.
including object existence,
Overall, LLaVA 1.5 performs well when meeting object hallucinations. However, attribute hallu- cination, relation hallucination, and knowledge hallucination still challenging for LLaVA 1.5. Therefore, our LRV IVV-nnstruttion dataset with various negative instructions and positive instructions is beneficial to address the hallucinations.
14

---

Published as a conference paper at ICLR 2024
mPL.UG-Owl-78 MiniGPT4-12-78 LLaVA1.5-78 Ours
EXISTENCE
0.29
0.80
0,83
0.81
ATTRIBUTE
0.34
0.41
0.64
0.70
RELATION
0.26
0,58
0.65
0.69
Table 10: Comparison results on AMBER make a fair comparison.
Wang et al. 2023). All the LMMs are 7B versions to
LLaVAL.--BB MiniGPT4-v2-7B Ours
ACCURACY (GT=YES)
0.92
0.85
0.88
ACCURACY (GT=NO)
0.15
0.11
0.47
Table 11: Comparison results Hallusionbench to make a fair comparison.
(Liu al. 2023a). All the LMMs are 7B versions
A.3 PROMPT DESIGN
A.3.1 POSITIVE INSTANCE GENERATION BASED VISUAL GENOME DATASET
We show two full examples of our input prompts in (i) Fig.
16 In and (ii) Fig. 14
Fig. II and Fig. 14 we first present the images for the two examples, but they are not included in the text prompt for GPT4. As for the text input, we leverage the groundtruth bounding boxes and dense captions to represent the visual content as GPT4 can see the image. After that, we randomly select 10 tasks from the 16 seeds and ask GPT4 to generate 20 instances for these tasks. Additionally, there can be more than one caption describing the same object with different attributes, such as "woman wearing long dress" and "woman wearing yellow dress" in Fig. II] Although we present the bounding box coordinates each caption to GPT4, it can be easily confused, treating them as two instances, one in long dress and the other in yellow dress. To mitigate this issue, add "highly overlapping bounding boxes may refer to the same object" into the prompt to help GPT4 understand the "visual" input better. To enrich the instructions, we ask GPT4 generate instances in both declarative and interrogative formats. We also explicitly instruct GPT4 with "The answers should be less than 30 words" " as a requirement to reduce the chance of generating extra unrelated information in the training data. In order to make the output of GPT4 in good format, we also ask GPT4 to generate an instruction, an answer, and task name in order at the end of the prompt (Fig.
11 and Fig. 14). The full output of instructions and answers are shown in Fig. 12 13 and Fig.
15
16
We also present more positive instances with the output from different LMMs Fig. 29
30
31
A.3.2 POSITIVE INSTANCE GENERATION BASED ON CHART IMAGES
We collect chart images from Tang et al. 2023), which has human-annotated captions describing the construction and patterns of charts. We instruct GPT-4 to generate question-answw answers pairs with captions as visual input. The detailed prompt is shown in Fig. 5
We also present more positive
instances with the output from different LMMs in Fig. 32
A.3.3 NEGATIVE INSTANCE GENERATION NONEXISTENTT/XXISTENN OBJECT MANIPULATION
We show two full examples of our input prompts in (i) Fig. 18 and Fig. 20
In Fig.
17 and Fig. 19. we present the images to help readers understand dense captions better but they are not included in the text prompt for GPT4. We leverage the bounding boxes and dense captions as the "visual" input. As for Nonexistent object Manipulation in 17 we ask GPT4 to generate 6 instructions with nonexistent elements (nonexistent objects, nonexistent activities, nonexistent attributes, nonexistent interactions). As for Existent object Manipulation in 19 we ask GPT4 to generate 6 instructions of existing objects with wrong attributes. At the end of the text prompt, ask GPT4 to generate an instruction and a reason to explain why the instruction is inconsistent with the image in order. The reason is regarded as the answer for the instruction in our training data. Fig.
15

---

Published as a conference paper at ICLR 2024
GAVIE
Ours
ILLVA1.55
MiniGPTv2
MiniGPT4
LLaVA
InstructBL.IP MMGPT mPLUG-OWl
Acc (0-10)
6.58
6.42
6.01
4.14
4.36
5.93
0.91
4.84
RELE (0-10)
8.46
8.20
8.10
5.81
6.11
7.34
1.79
6.35
Table 12: More comparison results on our evaluation set evaluated by GAVIE. Ours means Finetumed mPLUG-Owl-7B. All the LMMs are 7B versions to make fair comparison. Rele means Relevancy.
Prompt:
Here is the description of a chart "Here is area chart is labeled Cumulative number of patients diagnosed with coronavirus (COVID-19) in Japan of December 4, 2020, by place of infection. On the x-axis, Month is measured with a categorical scale starting with April and ending with October. There is linear scale with a minimum of 0 and a maximum of 150, 000 along the y-axis, labeled Patients within Japan
Please generate different questions and answers pairs about title, x-axis, y-axis, data range or data pattern of the chart. The answers should come from the descriptions above. Each Answer must be less than 20 words. The output format should be as follows:
Connutative number of patterts diagroned
question1=>
answert =>
question2=>
$2.000
answer2
question3=>
answer3=>
GPT4 OUTPUT Example:
question Tuestoonn=>WWatdooo does the area chart represent?
answer1 Cumulative COVID-19 cases in Japan by place of infection from April to October 2020.
question2=> What does x-axis represent?
answer2 => Months from April to October 2020
question3=> When did the greatest increase in COVID-19 cases in Japan occur? answer3=> Between November and December 2020.
Figure 5: An example prompt for text-only GPT4 we use generate instruction and answers for chart images. The sentence in BLUE is the captions of the chart.
18 and Fig. show the full output from GPT4. We also present more negative instances with the output from different LMMs in Fig. 27 28
A.3.4 NEGATIVE INSTANCE GENERATION KNOWLEDGE MANIPULATION
knowledge manipulation, we use GPT4 to manipulate the knowledge in the captions, including named entities and events.
As for the Neg3:
Prompt:
Please change the knowledge including keywords, name entities or event elements in the description "Cumulative COVID-19 cases in Japan by place of infection from April to October 2020" Output format should be as follows:
answer=>
GPT4 OUTPUT Example:
"Cumulative influenza cases France by region of infection from March to October 2020."
An example prompt for text-only GPT4 we use to generate negative instruction. The next step is to transfer the ouput into an interrogative sentence whose answer is "yes" or "no". As shown in Fig. 6 GPT4 manipulates the "Japan", "COVID-19" and "April"" in the original captions. After that, instruct GPT4 transfer the output sentence into interrogative sentence whose answer is "yes" or "no". Finally, we combine "No." and the original answer as the final answer: Question: Did the image show the cumulative influenza in France by region of infection from
Figure 6:
16

---

Published as a conference paper at ICLR 2024
March to October 2020? Answer: No. Cumulative COVID-19 cases in Japan by place of infection from April to October 2020".
A.3.5 PROMPT DESIGN FOR EVALUATING KNOWLEDGE HALLUCINATION
As for the knowledge level hallucination, we will use the groundtruth answers as a reference and compare them with predictions of models. A prompt example for GPT4 is shown in Fig. 7
Prompt:
Imagine you are an intelligent teacher. Thoroughly read both the reference answer and the prediction answer to ensure clear understanding of the information provided. Assess the accuracy of the predictions, and note that a prediction answer be considered incorrect if it contradicts the reference answer.
Question:
What does the volume of U.S. imports of trade goods in 2009 experienced increase, decrease or steady?
Reference answer:
The volume of U.S. imports of trade goods in 2009 experienced significant decrease, with drop between -0.2 and -0.3.
Prediction answer: The volume of U.S. imports of trade goods experienced a decrease
GPT4 OUTPUT Example:
The prediction answer states "The volume of U.S. imports of trade goods experienced decrease". This matches with the reference answer which says "The volume U.S. imports of trade goods in 2009 experienced a significant decrease". Therefore, the prediction answer is correct.
Figure 7: An example prompt for text-only GPT4 we use to evaluate knowledge manipulation instruction. The sentences in BLUE are the questions, reference answers, and predictions of models.
Categories
Metric
Ours
MiniGPT4
LLaVA
InstructBLII
MMGPT
Interrogative
AcCURAcY(GPT4)
6.61
4.14
4.60
5.95
1.01
Interrogative (ELEVANCY(GPTT)
8.46
6.20
5.88
7.67
2.00
Declarative
ACCURACY(GPT4)
6.50
3.98
3.82
5.47
0.90
Declarative
RELEVANCY(GPT4)
8.21
5.39
5.84
6.64
1.62
Table 13: Evaluation results on Interrogative Instructions and Declarative Instructions by GAVIE. The metric scores of ACCURACY(GPT44) and RELEVANCY(GPT4) are a scale of 0 to 10.
A.4 MORE DATASET STATISTIC
I summarized the popular words in the knowledge manipulation generated by GPT4 in Fig. and found they mainly include six categories: event, number, date, persons, place, and others. Some examples are shown below.
Canada, increase, decrease, lowest, 2009, United States, 2016, employment, unemployment, higher, 2013, 2017, 2015, drop, minimum, worst, consistent, kingdom, x-axis, y-axis, under, Italy, pie, bar..
17

---

Published as a conference paper at ICLR 2024
Accuracy(GPT4)
Relevancy(GPT4)
Durs
MiNGFPT
LLaVa
InstructLIP
MINPPT
Ours
MiNGPTI
LLaVa
listnutBUPP
Regattive
Pootthn
Konathee
(a) Accuracy Performance.
(b) Relevancy Performance.
Figure 8: Evaluation results positive and negative instructions by GAVIE.
Accuracy(GPT4)
Relevancy(GPT4)
Ours
MINFFT
LLIVA
InstNCBBPP
MIMEPT
Ours
MIGGFT
JLOVA
InSTuUBUUP
MMMFFT
mlongn-222
Lenghot2
#lenghi2
(a) Accuracy Performance.
(b) Relevancy Porformance.
Figure Evaluation results on different instruction lengths by GAVIE.
Perception
Existence Count Position Color Posters Celebrity
Scene Landmark Artwork OCR
Original MiniGPT4
68.33
55.00
43.33
75.00
41.84
54.41
71.75
54.00
60.50
57.50
Finetuned MiniGPT4
115.0
88.33
68.33
96.67
71.42
72.35
122,00
104.34
77.50
80.00
Original mPLUG-OW
120.00
50.00
50.00
55.00
136.05
100.29
135.50
159.25
96.25
65.00
Finetuned mPLUG-OWl
165.00
111.67
86.67
165.00
139.04
112.65
147.98
160.53
101.25
110.0
Table 14: Completed experiments of Perception on MME Fu et al.
(2033 benchmark.
Cognition
Commonsense Reasoning Numerical Calculation
Text Translation Code Reasoning
Original MiniGPT4
59.29
45.00
0.00
40.00
Finetuned MiniGPT4
76.42
55.00
77.50
67.50
Original mPLUG-Owl
78.57
60.00
80.00
57.50
Finetuned mPLUG-OWl
100.71
70.00
85.00
72.50
Table 15: Completed experiments of Cognition MME Fu et al. 2023 benchmark.
18

---

Published as a conference paper at ICLR 2024
Other
7.8%
Datc
18.0%
person
event
7.2%
29.9%
place
14.4%
number
22.8%
Figure 10:
Distribution of Knowledge Manipulations. The knowledge mainly includes six categories: event, number, date, persons, place, and others.
19

---

Published as a conference paper at ICLR 2024
Positive Instance Generation
Image:
Input Prompt:
Give an mage with following information: bounding box, positions that are the object left.top comer coordinates(X, Y), object sizes Height). overtapping bounding boxes may refer to object.
bounding box:
alephant heard on rocks X: 73 V: 80 Width: 418 Height: 4188
woman wearing straw hat X: 179 V. 285 Width: 23 Hoight 43
wearing dress 326 Width: 95
V. 258 Width: Height
group green
orange bucket sidewalk X:
Witth: Halght:
man wearing white chirt 204 Y: 439 Width: 51 Hoight 02
baby elephant behind adut elephant X 244 V: 235 Width: 119 Height: 158
of umbreliae ellee deck 82 Y. Width: 136 Height:
person weering yellow
270
Height::45
water green X:
Width: Height: 39
baby clephant x: 297 V. 317 Width: 60 Height
an orango bucket on the ground X: 91 V: 341 Widt:: 38 Height: 36
group while umbrellas x 59 82 Widti 112 Height: 28
group grean chairs
Y. Width: Height:
man orango shirt 204 Y:
Hoight
blue bn awning X Y: 190 Width: BB Height: 66
woman wearing yellow dress X. 169 V: 298 Width: 47 Height 76
wearing
hal 173 V: 268 Width: 38 Height
sitling
Width: Height
man wearing a white
Width: 80 Height
elephant coming out the water X: 384 Y. 219 Width: 68 Height:
man silling in chair X 200 V. 426 Width: 79 Height: 67
orange basket X: 68 V 329 Withh 77 Height: 69 blue chairs
525 Width: 58 Height:
alephants walking on
161
261 Height 239
baby elephant 280 Y: 285 Width: 98 Height: 105
white umbrellas outside building X: 91 68
Width: 181 Height 53
while liled staircase X 47 367 Width: 109 Height 126
people standing photo X: Y: Width: 75 Height 73
Wicth: Height:
plaelic blue chairs 148 318 Width: Height: 129
herd ssian elephants X: 83 Y: 110 Width 409 Height: 308
the elephant stopping out of the river X: 388 V: 216 Width: 86 Height 114
small alophant X: 302 Y: 309
Hoight:
man dressed white this man is dressed in white 208 Y: 416 Width: 49 Height
418 Width: Height
she is wearing sun hat and dress X: 175 V: 287 Width: 64 Height: 112
the people are gathered X: 88 Y: 185 Width: 163 Height: 157
watching the elephants X:87 167 Witth: Height
person recording elephants
Height:
repid Bowing river 349 73 Wothh Height: 243
herd a dlephants moving through a town 78 Y: 108 Width: 386 Height: 274
the trunk an clophant 395 261 Width: 15 Height 21
of buicing x:
Height: 104
patio
Widhh: Height:
white umbrollas patio
Y:
113 Haight: 30
trunk of an olaphant x: 335 Y: 254 Width: Hoight 27
the tail of an elephant X: 308 V: 233
Width: 16 Height 32
bucket
tree patio X Y: Width: Height: 33
Width: Height:
the leaves of tree x: 361 36 Width: 43 Height: 23
Task: image captioning, Image Sendmont Activit Recognition, Objoct Intoraction Analysis, Object Attribute Detection, Mull-choice VQA, Visual Entailment, Vieual Commoneense ressoning, Styled Image Caption, Referential Expression Grounding.
above different language styles accurate answers. instructions should contain interrogatvv sentences and declarativ sentences, The answars should be less than 30 words, Each task should have less than instructions.
diverse Instructions
Output format
Instructon:
instruction example
Answar:
answer example
Task: sample
Figure 11: The first example for generating positive instruction and answers (Partl). The image is not included in the text prompt.
20

---

Published as a conference paper at ICLR 2024
Positive Instance Generation
Output:
Instruction:
Craft briof narrative about the baby elephant and adult slophant.
Answer:
A baby elephant depicted behind edult elephaitt, possibly seeking protection.
Task:
Image Caplioning
Instruction:
Write concise description of the entire scene this image.
Answer:
clephants, including baby elephants, noar a river people watching and taking photos. also green chairs, umbrellas, and blue awnings nearty.
Task:
Image Caplioning
Instruction:
What is overall senfiment conveyed by this image?
The sentiment the image predominantly peaceful and joyful, showcasing the hermonious interaction of wildlife with human presence.
Task:
Image Sentiment Analysi
Instruction:
How would interprat the sentiment of the scene involving the man in while shirt?
The man while shirt exudes feeling calmness and interest as observes scene.
Task:
Image Sentimet Analysis
Instruction:
Explain how the clophants and the humans interact this image.
observing and photographing the clephants, elephants seem moving freely peacefully around them.
Task:
Object Interaction Analysis
Instruction:
Analyze the interaction between the objects in the foreground and the background.
Answer:
foreground, featuring green people, provides
background elephants moving.
Task:
Object Interaction Analysis
Instruction:
Which type of animal la fealured prominently in the image? Choices: A) Lions B) Elephents C) Girafles D) Zebras
Answer:
Elephents
Task:
Multichoice VQJA
Instruction:
What is the primary color the umbrallas present the image? Choices: A) Blue B) Green C) White D) Red
Answer:
Task:
Multi choice VQA
Basod on the image, does entail that the people are afraid of elephants?
Answer:
the does entail that people afraid of clephants.
Visual Entaiiment
Based on the image, does entail that locesion is in city center or on the outskirts?
Answer:
The entails the location the outskitts,
center. The presence herd elephants suggests location
Task:
Visual Entaliment
Figure 12: The first example for generating positive instruction and answers (Part2).
21

---

Published as a conference paper at ICLR 2024
Positive Instance Generation
Output:
Instruction:
Identity and describe the attributes of the bucket in the image.
Answer:
The bucket is orange, possialy made plastic, and appears to on sidewalk or palio.
Task:
Object Attribute Detection
Instruction:
Assess the color of the baby elephant. Is it gray brown?
Answer:
dota, cannot accurately
color baby elephant.
Task:
Object Artribute Detection
Instruction:
Based on the image, what the probable reason for the congregation of people?
Answer:
probable reason congregation people
passing through.
Task:
Visual Commonsense Reasoning
Why might the baby elephant be doee to the adult elephant?
Answer:
baby elephent likely
elephant protection guidance they move through
Vieual Commonseniee Reasoning
Locate "the man dressed white"
Answer:
coordinates (208, 416), width height 82.
Referential Expression Grounding
Identify the location of "the woman wearing straw hat".
Answer:
The woaring hat located 179 285 with Width: 23 and Heiight: 43.
Referentia Expression Grounding
Instruction:
Provide poetic caption for the image.
Answer:
Beneath sky kissed
parade ancient giants meanders, while observers stand stll hushed admiration.
Styled Image Caption
Instruction:
Caption image in romantic style.
Answer:
gently touched nature's grandeur, silent stories of love companionship as the elephents by:
Styled Image Caption
Instruction:
Identiffy primary activity happoning
by the image.
Answer:
The primary adtivity in the image is observing and photographing herd of elophants moving through a town.
Task:
Activity Recognilion
Instruction:
analyzing given
you about
image?
Answer:
The main event the image hard of elephants moving through town while baing observed and photographed by peaple.
Task:
Activity Recognttion
Figure 13: The first example for generating positive instruction and answers (Part3).
22

---

Published as a conference paper at ICLR 2024
Positive Instance Generation
Image:
AIT
Input Prompt:
image with following sizes(Width, Height). Highly everlapping bounding boxes may refer to the same object
bounding posilions object tll coordinates(X, object
baunding
eyebrow on the man X: 79 Y: 77 Width: 36 Height: 13 forehead
52
Height 28
two people playing game X: 2 Y: 1 Width: 330 Height: 487
hair
X: 52 11 Width: Height:
wriling on mans shirt X: 0 Y: 366 Width: 91 Height: 47
logo
Y: Width: 36 Height 57
wil controller in right hand X: 144 Y: 351 Width: 107 Height: 76
pair of glasses the
Width: Haight
the head of a man playing game X: Y: Width: 153 Height: 203
upper teeth the young X: 113 Y: 154 Width: 28 Height:
blurry left hand the man X: 192 310 Width: Height:
blue tshirt the man Y: 171 Width:
Height: 254
cords of the wii game 234 Y: 387 Width: 52 Height: 111
wide set of blinds 49 Y: Width: 279 Height: 270
a young man on the sofa 197 Y: 190 Width: 134 Height: 278
writing and an insignia a boy wearing glasses 34 37 Width: 132 Height: 118
shirt X: 325 Width: 88 Height 85
boy playign game 49 Y:
Width: 137 Height 133
a white remote 138 Y: 341
Width:
Height: 74
earing shirt 24 Y: 212 Width: 164 Height 269
glasses on mans face
Y: Width: Height 44
nintendo wii remote control X: 144 Y: 350 Width: 70 Height: 43
nintendo wii gaming nunchuk 198 Y:
Width: 48 Height: 47
the man is asian X: 11 Y: 20 Width: 247 Height: 430 the man has a controller X:
332 Width: 145 Height:
Task: Image Captioning, Image Sentiment Analysis, Activity Recognition, Object Interaction Analyysi, Object Altribule Delection, Muli-choice VQA, Visual Entaiiment, Visual Commonsense Reasoning, Styled Image Caption, Referential Expression Grounding.
Come up with 20 diverse instructions for all the tasks above with different language styles and accurate anewers. The instructions interrogative sentences have less than 3 instructions.
should
Oulput format:
Instruction:
instruction example
Answer:
answer example
Task:
task sample
Figure 14: The second example for generating positive instruction and answers (Partl), The image is not included in the text prompt.
23

---

Published as a conference paper at ICLR 2024
Positive Instance Generation
Output:
Instruction:
Identify the primary activity taking place in the described image.
Answer:
The primary activity two people playing game.
Task:
Activity Recognition
What the sentiment expressed in the image based on the given details?
Answer:
The sentiment is joyful and engaged, as depicted by man playing game.
Image Senliment Analysis
Provide concise caplion for the given image.
Answer:
An Asian man joyfully playing Wii game on the sofa, wearing blue shirt with an insignia.
Task:
Image Captioning
Instruction:
Specify the emotion depicted the image based the objects
Answer:
The emotion depicted happiness, inferred from the activity and facial features the
Task:
Image Sentiment Analysis
Instruction:
Based on the details given, state which object interacting with the man.
Answer:
The man is interacting with the controller.
Task:
Object Interaction Analysis
Instruction:
Identify the unique attribute associated with the man the given image.
Answer:
The unique attribute is that the man is Asian and wearing glasses.
Task:
Object Attribule Detection
Instruction:
From the objects, which
the
Answer:
The glesses closest man's
Task:
Referential Expression Grounding
Instruction:
Is the statement "The man is playing a video game" true or false based on the image details?
Answer:
True
Task:
Visual Enlailment
Instruction:
Based on the information, is reasonable conclude that the is indoors?
Answer:
Yes, It's reasonable, due to the presence of a sofa and set blinds.
Visual Commonsense Reasoning
Instruction:
caption for the image with humorous style.
Answer:
"Mulitasking man: mastering Wii, sporting stylish glasses, showing insignia
Task:
Styled Image Caplion
Figure 15: The second example for generating positive instruction and answers (Part2).
24

---

Published as a conference paper at ICLR 2024
Positive Instance Generation
Output:
Instruction:
image.
Anewer:
shirt
Task:
Object Attribute Detection
Instruction:
Based on the details, can we infer that the game being played is Nintendo Wii gamn??
Answer:
Yes, we can infer that the game being played is a Nintendo Wii game due the presence of Wii controller and nunchuk.
Task:
Visual Commonsense Reasoning
Instruction:
Describe the location
in relation the set blinds.
Answer:
The man posilioned front the set blinds.
Task:
Referential Expression Grounding
Instruction:
statement boy playing game
be
Answer:
specified controller, controller.
Task:
Entailment
Identify any identifiable attributes
given image.
Answer:
young, Asian, glasses,
logo writing, using controller.
Task:
Attribtee Detection
Instruction:
Identify the object the man holding in his right hand based on the image. A) PS4 controller B) Wil controller C) Mobile phone D) Book
Answer:
B) Wil controller
Task:
Muli-choice VQA
Instruction:
Detemine the color the man's shirt from the image description. A) Red Black Blue D) Green
Answer:
C) Blue
Task:
Muli-choiee VOA
Instruction:
Given the data, state the activity that the man and the boy are perticipating
Answer:
man bay participating gaming.
Task:
Activity Recognition
Instruction:
Using an old-fa ad-fashioned style of speech, describe the image portrayed..
Behold, a youthful gent of Asian descent, donned in speciacies and blue tunic, merrily engrossed in game of the Nintendo Wii.
Slyled Image Caplion
Instruction:
image,
description image.
Answer:
An engaged Asian with glasses and logo-clad blue shirt, situated sofa, immerses himself in exhilarating Wii
Task:
Image Captioning
Figure 16: The second example for generating positive instruction and answers (Part3).
25

---

Published as a conference paper at ICLR 2024
Negative Instance Generation (Nonexistent Element Manipulation)
Image:
Input Prompt:
Give an image with following information: bounding box, positions that are object left-top corner
coordinates(X, Y), object sizes(Width, Height). Highly overlapping bounding may refer the same object.
bounding box:
there light blue sky the distance X: 366 Y: 15 Width: 17 Height: 42
there white surfboard here X: 201 Y: 113 Width: 43 Height: 100
there are black swim trunks here X: 276 Y: 146 Width: 34 Height: 67
this man has long hair here X: 337 Y: 112 Width: 20 Height: 37
there is some wild water here X: 58 Y: 63 Width: 51 Height: 96
jackson mingus took this photo X: 173 31 Width: Height: 174
this man has very hairy legs X: 261
Width: 15 Height:
there wonderful amount of coloring 176
Width: 122
Height: 167
this photo will
published
magazine
Y: 27 Width: 64 Height: 120
this photo really artistic example X: 179 Y: 34 Width: 113 Height: 162
black and white surf board X: 190 Y: 85 Width: 106 Height: 150
black cotton swim trunks X: 268 Y: 149 Width: 49 Height: 59
water splashed in air X: 51 Y: 65 Width: 174 Height: 137
black decal on surf board X: 221 Y: 153 Width: 57 Height: 75
man standing on surf board X: 248 Y: 92 Width: 120 Height: 135
man surfing ocean 188 Y: 77 Width: horizon where ocean meets the sky 250 Y: 42 Width: 190 Height: 54 calm blue ocean
white surfboard mostly out 412 Y: 77 Width: Height:
water
Y: Width: 95 Height: 165
around surfboard X: 341 Y: 178 Width: 152 Height: 50
wave breaking on shore X: 2 Y: 177 Width: 494 Height: 57
white foam in water X: Y: 203 Width: 184 Height: 32
black piece surf board X: 190 Y: 96 Width: 22 Height: 19
man riding a surf board X: 189 Y: 82 Width: 180 Height: 147
man on surfboard X: 197 Y: 57 Width: 206 Height: 163
white ocean spray from surfer X: 62 Y: 64 Width: 122 Height: 166
three blue fins surfboard with arms up balance X: 212 photograph taken the design bottom of surfboard X: 210 Y: 149 Width: 62
185 Y: 88 Width: 51 Height: 60
Width: 162 Height: 157
X: 62 Y: 12 Width: 430 Height: 218
Height: 67
lling off surfboard X: 26 Y: 12 Width: 470 Height: 223
Come up with misleading instructions with nonexistent elements (nonexistent objects, nonexistent activities, nonexistent attributes, nonexistent interactions) in the images with different language styles. The instructions should contain interrogative and declarative sentences. Please also explain the reason.
Output format:
Instruction:
instruction example
Reason: Reason example
Figure 17: The example for generating negative instruction and answers by nonexistent object manipulation (Partl). The image is not included in the text prompt.
26

---

Published as a conference paper at ICLR 2024
Negative Instance Generation (Nonexistent Element Manipulation)
Output:
Instruction:
Could you please zoom in the red bird sitting on the surfboard?
Reason:
There is no mention any bird, let alone a red one, the provided bounding box information, which means this element is nonexistent in the image.
Instruction:
Find the green umbrella on the beach and note its position.
Reason:
The provided bounding boxes do not include any information about beach or green umbrella. This makes the object nonexistent within the context of the image.
Instruction:
Identify the interaction between the surfer and the shark the water.
Reason:
Although there's a man and water in the image, there's no indication a shark. Therefore, there's interaction between surfer and shark in this image, making it nonexistent interaction.
Instruction:
Can you locate the cluster of palm trees in the background?
Reason:
The bounding boxes only describe light blue sky, surfboard, swim trunks, man, water, and wave there's no mention of any palm trees. Therefore, palm trees nonexistent element in this image.
Instruction:
Look for the sunset reflecting off the ocean waves.
Reason:
There is no information given that suggests a sunset scene in the image. The sky is
described as light blue, which suggests day scene, making sunset nonexistent attribute.
Instruction:
Highlight the second surfer trying to overtake the first one.
Reason:
The image only contains bounding box data for single man, who can be assumed to be
surfer based on other objects (like the surfboard). The second surfer therefore a nonexistent object.
Figure 18: The example for generating negative instruction and answers by nonexistent object manipulation (Part2).
27

---

Published as a conference paper at ICLR 2024
Negative Instance Generation (Existent Element Manipulation)
Image:
Input Prompt:
Give an image with following information: bounding box, positions that are object left-top corner
coordinates(X, Y), object sizes(Width, Height). Highly overlapping bounding boxes may refer the same object.
bounding box:
there light blue sky the distance X: 366 Y: 15 Width: 17 Height: 42
there white surfboard here X: 201 Y: 113 Width: 43 Height: 100
there are black swim trunks here X: 276 Y: 146 Width: 34 Height: 67
this man has long hair here X: 337 Y: 112 Width: 20 Height: 37
there is some wild water here X: 58 Y: 63 Width: 51 Height: 96
jackson mingus took this photo X: 173 31 Width:
Height: 174
man has very hairy legs X: 261 there wonderful amount of coloring X: 176
Width: 15 Height:
Width: 122
Height: 167
this photo will
published
magazine
Y: 27 Width: 64 Height: 120
this photo really artistic example X: 179 Y: 34 Width: 113 Height: 162
black and white surf board X: 190 Y: 85 Width: 106 Height: 150
black cotton swim trunks X: 268 Y: 149 Width: 49 Height: 59
water splashed in air X: 51 Y: 65 Width: 174 Height: 137
black decal on surf board X: 221 Y: 153 Width: 57 Height: 75
man standing on surf board X: 248 Y: 92 Width: 120 Height: 135
man surfing ocean 188 Y: 77 Width: horizon where ocean meets the sky 250 Y: 42 Width: 190 Height: 54 calm blue ocean white surfboard mostly out 412 Y: 77 Width: Height:
water
Y: Width: 95 Height: 165
wave around surfboard X: 341 Y: 178 Width: 152 Height: 50
wave breaking on shore X: 2 Y. 177 Width: 494 Height: 57
white foam in water X: Y: 203 Width: 184 Height: 32
black piece on surf board X: 190 Y: 96 Width: 22 Height: 19
man riding a surf board X: 189 Y: 82 Width: 180 Height: 147
man on surfboard X: 197 Y: 57 Width: 206 Height: 163
white ocean spray from surfer X: 62 Y: 64 Width: 122 Height: 166
three blue fins on surfboard with arms up to balance X: 212 photograph taken the design bottom of surfboard X: 210 Y: 149 Width: 62 Height: 67
185 Y: 88 Width: 51 Height: 60
Width: 162 Height: 157
X: 62 Y: 12 Width: 430 Height: 218
man falling off surfboard X: 26 Y: 12 Width: 470 Height: 223
Come up with misleading instructions with existing objects with wrong attributes in the images with different language styles. The instructions should contain interrogative and declarative sentences. Please also explain the reason.
Output format:
Instruction:
instruction example
Reason: Reason example
Figure 19: The example for generating negative instruction and answers by existent object manipula- tion (Partl). The image is not included in the text prompt.
28

---

Published as a conference paper at ICLR 2024
Negative Instance Generation (Existent Element Manipulation)
Output
Instruction:
Did you spot the surfer's pink board riding the wave?
Reason:
The surfboard in the image is described as white, not pink. This misleading instruction changes the color attribute of the surfboard.
Instruction:
Look closely, isn't that calm lake in the distance?
Reason:
The water body in the image is described part of ocean, not calm lake. This misleading instruction alters the context and setting of the photo.
Instruction:
The man in the picture is sporting a buzz cut, isn't he?
Reason:
The man in the image is described as having long hair, not a buzz cut. This misleading instruction misrepresents the attribute related to the man's hairstyle.
Instruction:
That's serene morning sky in the background, don't you think?
Reason:
The sky the image is described as light blue sky, which does not necessarily mean it's morning. This misleading instruction gives an incorrect time context.
Instruction:
Isn't that surfboard fully immersed in the water?
Reason:
The surfboard is described as being mostly out of the water, not fully immersed. This misleading instruction changes the positioning of the surfboard relative to the water.
Instruction:
The man on the surfboard is performing a complex trick, isn't he?
Reason:
The man on the surfboard is described as having his arms up to balance, which does
not imply he is performing a complex trick. This misleading instruction misrepresents
the man's action on the surfboard.
Figure 20: The first example for generating negative instruction and answers by existent object manipulation (Part2).
29

---

Published as a conference paper at ICLR 2024
GPT4-Assisted Visual Instruction Evaluation
Image:
Input Prompt:
Give an image with following information: bounding box. positions that are the object left-top comer coordinates(X,, Y), object sizes(WWdth, Height). Highly overlapping bounding boxes may refer to the same object.
bounding box: tray
Height:470
the tray white X:18 Y:56 Width:71 Height:471
some pieces pile of white rice X:218 Y:11 Width:196 196 Height:196
Width:142 14ight:142
carrol orange
Height:67
a piece of broccoll X:83 Y:130 Width:52 Height the spoon spoon and napkin in plastic wrapper X:0 Y:0 Width:135 Height:135
X:0 Y:7 Width:69 Height699
table beige colored X:0 Y:17 Width:488 Height-498
sauce on the tray X:362 Y/241 Width:72 Height:72
plastic spoon a beige tray X:0 Y:17 W98th-499 Height:499
wrapper X-1 Y:4 Width:70 Height:70
serving of white rice X:220 Y:115 Width:194 Height:194
beef cubes with brown sauce X:88 Y:151 Width:140 Height:140 brown a baby carrot X:173 Y:115 Width.70 Height.70
the side while container X:363 Y:228 Width:102 1eight:102
bots of cauliflower belween carrots X:136 Y:95 Widlh:76 Height.76
a bit of broccoli X:82 Y:127 Width:51 Height:51
niee beef and veggie plastic container X:83 Y:96 Width:332 Height:332
a white plastic container X:19 V:57 Width472 Height:472
circle of rice out up pieces of meat X:83 Y:173 Width:45 Height.45
plate X:280 Y:119 Width:135 Height 135
smail part of brocooli next to meat X:88 Y:130 Width:54 Height:54
up carrot
Y:98 Width:73 Height:73
meat sauce on the side of plate X-428 Y:228 Width:35 Height:35 cut up cauliflower small part of plastic spoon in the corner X:1 Y:2 Width:75 Height:75 folding
corner
Height:49
holding
small piece of napkin wrapped plastic X:67 Y:8 Width:84 Height
table attached
Width:72 Height:72
portion of cooked white rice X:213 Y:114 Width:200 Height:200
kalua pork mixed cooked vegetables X:85 Y:95 Width:162 1eight:162
Height:140
go container
vegelables
Height:469
meal in bottom half of go container X:21 Y:55 Width:469 Height:469
cooked piece X:177 Y:11 Width:70 Height:70
cooked caulfflower piece X:164 Y:95 Width:54 Height:54
piece of cooked chopped broccoli X-84 Y:126 Width:49 Height:49
cooked meat with skin on X:82 Y:153 Width:149 Height:149
go spoon and napkin plastic
X:3 Width:132 Height:132
serving white rice X:230 Y:131 Width:151 Height:187
mixed veggies in container X:99 Y:104 Width:145 Heighh::145
chicken with sauce in container X-89 Y:162 Width:132 Height:132
white container with food X:89 Y:65 Widthx428 Height:488
plastic spoon in wrap X:3 Y:3 Width:98 Height:98 beige with food on sauce inside white container X:57 Y:252 Width:67 Height:67
Y:28 Widthx487 Height:4B7
baby carrot beside rice X:167 Y:117 Width:78 Height:78
cauliflower beside baby carrot X:145 Y:105 W7 Height:7
crispy chicken with glaze X:91 Y:187 Width: 130 Height:130
Figure 21: The first example for the GPT4-Assisted Visual Instruction Evaluation (Partl). The image is not included in the text prompt.
30

---

Published as a conference paper at ICLR 2024
GPT4-Assisted Visual Instruction Evaluation
Here is the instruction for the image: Don't forget to notice the red wine glass beside the white container.
Answer1: The instruction is misleading because there are mentions of red wine glass the image. The declarative sentence this instruction suggests that the reader should pay attention to a non-existent object.
Answer2: Yes, I see the red wine glass and the white container.
Answer3: Based the image, the meal consists of combination of meat and vegetables, including rice, broccoli, and carrots, served on a white plate on a dining table. What can do for you today?
Answer4: there red wine glass beside the white container
Answer5: Don't forget to notice the red wine glass beside the white container. It's a nice touch. This a great idea. I'll have to try it. Thanks for sharing! love this ideal I'm definitely going to try it. Thanks for sharing! love this ideal I'm definitely going to try it. Thanks for sharing! love this idea! definitely going try Thanks sharing! love this idea! I'm definitely going to try it. Thanks for sharing! love this.
Suppose you are a smart teacher, after looking at the image information above, please score the above answers(O-10) according to the following criteria: whether the response directly follows the instruction whether the response is accurate concerning the image content.
Output format:
relevancy:
score of answer1:
reason:
score of answer2:
reason:
score of answer3:
reason:
score of answer4:
reason:
score of answer5:
reason:
accuracy:
of answer1:
reason:
score of answer2:
reason:
score of answer3:
reason:
score of answer4:
reason:
score of answer5:
reason:
Figure 22: The first example for the GPT4-Assisted Visual Instruction Evaluation (Part2).
31

---

Published as a conference paper at ICLR 2024
GPT4-Assisted Visual Instruction Evaluation
Output:
relevancy:
score of answer1: 10
reason: Answer 1 directly responds to the instruction by stating that the red wine glass isn't present in the image.
score of answer2:
reason: Even though the answer refers to the red wine glass, it's not relevant because there's mention of red wine glass image details.
score of answer3: 2
reason: Answer 3 does not directly respond to the instruction. describes food other objects in the image, but doesn't mention the red wine glass at all.
of answer4:
reason: Answer 4 directly states there is a red wine glass beside the white container, even though the image information doesn't support this.
score of answer5:
reason: Answer 5 doesn't address the instruction and instead offers multiple vague, irrelevant compliments about the idea.
accuracy:
score of answer1: 10
reason: Answer 1 accurately refers to the fact that there's no mention of a red wine glass in the image details.
score of answer2:
reason: Answer 2 inaccurately claims that there is red wine glass, even though there's mention of one the image details.
of answer3:
reason: Even though doesn't mention the wine glass, Answer provides a detailed and accurate description of the image based on the information given.
score of answer4:
reason: Answer 4 inaccurately claims that there is red wine glass beside the white container, contradicting the image details.
score of answer5:
reason: Answer 5 neither addresses the instruction nor accurately reflects the image content. It mentions a wine glass but there's no information about it the image details. Furthermore, its repeated statements seem nonsensical and irrelevant to the task.
Figure 23: The first example for the GPT4-Assisted Visual Instruction Evaluation (Part3).
32

---

Published as a conference paper at ICLR 2024
GPT4-Assisted Visual Instruction Evaluation
Image:
Input Prompt:
Give an image with following information: bounding box. positions that are the object left-top comer coordinates(X,, Y), object sizes(WWdth, Height). Highly overlapping bounding boxes may refer to the same object.
bounding box: tray
Height:470
the tray white X:18 Y:56 Width:71 Height:471
some pieces pile of white rice X:218 Y:11 Width:196 196 Height:196
Width:142 14ight:142
carrol orange
Height:67
a piece of broccoll X:83 Y:130 Width:52 Height the spoon spoon and napkin in plastic wrapper X:0 Y:0 Width:135 Height:135
X:0 Y:7 Width:69 Height699
table beige colored X:0 Y:17 Width:488 Height-498
sauce on the tray X:362 Y/241 Width:72 Height:72
plastic spoon a beige tray X:0 Y:17 W98th-499 Height:499
wrapper X-1 Y:4 Width:70 Height:70
serving of white rice X:220 Y:115 Width:194 Height:194
beef cubes with brown sauce X:88 Y:151 Width:140 Height:140 brown a baby carrot X:173 Y:115 Width.70 Height.70
the side while container X:363 Y:228 Width:102 1eight:102
bots of cauliflower belween carrots X:136 Y:95 Widlh:76 Height.76
a bit of broccoli X:82 Y:127 Width:51 Height:51
niee beef and veggie plastic container X:83 Y:96 Width:332 Height:332
a white plastic container X:19 V:57 Width472 Height:472
circle of rice out up pieces of meat X:83 Y:173 Width:45 Height.45
plate X:280 Y:119 Width:135 Height 135
smail part of brocooli next to meat X:88 Y:130 Width:54 Height:54
up carrot
Y:98 Width:73 Height:73
meat sauce on the side of plate X-428 Y:228 Width:35 Height:35
cut up cauliflower small part of plastic spoon in the corner X:1 Y:2 Width:75 Height:75 folding
corner
Height:49
holding
small piece of napkin wrapped plastic X:67 Y:8 Width:84 Height
table attached
Width:72 Height:72
portion of cooked white rice X:213 Y:114 Width:200 Height:200
kalua pork mixed cooked vegetables X:85 Y:95 Width:162 1eight:162
Height:140
go container
vegelables
Height:469
meal in bottom half of go container X:21 Y:55 Width:469 Height:469
cooked piece X:177 Y:11 Width:70 Height:70
cooked caulfflower piece X:164 Y:95 Width:54 Height:54
piece of cooked chopped broccoli X-84 Y:126 Width:49 Height:49
cooked meat with skin on X:82 Y:153 Width:149 Height:149
go spoon and napkin plastic
X:3 Width:132 Height:132
serving white rice X:230 Y:131 Width: Height:187
mixed veggies in container X:99 Y:104 Width:145 Heighh::145
chicken with sauce in container X-89 Y:162 Width:132 Height:122
white container with food X:89 Y:65 Widthx428 Height:488
plastic spoon in wrap X:3 Y:3 Width:98 Height:98 beige with food on sauce inside white container X:57 Y:252 Width:67 Height:67
Y:28 Widthx487 Height:4B7
baby carrot beside rice X:167 Y:117 Width:78 Height:78
cauliflower beside baby carrot X:145 Y:105 W7 Height
crispy chicken with glaze X:91 Y:187 Width: 130 Height:130
Figure 24:
The second example for the GPT4-Assisted Visual Instruction Evaluation (Partl). The image is not included in the text prompt.
33

---

Published as a conference paper at ICLR 2024
GPT4-Assisted Visual Instruction Evaluation
Here is the question for the image:
With respect to the image, is the statement "The napkin is wrapped in the plastic spoon" true or false?
Answer1:
True
Answer2: True
Answer3: In the image, the statement is false. The napkin is not wrapped in the plastic spoon.
Instead, there is spoon resting on the tray beside the foam container, which is filled with rice, meat,
and vegetables.
Answer4:
|0000000000000000000000000000000 00000000000000000000000000000000
00000000000000
Answer5: With respect to the image, is the statement "The napkin is wrapped in the plastic spoon" true or false? With respect to the image, is the statement "The plastic spoon is wrapped in the napkin" or false? With respect the image, is statement "The napkin wrapped the plastic spoon" true or false? With respect to the image, is the statement "The plastic spoon is wrapped in the napkin" true or false? With respect to the image, is the statement "The napkin is wrapped in the plastic spoon"
Suppose you are smart teacher, after looking at the image information above, please score the above answers(D-10) according to the following criteria: whether the response directly follows instruction 2: whether the response is accurate concerning the image content.
Output format:
relevancy:
score of answer1:
reason:
score of answer2:
reason:
score of answer3:
reason:
score of answer4:
reason:
score of answer5:
reason:
accuracy:
score of answer1:
reason:
score of answer2:
reason:
score of answer3:
reason:
score of answer4:
reason:
score of answer5:
reason:
Figure 25; The second example for the GPT4-Assisted Visual Instruction Evaluation (Part2).
34

---

Published as a conference paper at ICLR 2024
GPT4-Assisted Visual Instruction Evaluation
Output:
relevancy:
score of answer1: 10
reason: Answert gives a straight answer to the question as either or false.
score of answer2: 10
reason: Answer2 directly responds the question by stating if it's or false.
score of answer3: 10
reason: While Answer3 provides more details than necessary, it does answer the question directly and completely.
score of answer4: 1
reason: Answer4 does not address the question at appears to series numbers without any clear relation to the question.
score answer5:
reason: Answer5 does not directly answer the question. It repeats the question multiple times without providing answer.
Accuracy:
score of answer1: 1
reason: Answer1 is not accurate. The image information doesn't state that the napkin is wrapped in the spoon.
score answer2: 1
reason: Answer2 is incorrect. The image information doesn't state that the napkin is wrapped in the spoon.
score of answer3: 10
reason: Answer3 is accurate. The image information supports the statement made the answer that spoon and the napkin both plastic, that napkin is wrapped in the spoon.
score of answer4: 1
reason: The response is not relevant to the question and does not refer to the image content. It is not an accurate answer.
score answer5:
reason: This response fails to provide an accurate answer, instead repeating the question multiple times without providing any answer.
Figure 26; The second example for the GPT4-Assisted Visual Instruction Evaluation (Part3).
35

---

Published as a conference paper at ICLR 2024
Can you locate the water bottle placed near the yellow tennis ball on the ground?
Instruction:
MiniGPT4: The water bottle located near the yeliow tennis hall on the ground. Can you locate the water buttle placed near the yellow tunnis bull on the ground?
Unfortunately, objects specific environment. However, given image description, near the yellow tewnis ball on the ground, and the boy walking up to it, so you should be able to see the hottle soon enough.
1LaVA:
visual assistant, not capable moving locating bottle
InstructBLIP: Yes, the water bottle is located near the yellow tenels bali on the ground.
MMGPT: Yes, there water placed near the yellow tennis ball on ground.
Ours: There no object resembling a water bottle mentioned the image. This question Intraduces new abjert that doesn't mxist in the image.
(a) All LMMs except ours produce inconsistent descriptions with the images. MiniGPT4 also has repetitive
sentences.
Instruction: You might notice painting of sunset hanging over stove in the kitchen.
There i pointing of sunser hanging ahove the stowe in the kinchen, The kitchen hos blue and white color scheme, and there are some grey tiles on the finor. There is white oven on ane wall, a fiidge on another wall, and a microwave oven on the thind wail. The kitchen also has small dining table with two chairs, and there is small window above the cable.
MinKiPT4:
The polnting of surser hanging ahove stove creates a viuuell--ppesllig appeatling ambionce kicchen
LLVA:
warmth sence monquiiity space The lotchen appears empty, with stove, and some coblnets. Perhaps you could also consider adding a few decorarive elements, more inviting.
plants colorfal kitchen
towels, to further enhance the space
InstructBLIP There painting a sunset hunging obove the of wurmih and beusey the spuce. pointing decorative element complements overall visually appealing and inviting.
kitchen, the kitchen, making
MMGPT: What the color of the sunset in the pointing?
This instrction misleads the viewer into believing that there pointing the kitchen when there is mention o pointing in provided image.
Ours:
(b) MiniGPT4, LLa and InstructBLIP respond with long descriptive answers about the "painting", which does not exist in the image. Multimodal Mil produces question about the "painting." " but it does not address the instruction.
Instruction: Identify the pink flowers blooming at the bottom left corner of the image. Are they a seasonal variety or do they growyear- year- round?.
MinioPTi; image shows group cloai' the sky.
grazing the side mountain The appear appears
mountole covened norks and grazing field of green grass. The clouds the sky quite stormy day.
LLaVA: The pink flowers blooming ot the bottors left corner the image are libely sessonal variety, they soe growing your round. Their apporrance the image bloom.
that the photo was toärn during qpecific sesson when these flowers ane in
InstructBLIP: The pink-lowerr flowers are blouming year-round
MMGPT: Mdmntiy The pink flnwers blooming at the bottom left corner of the Image, Are they a seusonal variety or do they grow peat-round? they are seasonal, when they bloom? if they are yest-round, what is the name of the plant The pink flowers bluwming at the bottumn ieft corner of the imoge are a secsonal voriety, os they are in bloom during the ypring seuson.
Ours: This insruction misleading because are pink flowers image.
géven image primurit focuses outdoor objects sheep, mountaine, and rocks.
(c) MiniGPT4, LLaVA, and Multimodal-GPT respond with long descriptive answer about the "pink flowers", which do not exist in the image. Although InstructBLIIP answer is brief, it's still false.
Model predictions on negative instruction examples of Negl: object Manipu- lation". ". RED represents the nonexistent elements in the images.
Figure 27:
36

---

Published as a conference paper at ICLR 2024
Instruction: Is the white curtain on top of the grey cat actually a colorful curtain with butterfly patterns?
MiniGPT4: The cat actually
the curtain, which is not white but has different colors, such grey, brown and white.
LLaVA: Yes, white curtalic on top of the grey cat actually has butterfly pattern.
InstructBLIP: Yes, the loose stronds from the yorn blonker form butterfly pattern.
on top the grey actually colorful with
butterfly patterns.
Ours: This instruction misleading because the white curtain mentioned to be the the window, but not desrrihed colorful curtain with butterfly potterns.
(a) All LMMs except ours produce inconsistent descriptions with the images. Our model can point out that the loose strands not form butterfly pattern.
Instruction: Are the windows on the brick building red- framed?
MinGGPT4: The windows brick building red framed.
LLaVA: Yes, the windows on the brick building have red frames
InstructBLIP: The windows the building white-framed
MMGPT: yes
Ours: The windows the building are white white-framed
(b) All LMMs except ours produce false attributes of windows in the images. Our model can point that the windows are white instead of red.
Instruction: Does the woman with green hair have black backpack on her shoulder?
MiniGPT4: the woman has black backpack on her shoulder.
Yes, with green black backpuck un shoulder.
InstructBLIP Yes, the woman with green hair has black backpack on ber shoulder.
Yes, woman with green
backpock shoulder.
Ours: There is no mention of woman with green hair and black backpack in the image. This instruction misleading because it presents fulse attribute (the backpuck) that present in the image.
(c) All LMMs except ours produce inconsistent descriptions with the images. Our model can point out that the woman with green hair doesn't 'I have black backpack on her shoulder.
Figure 28: Model predictions on negative instruction examples of Neg2: "Existent object Manipula- tion". RED represents the wrong attributes of existent objects in the images.
37

---

Published as a conference paper at ICLR 2024
Instruction:
What objects are the toddler's feet?
MiniGPT4: Two teddy bears are sitting on the toddler's feet.
LLaVA: has teddy and shoe on their feet.
InstructBLIP: objects the toddler's feet socks shoes.
MMGPT: The toddler's feet are covered with blonket, and there is teddy bear on top of the blanket.
Ours: White
Object Detection Task. MiniGPT4, LLaVa, and MMGPT can't locate the objects correctly.
Instruction:
Choose the correct statement about the weather conditions image: (a) Cloudy and rainy, (b) Clear blue sky, Foggy and misty, (d) Snowy and cold
MiniGPT4: (d) Snowy and cold
LLAVA: The correct statement about the weather conditions in the image is Cloudy and roiny. The image features red stop sign the corner of street, and sky is filled with clouds, suggesting a rainy day. The presence rain in the image, as opposed to the other options, provides specific information about the weather conditions
an Supdiaa SY
STOP,
InstructBLIP: #.
MMGPT: The correct statement about the weather conditions in the image Foggy and misty.
(b) Clear blue sky
(b) Multi-choice VQA Task. InstructBLIP can't 't follow the instruction well by generating anything. LLaVA's answer seems reasonable, but it incorrectly detects the clouds in the image.
Instruction: From the image, tell me what part of the room tiled.
MiniGPT4: The room is tiled with black and white tiles.
LLaVA: The tiled area the room the kitchen.
InstructBLIP: The part of room that the floor.
MMGPT: The kitchen.
Ours: The floor of the room is tiled.
(c) Referential Expression Grounding Task. Other LMMs can't 't directly address the instruction by giving general answer, "kitchen," while our model can elearly predict the "floor".
Figure 29: Positive instruction demos with different tasks and predictions from different models.
38

---

Published as a conference paper at ICLR 2024
Instruction: Detect any unusual elements in image.
MiniGPT4: some onions, carrors, mushraoms, and parsley in the image.
pieces of located cutting board. Additionally, there is group of mushrooms and carrots on the table. These vegetables appear fresh and ready used ingredients for a meal. However, there are unusual elements the scene, as shows a typical assortment of cooking ingredients.
image,
The image features wooden cutting board topped with a variety of vegetables, including mushrooms, carrols, and parsley. These ingredients arranged pile the cutting board, ready to chopped or sliced for use in recipe or meal.
InstructBLIP:
MMGPT: The image features bowl of vegetables, including carrots, broccoll, and celery. There also knije placed on top of the bowl.
Ours:
There are no unusaal elements in the image.
(a) Image Anomaly Detection Task. Our model and LLaVA point there is normal image without unusual elements, while MiniGPT4, InstructBLIP, and MMGPT describe the image instead of addressing the instruction.
Instruction: The coffee table looks wooden. True False?
MiniGPT4: Folse
LLaVA: The coffee toble wooden.
InstructBLIP: The coffee table looks wooden. True.
MMGPT: True
Ours:
(b) Visual Entailment Task.
Instruction: Identify the interaction between the red chair and the book
MiniGPT4: The Interaction between the red chair and the book
LLaVA: In the image, the red chair is positioned next the bed, and arrangement suggests might have been reading in the bedroom or preparing to read before leaving the
book placed the
The presence the book on the chair indicates interaction between chair the book, they are proximity to each other, creating cozy and comfortable atmosphere for reading relaxing the bedroom.
InstructBLIP: The red chair placed near the white book.
MMGPT: interoction between the chair and the book
the chair is placed next to the book, creating cozy reading area.
Ours: book laying on the chair.
(c) Object Interaction Analysis Task. LMMs except ours describe the wrong location of the book. LLaVA generates long text with unrelated information to address the instruction.
Figure 30: Positive instruction demos with different tasks and predictions from different models.
39

---

Published as a conference paper at ICLR 2024
Instruction: What is the color and state of the horse's bridle?
MiniGPT4: The horse's bridle black and located red truck..
LLaVA: The horse has white bridle, and its harness appears to be thrown trailer.
InstructBLIP: The horse's bridle is brown.
MMGPT: The horse's bridle and is in a ready- to--ot-eeee wear state.
Ours: The bridle is pink and attaches the horse's head.
(a) Dense Description Task. LMMs except ours incorrettly recognize the color the bridle. InstructBl also fails to answer the "state of the horse's 's bridle" in the instruction.
Instruction: Create caption for the image using sentence that describes the goats and their surroundings.
MiniGPT4:
The goats grazing the hillsiee, surrounded by rocks and goots
mountains, sky brilliant environment.
their
the image, group of four goats seen grazing on grass plants on top of a rocky hillidee. They are scattered around the area, with some of them eating grass close the rocks, while others grazing further away. The goots surrounded by mountainous landscape, which provides backdrap for this peaceful moment.
of goats grazing on racky hillside, surrounded by mountains in the background The goats appear to be eating grass and other vegetation growing the rocky terrain. The mountuinous landscape provides backdrap animals, adding
InstructBLIP:
natural beauty of the
MMGPT: The caption should be between words. herd of goats grazing in
Ours:
Flock of goats feasting green shrubs bockground, blue sky and cloudy weather.
a hilltop with large rocky mountains
(b) Image Caption Task. LLaVA and InstructBLP follow the instruction for generating one caption.
Instead, they generate long text with several sentences.
Instruction: What action is the person closest to the frisbee performing?
MiniGPT4: The person closest the frisbee throwing
The closest the frisbee preparing catch it, with their amms outstreiched.
LLaVA:
The person closest the frisbee attempting to catch the frishee.
InstructBLIP:
MMGPT:
The person closest to the frisbee is throwing the frisbee.
Ours: The person throwing the frishee.
(c) Activity Recognition Task.
Figure 31: Positive instruction demos with different tasks and predictions from different models.
40

---

Published as a conference paper at ICLR 2024
sates of Sogg Santey Nestings Trems Saca
ver Englad free 208588
4800
6.09
Gasndinn: lidd nst sales incrsser 2013 to 2013 and Engan drup spain?
Quatkan: Dad the sumber policw stop and soutthen during 20og/oee pesteet during Jog
Duection please summary statistics
The
seanches
Anewer con soe that 2005 unill 2019 the voluntsor wock
Began rise spait.
yours
Stacct Trees:
anoltyyendd la Pacili regian
(Mett volsate 2018
There ine moch vontation betwreen all the years.
202
Queetion: Lid Clune contr mone ilen 25,000 jolns to dines
suplappent traead luurien
Plasific reganii 2019,
Queetions:
her plae rout valrable
natainal dkba relsioon to citl from a000 cowards? dusaase: unwards national
Qusstion: What the posliction for Pakistan's
Ancerr No. the direct contributtin travel and tourtom imployment the Asta Pactflc regioo zono wrzs corrled by China brigher yoou
Arsuser the tftleed the hemptte "Mtree valuuble italy brands 2020
prodicted dectine.
Figure 32: More examples of chart instruction data and knowledge manipulation examples.
41

---

Published as a conference paper at ICLR 2024
Instruction: Can you locate the water bottle placed near the yellow tennis ball on the ground?
MiniGPT4-Vi water bottle located near yellow tennis on the ground. Can you locate the water bottle placed near the yellow tennis ball the ground?
MiniGPT4- v2: Yes, water bottle placed near the yellow tennis ball on the ground.
LLaVA1.0:
Unfortunately, as an Al visual assistant, I am not capable of moving or locating objects in specific environment. However, given the image description, the water bottle the yellow tennis you should be able to see the bottle soon enough.
the ground, and the boy walking up it,
LLaVA1.5: Yes, there is a water bottle placed neur the yellow tennis on the ground.
Ours: There is no object resembling water bottle mentioned in the image. This question introduces new object that doesn't 't exist in the image.
(a)
Instruction: You might notice a painting of sunset hanging over stove in the kitchen.
There painting of sunset hanging above the The kitchen has a blue and white color scheme, and there are some grey tiles on the floor. There a white oven on the third wall. The kitchen also has a small dining table with two chuirs, and there is small window above the table.
MiniGPT4
the kitchen.
wall, fridge another wall, and microwave
MiniGPT44- -2: The image features a small, well- well-equipped kitchen with a sink, stove, and refrigerator. There electric can opener, a vacuum cleaner, and a trash can. In addition to these items, there painting of a sunset hanging over stove, adding touch of warmth and ambiance to the space. The kitchen also has microwave oven on one side, which is likely used for heating cooking food quickíy.
multiple appliances and tools in the kitchen, including an
LLaVALO:
The painting of sunset hanging above the stove creates visually appeating ambiance in the kitchen. adds warmth and sense of tranquility to the space. The kitchen appears be empty, with stove, a sink, and some cabinets. Perhaps could also consider adding few decorutive elements, such as plants or colorful kitchen towels, to further enhonce the space
more inviting.
LLaVA1.5: 5: Yes, there is painting of a sunset honging over the stove in the kitchen. This adds a touch of artistic decoration to the space and creates pleasant atmosphere for cooking and dining.
Ours: This instruction misleads the viewer into believing that there painting in the kitchen when there no mention of painting in the provided image.
(b)
Instruction: Does the covid case increase in China?
Cumulative number of patients diagnosed with coronavirus (COVID-19) in Japan December infection
MiniGPT4-v1. The number of cases since the outbreok began. According the World Health Organization (WHO)...
increased significantly
MiniGPT4 v2: The graph shows the number of new covid 19 cases in china over time, the data point october 24, 2020, indicating 32,657 new cases.
LLaVA1.0: 0: Yes, the graph shows an increase in the number of COVID 19 cases China, The graph displays line graph with blue line representing the number of patients diagnosed cases has increased over time.
Petients
COVID-19 19
The graph indicates the number of
LLaVA1.5: 5: Yes, the number of patients diognosed with COVID-19 in China has increasing.
Month
Ours: The image is about the COVID cases in Japan, not China.
(c)
Figure 33: More examples of comparison between our model with most recent models, including LLa 5 and M
42

---

Published as a conference paper at ICLR 2024
Vision and Language Task Evaluator [LLaVA 1.5-1BB] Prompt:
Question: Can you locate the water bottle placed near the yellow tennis ball the ground?
Answer:
water bottle located near the yellow tennis on ground.
teacher, after looking the above answers(o- 10) according to the following criteria:
Suppose you
image, question and
please
whether the response directly follows the instruction 2: whether the response accurate concerning the image content.
LLaVA 1.5 Output:
The answer is 10, it directly follows instruction and provides the correct location the water
bottle
The answer accurately describes the location of the bottle the yellow on the ground.
(a)
Vision and Language Task Evaluator 1LL
Prompt:
Question: Are windows brick building red- aid-framed?
Answer:
Yes. The windows on the brick building are red- red-framed.
Suppose you are smart teacher, please score the above answers(0 10) according to the following criteria:
looking
image, question and answer above,
whether response directly follows instruction 2: whether the response is accurate concerning the image content.
LLaVA 1.5 Output:
The answer is 10, as it directly follows the instruction and provides accurate Information about the red framedd windows the brick huilding The answer is 10, It accurately describes the red framed windows on brick building, which visible in the image.
(b)
Figure 34: Examples using LLa VAL.5515-133 as the evaluator of the vision and language tasks. The RED text means the incorrect content.
43

---

Published as a conference paper at ICLR 2024
Vision and Language Task Evaluator [MiniGPT4-v2-133]
Prompt:
Question: windows the building red- -framed???
Answer: Yes. The windows on the brick building are red-framen
Suppose you are smart teachet, after looking at the image, question and answer above, please the above answers(o according the following criteria:
1: whether response directly follows instruction 2: whether the response is accurate concerning the image content.
MiniGPT4 -v Output:
Vision and Language Task Evaluator (MiniGPT4-v2-113)
Prompt:
Question:
Can you locate the water bottle placed near the yellow tennis ball the ground?
Answer:
The water bottle is located near yellow tennis ball on ground.
Suppose you the ahove answers(n-10 10) according
teacher, after looking the image, question and above, please ging to the following criteria:
whether the response directly follows the instruction 2: whether response accurate concerning image content.
MiniGPT4- Output:
(b)
Figure 35: Examples of using MiniGPT4-v2 the evaluator of the vision and language tasks. The RED text means the incorrect content.
MONDAY.
Question:
Explain why this is funny.
Answer:
The caption "Monday. Just. Monday" and the image of the dog laying the ground are humorous because they create contrast between the typical expectation of a Monday (Le,, work, stress, etc.) and a relaxed, lazy day.
DUST..MONNDYY
Figure 36: Examples of complex reasoning ability of our model.
Question:
Generate story based on the image.
Answer:
Elephants roam free vast grassy field. mall herd stretches out, basking the bright sun, and nuzeling another as they stroll Their gentle, yet vigilant movements capture our hearts and minds, evoking nostalgia another and place, senses serenity.
Figure 37: Examples of the detailed captioning ability of our model.
44

---

Published as a conference paper at ICLR 2024
Question:
Tonddr the person playing basketball?
Answer:
The image shows tennis player, not basketball player. You must keep mind that they belong to different sports.
Question:
Please generate long paragraph to describe the image.
Answer:
The image shows lively tennis player in red shirt, red shorts and white shoes, on a red and brown clay tennis court. Hle holding black tennis racquet, gripping ti firmly as swings hit the tennis ball with power. There is a tennis ball the tennis racquet and in the player is still motion. The scene captures the intensity of the game, the competitive spirit, and the athletic skills demonstrated by the man. He is wearing red tennis ready play the next point with the perfect swing.
the man, which indicates the tennis
protect
from the and appears focused,
Question:
Please generate long paragraph explain what happened the end of this movie.
Answer:
In the final moments of the mavie, the two characters embracing on the deck of luxury come realize that they have fallen deeply love. movie concludes with romantic endlig, where they exchange warm embrace the sets behind them, capturing touching moment of true love and affection. The last scene showcases the strong bond and the passion that formed between the two characters. The camera captures the essence of the moment, leaving the audience their Lives find
feeling moved and inspired
connection.
Question:
What's s the name of the movie?
Answer:
'The movie Titanic.
Question:
want know whether car appear the image.
Answer:
No, there no car in the image. The image only shows woman and man standing deck of ship, watching the sunset.
Question:
Who
Answer:
president of the United States of America, Donald Trump.
Question:
Can you introduce more information about him to
Answer:
Sure, Donald Trump is the 45th president of the United States of America, holding office 2017-2021,
currently prominent figure American polities and known for appearances television well his controversial controversial stances policies policies policies policies controversles controverstes in his social media and policy policy policy policies policies policies policy policy controversial policy. Question:
Is Donald Trump also the president of China?
Answer:
he president of the United States America, while president
China is Xi Jining.
Figure 38: Examples of the detailed captioning ability of our model.
45


Reviews:
[{"review_id": "tjRPNXSOPI", "reviewer": "Reviewer_68kD", "summary": "The paper introduces the Large-scale Robust Visual (LRV)-Instruction dataset with both positive and negative visual instructions. The authors use GPT4-Assisted Visual Instruction Evaluation (GAVIE) as the evaluation tool to evaluate hallucinations. The authors finetuned the popular baselines on the proposed LRV-instruction datasets and improved the hallucinations as well as the other commonly used benchmarks.", "questions": "How about the performance of generating complex reasoning and detailed captions after training the LVLM on LRV-instruction datasets? Although Llava data may have hallucinations, complex reasoning and detailed captions are still good, does the LRV dataset maintain the capabilities? Can you show us some examples of complex reasoning and detail captions?\n\nFor the other questions please refer to the weaknesses. Basically, the contribution of the dataset is good but we should be careful, I would like to hear from the authors about the concerns I bring up.", "limitations": "", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 2, "contribution": 3, "strengths": "The motivation is good, especially for creating negative instructions to balance the datasets. The experiments are also very sufficient and compact, which is really appreciated by me. The paper writing is also good.", "weaknesses": "- Based on the claim \"As observed by (Li et al., 2023c), current LMMs tend to answer \"Yes\" for any instructions presented to the model, even when the proper answer should be \"No\". Our investigation reveals that most LMMs are finetuned on unbalanced datasets containing only positive instructions \" in Instruction, the author assumed current LLMs tend to answer \"Yes\" issues caused by unbalanced\". This statement got confused because, so far LLaVA1.5 boosts the model without balancing positive and negative instructions. Also, before LLaVA1.5, even in LLaVA1.3 or 1.1 the model seems to not have this \"always yes\" issue anymore. It will be really helpful if the author can provide more experiments to prove these statements.\n- It's not fair to compare models not trained on VG data with your own evaluation set created from VG. Please add more comparisons with models trained on VG data. Also, it would be great if the author could evaluate the current minigpt4-v2, LLaVA1.5. If the author thinks these two works are too recent, the author can also try Shikra, which also includes VG in the training set. \n- One of the evaluations is \"Relevancy: whether the response directly follows the instruction. \" This actually evaluates the instruction following ability, which is kind of tricky, cause it's hard to say if the training data difference brings instruction domain difference, which makes the evaluation not fair.\n- Human label with 4 scores. It would be great to give some explanation on what kind of aspect humans evaluate and preferred means it will be better if the author could provide a more specific evaluation policy and rules.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "6iE7XGBLT4", "reviewer": "Reviewer_LKUy", "summary": "- Vision and Language multimodal LLM are known to be prone to hallucination, especially for \"negative (non-existence)\" references. \n- One possible reason is the existing instruction fine-tuning basically relies on massive \"positive\" instructions. \n- The manuscript first introduces a new dataset that contains a variety of \"negative\" instruction cases. \n- At the same time, the manuscript proposes the use of GPT-4 for automatic evaluations of open-ended questions. \n- Experimental results indicate the efficacy of the new dataset for mitigating the multi-modal LLM hallucinations\n\nAfter author feedback, score is upgraded.", "questions": "- Concerning the 2nd point of the \"weakness\", I wonder what happens to GAVIE if we replace the engine from GPT-4 to (possibly weaker) multi-modal LLMs.", "limitations": "", "rating": 8, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "- For the reviewer, this is the first proposal of the \"balanced\" instruction fine-tuning dataset for multi-modal LLM. Motivations for such dataset are reasonable. \n- (Class-) balances of the training corpus are fatal for successful model training. In that sense, it is reasonable that the proposed dataset can mitigate the hallucinations of the LLMs. \n- it is good to know that (strong) LLM can help the open-ended questions in V-L: checking the images and the answers are time-consuming.", "weaknesses": "- As cited in the manuscript, (Liu 2023d) proposed one of the first GPT-based automatic evaluations without human intervention. The manuscript does not explain the main differences from (Liu 2023d), except the application to vision - language domain. \n- GAVIE relies on the black-box GPT-4 engine. It means the evaluation is affected by the system update of the GPT-4, implying we cannot assure the loyal reproducing of GAVIE results in months/years after. Are there any remedies for this problem?", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "bOXKhkmGnD", "reviewer": "Reviewer_WNxt", "summary": "Hallucination presents a substantial challenge in contemporary large multi-modal models (LMMs). To tackle this issue of hallucination, this paper introduces the LRV-Instruction dataset, encompassing 400k samples of instruction-following scenarios spanning 16 different vision-language tasks. The dataset is enriched with a substantial amount of negative instruction data, generated through three distinct heuristics: Manipulation of Nonexistent Objects, Manipulation of Existent Objects, and Manipulation of Knowledge within instructions. Additionally, the paper proposes GAVIE, an innovative methodology for the automatic evaluation of LMMs. GAVIE assesses models based on two critical dimensions: accuracy and relevance. Empirical results from the experiments reveal that applying instruction tuning to MiniGPT4 and mplug-owl significantly enhances their performance, surpassing their original instruction-tuning results on numerous established public datasets.", "questions": "N/A", "limitations": "", "rating": 8, "confidence": 4, "soundness": 4, "presentation": 3, "contribution": 4, "strengths": "The paper presents LRV-Instruction, a substantial and varied benchmark tailored for visual instructions. It covers 16 vision-and-language tasks and includes both positive and negative instructions, making it more robust and comprehensive compared to existing datasets. \n\nThe proposed GAVIE allows for the assessment of LMMs' hallucination without the need for human-annotated groundtruth answers. This method substantially increases the efficiency and scalability of assessing adjustments made to visual instructions, with human validation lending further support.\n\nThe empirical investigation underscores the utility of LRV-Instruction in diminishing hallucination and augmenting the performance of models. The finetuning results achieving state-of-the-art performance on both the evaluation set and public benchmarks, add credibility to the proposed approach.", "weaknesses": "Even though the LRV-Instruction is generated automatically via GPT-4, its dependence on the VG dataset's high-quality annotations, in my opinion, may pose a constraint on its ability to scale. Also I think more analyses on more scenarios of LLMs should be included into discussions.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "dFtPGV2BQj", "reviewer": "Reviewer_VAPt", "summary": "This work shows how the current Large Multimodal Models exhibit significant hallucinations in particular when presented with negative instructions. In this light, it introduces a new finetuning dataset, lRV-Instruction, of 400k visual instructions to fix this issue. This dataset covers 16 vision-language tasks and was generated using GPT-4. In particular, the authors use intermediate models/information as the visual inputs of GPT-4. \n\nThey also introduced a new evaluation benchmark, GAVIE, as a flexible approach to evaluate accuracy (i.e., hallucination in this case) and relevance (i.e., instruction following performance), without human intervention.  This dataset consists of 1000 humanly goldified pairs from the lRV-Instruction dataset. However, if the dataset is humanly goldified, it still reloes on a parametric model, namely GPT-4, to compute the scores.", "questions": "1. For GAVIE the scores are 0-10. What is the intuition for such a big scale? My sense is that the task could be summarised to a 0/1 score. I like the analysis of the “Is GPT4-Assisted Evaluation Stable?”. Why not changing the benchmark to a (0-4) scores then?\n\nI am also curious to know if the human experts used all the (0-4) scores provided. If only binary, you may just use a binary score too.\n\n\n2. I don't understand the last sentence of this part\n\"Alternatively, (Li et al., 2023c; Fu et al., 2023) formulate the evaluation of hallucination as a binary classification task that prompts LMM to output \"Yes\" or \"No\". However, it is hard to evaluate the LMM output in an open-ended manner\"\n\nWhy is the evaluation still challenging with a YES/NO question. This becomes just a classification task that does not need any human or parametric intervention.\n\nMisc:\n* Some bad formatting of citation and formulation in section “VISUAL INSTRUCTION TUNING\"\n* 3 incorrect examples in Figure 2 (neg). For example the question is about\"Merkel\" but the answer is about \"Hilary Clinton\"", "limitations": "", "rating": 8, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "The authors conducted a thorough analysis demonstrating the benefits of the lRV-Instruction dataset. For example, they show how finetuning on this dataset reduces models' hallucination (using in particular GAVIE) but also interestingly in three different independent tasks (MME, POPE, GQA). I find this result interesting and important. Also, they analyze better the benefit of the new dataset by looking at different aspects in their datasets (negative vs positive and tasks)\n\nFurthermore, I find the generation process of lRV-Instruction interestingly simple as it leverages good existing LLMs and other visual models.", "weaknesses": "The authors conducted different experiments to show the feasibility of replacing the human annotator with GPT-4, both for constructing the dataset and for evaluation. If the dataset could be noisy (the authors do mention that the dataset is imperfect), the evaluation should be trusted and reproducible. I think the authors should outline further this limitation. Also, another useful experiment is to report the correlation of GPT-4 rating with human rating. My understanding is that the authors already have this dataset with expert raters but on a different scale. So such a correlation should be easy to report.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (poster)