Mitigating Object Hallucination in MLLMs via Data-augmented Phrase-level Alignment
Authors: Pritam Sarkar
Sayna Ebrahimi
Ali Etemad
Ahmad Beirami
Sercan O Arik
Tomas Pfister
Venue: ICLR 2025 Poster (2025)
Keywords: Multimodal LLMs
Object Hallucination
Vision-language Models
Primary area: applications to computer vision, audio, language, and other modalities
TL;DR: We introduce phrase-level alignment method that can be applied to off-the-shelf MLLMs for mitigating hallucinations, while preserving their general vision-language capabilities.

Abstract: Despite their significant advancements, Multimodal Large Language Models
(MLLMs) often generate factually inaccurate information, referred to as hallucination.
In this work, we address object hallucinations in MLLMs, where information
is generated about an object not present in the input image. We introduce Data-augmented
Phrase-level Alignment (DPA), a novel loss which can be applied to
instruction-tuned off-the-shelf MLLMs to mitigate hallucinations, while preserving
their general vision-language capabilities. To fine-tune MLLMs with DPA, we first
generate a set of 'hallucinated' and 'correct' response pairs through generative data
augmentation by selectively altering the ground-truth information of the correct
responses at a phrase level. The DPA loss is then used to train MLLMs to reduce
the likelihood of hallucinated phrases compared to the correct ones. Our thorough
evaluation on various benchmarks confirms the effectiveness of DPA in mitigating
hallucination while retaining the out-of-the-box performance of the MLLMs on
general tasks. For instance, MLLMs finetuned with DPA, which we refer to as Hallucination
Attenuated Language and Vision Assistant (HALVA), improve F1 by up
to 13.4% on hallucination visual question-answering and reduce the hallucination
rate by up to 4.2% on image description tasks.

Published as a conference paper at ICLR 2025
MITIGATING OBJECT HALLUCINATION IN MLLMS VIA DATA-AUGMENTER PHRASE-LEVEL ALIGNMENT
Pritam Sarkar Sayna Ebrahimi*, Sercan Ö. Arik Tomas Pfister*
Ali Etemad
Ahmad Beirami",
Queen'ss University. Vector Institute, Google DeepMind, {pritam. sarkar, ali. sttemad ca (saynae, bairami, soarik, tpfisterrtrre wegfiisterrroogleee...eee @eooolllo..oooo. com attps://githie wttptttc.oopiiiieeeeeeeeeeeeeeee pritamqu/HALVa
*Google Cloud Al Research
com/
ABSTRACT
Despite their significant advancements, Multimodal Large Language Models (MLLMs) often generate factually inaccurate information, referred to as hallucina- tion. In this work, address object hallucinations in MILMs, where information is generated about an object not present in the imput image. We introduce Data- augmented Phrase-level Alignment (DPA), a novel loss which can be applied instruction-tuned off-the-shed MILMS to mitigate hallucinations, while preserving their general vision- language capabilities. To fine-tune MILMs with DPA, we first generate set of "hallucinated" and 'correct' response pairs through generative data augmentation by selectively altering the ground-trutt information of the correct responses at phrase level. The DPA loss is then used to train MLLMs to reduce the likelihood of hallucinated phrases compared to the correct ones. In contrast, existing alignment techniques act at the sequence level and often lead sharp trade off between mitigating hallucinations and preserving model capabilities. Our thorough evaluation various benchmarks confirms the effectiveness of DPA in mitigating hallucination while retaining the out-of-the-box performance of the MLLMs on general tasks. For instance, MLLMs finetuned with DPA, which we refer to as Hallucination Attenuated Language and Vision Assistant (HALVA), improve F1 by up 13.4% on hallucination visual question-anse answering and reduce the hallucination rate by up to 4.2% on image description tasks.
INTRODUCTION
Recent advancements in Large Language Models (LLMs) (Chowdhery et al., 2023; Anil et al., 2023; Raffel et al., 2020; Touvron et al., 2023a;b; Team et al., 2023; Brown et al., 2020) have laid the foun- dation for the development of highly capable mul- timodal LLMS (MLLMs) (Team et al., 2023; Liu al., 2024; 2023c; Dai et al., 2023; Li al., 2023c; Achiam et al., 2023). MLLMs can process addi- tional modalities such image or video, while re- taining language understanding and generation capa- bilities. Despite their impressive performance across variety of tasks, the issue of object hallucination in MLLMS presents a significant challenge to their widespread and reliable use (Wang al., 2023c; Hu et al., 2023; Rohrbach et al., 2018; Bai et al., 2024).
8
there Please sww.errye....
Image
image,
of Llentine tock
knll. spoon a Billaea
the inape, Bher ono sesero made lagans
Figure 1: Examples of object hallucinations.
Object hallucination refers to generated language that includes descriptions of objects their at- tributes that are not present in, or cannot be verified by, the given input. We illustrate few examples
"This work was partially done when PS was an intern at Google Cloud AI Research. This work was partially done when AE was visiting faculty researcher Google Research.

---

Published as a conference paper at ICLR 2025
23.0
25
00:3
-1.0
OOject
H96 9e0
600
100
60
Divggerce
ShChhagence
(A)
(B)
Figure 2: (A): A high-level overview comparing the performance of HALVA (the finetuned model with DPA) with existing finetuning methods in mitigating object hallucination, and their ability on general vision-language tasks. (B): Unlike HALVA, the existing finetuning approaches (e.g.. HA-DPO and EOS) substantially diverge from their base model (LLaVA-vI.57p).
of object hallucinations in Figure 1, where on the left LLaVA-v pick' in an image of utensils (knife, spoon, fork) these items frequently appear together, while it missed identifying 'Legos' due to their rare occurrence with utensils. On the right, LLaVA-v1.5 incorrectly confirms the presence of 'tie" for the image of a 'wedding cake'. This is likely due to two reasons: first, the frequent co-occurrence of wedding attire such as 'ties' and 'wedding cakes', and second, MLLMs tend answer 'Yes' for most instructions presented due to positive instruction bias in the training data (Liu et al., 2023b; Bai et al., 2024).
inaccurately deseribes a 'tooth-
Prior research has attempted to address object hallucination in one of three key stages: inference (Deng et al., 2024a; Yin et al., 2023; Leng et al., 2023; Lee et al., 2023; Zhou et al., 2023; Biten et al., 2022). pretraining (Sun et al., 2023; Jiang et al., 2023; Liu al., 2023b), and finetuning (Zhao et al., 2023b; Yue et al., 2024). Inference-based methods aim to mitigate hallucinations during text generation, either through specialized decoding (Leng et al., 2023; Deng et al., 2024a; Zhu et al., 2024) through iterative corrections (Lee et al., 2023; Wu et al., 2024; Zhou et al., 2023), among others. One of the key limitations of such approaches is that they can substantially increase inference time and cost, and often require modifications to the serving infrastructure (Lee et al., 2023; Bai et al., 2024). Pretraining techniques, such as negative instruction tuning or contrastive learning, have also been used to mitigate object hallucination (Liu et al., 2023b; Jiang et al., 2023). The main limitation of such approaches is that they require massive training data (>500K samples) and can not be applied to off-the-shelf MLLMs. Finally, finetuning- -aauun approaches attempt to mitigate object hallucination through preference optimization (Zhao et al., 2023b) or human feedback (Sun et al., 2023: Yu et al., 2023a), among others (Ben-Kish et al., 2023; Yue et al., 2024).
We note that hallucinations typically occur locally and can be pinpointed to specific words or phrases, such as 'tooth-pick" in Figure This is in contrast to other alignment problems such as helpfulness, where it is difficult to identify if a particular word contributes to the overall helpfulness (or lack thereof) in a response. Existing alignment methods (e.g., DPO (Rafailov al., 2023)) do not leverage this and instead attempt to mitigate hallucinations using a sequence-level loss. Such sequence level loss provides a coarse and noisy signal, making it less effective and causing the model to degenerate from its initial state, leading deterioration in general vision- languaee capabilities (see Figure 2).
Our goal is to achieve a fine-grained mechanism to mitigate hallucinations that allows to tackle hallucinations while not hurting the general capabilities of the model without adding to inference time or requiring substantial re-training. To this end, we first use generative data augmentation (Qin et al., 2022; Zheng et al., 2024) to construct training set of "hallucinated" and 'correct' response pairs, by selectively altering the ground-truth phrases in the correct responses, while keeping the overall structure intact. Next, to reduce the likelihood of hallucinations, we introduce training objective called Data-augmented Phrase-level Aligument (DPA), to finetune MLLMS using the constructed correct and hallucinated response pairs. Our proposed DPA loss consists of two terms: the first term computes the relative log-pro probability of the hallucinated tokens compared to the correct ones, and the second term calculates the token-wise KI. divergence using a frozen reference model. Accordingly, the MILM is trained to minimize the likelihood of hallucinated tokens while keeping the divergence minimal. As a result, while DPA is effective in mitigating hallucination closely retains the general capabilities of the base MLLM. We refer to MLLMs trained with our proposed DPA loss as Hallucination Attenuated Language and Vision Assistant (HALVA). We perform rigorous evaluations on hallucination benchmarks, showing the benefits of our method in mitigating hallucination in both generative and discriminative vision- languae tasks. While the primary goal of this work is to mitigate object hallucinations, we take further step also evaluate on
2

---

Published as a conference paper at ICLR 2025
general vision-language hallucination benchmarks. The results show that DPA also provides benefits toward other forms of vision- languuge hallucinations that may arise due to visual illusions among others. Finally, to ensure that the proposed DPA does not adversely affect the general capabilities of MLLMs,, we evaluate HALVA on popular vision-language benchmarks. Our extensive studies confirm the effectiveness of the proposed method in mitigating object hallucinations while retaining or improving the performance in general vision-language tasks.
In summary, our main contribution is DPA, a novel method finetune MLLMs for mitigating object hallucination in vision-language tasks. Unlike existing finetuning- hallucination mitigation methods, DPA works at a phrase-level and penalizes the tokens where hallucination occurs and not across all the tokens. Such localized and fine-grained feedback reduces object hallucination while retaining the general performance of ML..Ms. We open-source the code, checkpoints, and the generated hallucinated and correct response pairs used in training, at GitHub.
2 METHOD: DATA-AUGMENTE PHRASE-LEVEL ALIGNMENT (DPA)
Consider an MLLM, denoted as Ta trained in an auto-regressive manner to predict an output for a given vision-language instruction == fæe. Fig) where 2, is an image and Zy is the corresponding instruction. During inference, the generated sequence s of length T, is represented as (t1, .2....................... ................................ tz, }. where each I represents a language token. The sequence is said to contain hallucinations the occurrence of l; is not grounded in, or cannot be verified from, the input x. If the data used to train zy comprises frequent appearance of certain concepts (e.g., S., objects, object-atrribute pairs), the MILM may generate responses based on learned spurious correlations while ignoring the given inputs (Zhou ct al., 2023; Bai et al., 2024; Rohrbach et al., 2018; Li et al., 2023d). Here, we present our strategy to mitigate object hallucinations that may occur due to such co-occurrences.
Generative data augmentation. We discuss our strategy to construct "hallucinated" and 'correct" response pairs through generative data augmentation. Let y and y² be a vision-
correct and hallucinated response, respectively, language instruction {I,, J. We design generative data-augmentation setup to generate y by selectively alter- ing the ground-truth concepts in yf, thus introducing hallu- cinated concepts that are not present in the vision input 2... Note that there is no overlap between the correct and the induced hallucinated concepts. Formally, we generate y²
Altributes
Objects
Actions
Locatione
by replacing the ground-truth set o containing the true con- cepts in yf, with the hallucinated set o', where o' E O and & x.. Here, O is a set containing hallucinated concepts. We define O- {(o,, c;) | og E U and c; C U). where or is a concept (e.g.. object, attribute, or action), o is a subset of concepts that co-occur with o;, and U represents the universal set of all possible concepts of objects and object-related attributes. See example in Figure 3.
Instruction: Provide one-sentence caption for the provided image.
Correct A young mann In white shirt and blue jeans skateboarding skate park.
Hallucinated: A young woman In a black dress and red sneakers rolierblading roller rink.
An example of correct and hallucinated response pairs con- structed through our generative data- augmentation. The hallucinated re- sponses are generated by selectively al- tering the true concepts in the correct re- sponse. For instance, we alter 'objects': shirt dress, & jeans sneakers; 'at- tributes': white > black, & blue red; 'actions': skateboarding rollerblad- ing: and other object-related information such as 'location': skate park roller rink. Best viewed in color.
Figure 3:
We approximate O for hallucinated concepts that are both closed set (Oxz) and open-set (0o.). We prepare Oce based on the co-occurring concepts in a large object-centric dataset. For Doc we sample hallucinated concepts by directly prompting an LLM. In addition to generating de- scriptive responses, we also use a small set of Yes-or-No questions based on an existing visual question-answering dataset, for which we generate n by simply inverting y² This yiclds the correct and hallucinated response pairs {vf 3²) which we subsequently use in DPA. Additional details of generative data augmentation, including the tem- plates for generating correct and hallucinated responses, as well end-to-end examples of the entire augmentation process, presented in Appendix D.3.
Proposed phrase-level loss. Given an off shel trained MLLM susceptible to hallucinations, objective is minimize the likelihood of generating hallucinated tokens using the correct and
3

---

Published as a conference paper at ICLR 2025
MUTDER pattanyyy +
Hogp bngtrribbliityy
Muttimodal LLM
Multimodal LLM (m²)
(Trer)
yourg whiti and calall park.
Myperg mat
isage
young W0
Figure 4: Overview of our method: Given a vision-language instruction and its correct and hallucinated response pair, the alignment objective (La) reduces the log-likelihood of hallucinated tokens compared to the correct ones. Also, a token-wiss KL divergence regularizer (Ca) is employed using a reference model (TTef), to restrict the divergence of the MLLM (TT) during DPA training.
hallucinated response pairs {y/ ./"."") obtained through generative data-augmentation. To this end, we define an alignment objective based on the relative probabilities of correct and hallucinated phrases.
Let's take an example with a correct response yf as 'A young man in a white shirt' and its correspond- ing hallucinated response y 'A young woman in a black dress'. Let y/" denote the i-th hallucinated phrase in vd and of be the corresponding correct phrase in y.. In this example, the hallucinated phrases are 'woman' and 'black dress', while their corresponding correct phrases are 'man' and 'white shirt'. sd can be expressed as sequence of tokens Ty { 12 ..,,...., Mr, J. according to which va¹ = Ts e, where se and et are the start and end indices of vf with I sf S cf S (Tal- Accordingly, we can compute the probability of hallucinated phrase v as IT mo(th) Iz, t
Similarly, the probability of the correct phrase yf can be expressed as: Iī To To(4511; Ezy), where of and ef are the start and end indices of yf. Note that for every v/ E y there exists a corresponding of E y². To reduce the relative likelihood of hallucinated phrases compared to the correct ones, we define the alignment loss La as:
N
f To (1(G,,1,25))
L.
log
(I)
To (19,,, +IT To To
where N represents the total number of hallucinated phrases in y². Note that our loss is designed to penalize the model Ty only for the hallucinated tokens rather than for all tokens in the sequence. This localized and fine-grained feedback is one of the key concepts that sets our method apart from existing preference optimization techniques, e.g., (Christiano et al., 2017; Rafailov et al., 2023).
Note that simply optimizing अ. to minimize L, may cause Te to substantially diverge from its initial state, which may hurt its ability in general vision-language tasks. To mitigate this effect, we train To with a KL-divergence constraint using a frozen reference model Tet. For a given reference sample {x", v² 1. y" can be expressed as a sequence of tokens T, = (t), t5. ... ffr,) }. We formulate the loken-wise KL-divergence regularization term La as:
[Tyl TerrCG (ill=",Ce)) .Ces) (Log [1oe
log (Os(TT(GGE (To((G)e", :2a)))
La
(2)
Our formulation of L. serves token-level regularizer to restrict the model from diverging too far from its initial state, thus losing its general initial abilities. Note that {x", """}"" represent any set of vasion-language instructions and their correct responses, which may may not include {af, vf Moreover, note that गet and Ta are initialized from the same checkpoint, therefore La estimates the divergence of Te from its initial state during training. It should be noted that we adopt forward KL-divergence approach in calculating La which is different from the reverse KL-divergence used in RLHF (Christiano et al., 2017). This choice is essential in our case, as we do not conduct rollouts of To during training and rely solely on responses from Tees ensuring that To focuses on high-probabillty tokens of the reference distribution. Finally, we train Ty to minimize the final DPA objective:
Ldga La + a Lad
(3)
4

---

Published as a conference paper at ICLR 2025
where a is coefficient to control the divergence of Te during training. The value of is set based on ablation studies presented in Section 4.4. We present the pseudo code in Appendix A.
EXPERIMENT SETUP
Training data. We prepare vision-language instructions based on Visual Genome (VG) (Krishna et al., 2017), which is an object-centric image dataset consisting of a total of 108K images and their annotations. Accordingly, we prepare the correct responses with both descriptive (e.g., Describe the image in detail.) and non-descriptive (e.g., <Question>> Please answer in one word, yes no) instructions. Descriptive instructions include one-sentence captions, short descriptions, and detailed descriptions of images. Moreover, the non-descriptive question- answers directly taken from (Zhao et al., 2023b). We prepare the correct responses using Gemini Vision Pro (Team et al., 2023) and based on the original images and ground-truth annotations. Subsequently, we perform generative data augmentation obtain hallucinated responses, as described in Section 2. Our final training set consists of a total of 21.5K vision- lanion-languag instructions and their corresponding correct and hallucinated responses.
Implementation details. We use LLaVA-vI. 1.5 (Liu et al., 2023c) and VILA-v1.5 (Lin et al., 2024) as our base models considering their superior performance in general vision- Lasion-angguage tasks and the availability of their code and models. LLaVA- et al., 2023b) as the language encoder and CLIP ViT-L114 (Radford et al., 2021) as the vision encoder. VILA-v1.5 uses Vicuna-v1. 1.5 (Chiang et al., 2023; Touvron et al., 2023b) as the language encoder and SigLip-L-400M (Zhai et al., 2023) as the vision encoder. Note that while LLaVA-v1.5 uses images of resolution 336 pixels. VILA-v1.5 1.5 is trained with images of resolution 384 pixels. During training. we freeze the vision encoder and projection layers, and only train the LLM using LoRA (Hu et al., 2021). We refer to the resulting DPA trained checkpoints HALVA, i.e., HALVAr based on LLLVA-v1.55am HALVAI based on LLaVA-vI 15 and HALVA based on VILAAvV..5 All experiments are conducted on 4 A A100-80GB GPUs We utilize effective batch size of 64 and train for epoch or 342 steps. The training time ranges from 1.5 to 3 hours for 7B and 13B variants. The additional implementation details are presented in Appendix D.
uses Vicuna-v1.5 (Chiang et al., 2023; Touvron
Evaluation setup. First, we evaluate HALVA on four object hallucination benchmarks encompassing both generative and discriminative tasks, including CHAIR (Rohrbach al., 2018), MME-Hall (Fu et all,, 2023). AMBER (Wang et al., 2023b), and MM (Sun et al., 2023). Additionally, we perform a curiosity driven experiment to critically test the impact of our proposed DPA beyond object hallucination, using HallusionBench (Liu et al., 2023a). Furthermore, to ensure that DPA does not adversely affect the general language generation capabilities of MILMMs we evaluate HALVA on five popular vision-language benchmarks: -QA-v2 (Goyal et al., 2017), MM-Vet (Yu et al., 2023b), TextVQA (Singh et al., 2019), MME (Fu et al., 2023) and LLa VA-Bnnch (Liu et al., 2024). All evaluations are conducted thrice, and report average scores. In the case of GPT-4-based evaluation, the performance slightly varies due to the randomness, where we also report the standard deviations.
4 RESULTS
Earlier in Figure 2, we present a high-level overview of HALVA vs. existing finetuning approaches (e.g., HA-DPO and BOS) in mitigating object hallucinations and their effect on the general vision- language capabilities. Note that both HA-DPO and EOS are based on the same LLaVA-v1155 as HALVA, ensuring a fair comparison. We consider LLaVA-v1..5pp as the lower bound and GPT-4V as strong reference point given its performance on the standard benchmarks.
Image description task. In Figure 2 (A) Left, we compare MILMS on image description tasks in terms of both hallucination rate (AMBER CHAIR) and their detailedness, captured through the number of ground-truth objects covered (AMBER Cover). Our goal is to mitigate hallucinations while retaining improving the richness of image descriptions compared to the base model. As shown, HALVA captures more ground -trut objects while hallucinating less than HA-DPO. Moreover, while EOS achieves lower hallucination rate, it degrades the detailedness of image descriptions, performing worse than the base model. This is an undesired artifact in MLLMs, particularly for tasks that require detailedness such medical imaging analysis (Wang et al., 2023e; Hu et al., 2023).
5

---

Published as a conference paper at ICLR 2025
Table I: Results on CHAIR. = and I indicate that the reported values are from (Chen al., 2023a) and (Yue et al., 2024). 'Results are computed by us, using their of- ficial checkpoints. C and C, refer to CHAIR at instance and sentence levels.
Table 2: Results on MME-Hall. i 'indicat- ing reported values from (Bai et al., 2024).
"Results are computed by us, using official
checkpoints. Red: worse than base model.
Method
MME-Haall (1)
Method
Ci(.)
(,(
Len.
Cheetorza (Li et 2023b)
473.4
LRV-Instructioona (Liu et al. 20236)
528.4
mPLUGG-Ow'7B (Ye et al., 2023a)
30.2
76.8
98.5
Otterna (Li et. 2023a)
483.3
MullMModal-PPTmm (Gong et al., 2023)
18.2
36.2
45.7
mPLUG-OWl22s aYl
578.3
MiniGPT-v255 (Chen et al., 2023a)
8.7
25.3
56.5
Lyuxes' (Zeng et. 2023)
606.7
InstructBlipra (Dai et al., 2023)
17.5
62,9
102.9
Quen-VL-Chates (Bai et al., 2023)
606.6
LLaMA-Adapter V2n=* (Gao 2023)
493.3
LLaVA-v1.5 nu (Liu et al., 2023c)
15.4
50.0
100.6
LLaVA-w1 (Liu et al., 2023c)
648.3
EOST (Yue et al., 2024)
12.3
40.2
79.7
HA-DPO 1 (Zhao al., 2023b)
618.3
OPERAa (Huang al., 2023)
12.8
44.6
BOS (Yur et al., 2024)
606.7
DoLAra (Chuang al., 2023)
13.8
47.8
VCDaa (Leng et al., 2023)
604.7
IIA-DPO' 7u (Zhao et al., 2023b)
11.0
38.2
91.0
Wondpesker (Yin et al., 2023)
366.7
MEMVR (Zou et al., 2024)
648.3
MEMVR3 (Zou et al., 2024)
13.0
46.6
99.6
ARAm (Qu et al,, 2024)
648.3
AGLAAB (An et al., 2024)
14.1
43.0
98.8
AGLAA
al., 2024)
640.0
HALVA (Ours)
11.7
41.4,8.a
92.2
HALVA (Ours)
665.0-16.
MiniGPT-4"135 (Zhu al., 2023)
9.2
31.5
116.2
BLIVAne * (Hu 2024)
$80.0
InstructBlipur (Dai et al.. 2023)
16.0
51.2
95.6
MMICL.m (Zhao al., 2023a)
568.4
LLaVA' 13B (Liu al., 2024)
18.8
62.7
90.7
InstruciBLIPP.g (Dui et. 2023)
548.3
SPHINX
al., 2023)
668.3
LLaVA-v1.51 1s8 (Liu et al., 2023c)
13.0
47.2
100.9
Muflinus (Lou et 2022)
$90.0
EOS (Yue el al., 2024)
11.4
36.8
85.1
RLHF-V (Yu 2023a)
585.0
HALVA (Ours)
12.8.0.2
45.4]1.8
98.0
LLaVA-v1 (L.u et al., 2023c)
643.3
HALVA (Ours)
675.0:s1.7
VILA-vI. (Lin et al., 2024)
9.2
33.0
183.4
VIL.A-vI.Ssann (1.in et al., 2024)
688.3
HALVA (Ours)
8.4
30.0
182.6
HALVA (Ours)
691.7 15.4
Question answering task. In Figure 2 (A) Right, we compare the performance of MLLMs on visual question-answering tasks using both object hallucination (AMBER) and general vision-language (TextVQA) benchmarks. As shown, both HA-DPO and EOS underperform HALVA in mitigating object hallucination and even deteriorate general vision- Laiionnlanguage abilities compared to the base model. These results show the shortcomings of existing approaches, which we address in this work.
To further understand the limitations of existing methods in greater detail, we measure divergence from the base model in Figure 2 (B). Here observe that unlike HALVA, both HA-DPO and EOS substantially diverge from the base model, resulting in poor performance in general tasks.
4.1 EVALUATION ON OBJECT HALLUCINATION
CHAIR. MILMs can be prone to hallucinations when generating detailed image descriptions (Bai et al., 2024; Rohrbach et al., 2018; Wang al., 2023b). To assess the impact of DPA in such scenarios, we evaluate HALVA on CHAIR, which stands for Caption Hallucination Assessment with Image Relevance (Rohrbach ct al., 2018). This metric calculates the number of objects that appear in the image caption but are not present in the image. Specifically, CHAIR measures hallucination at two levels: instance-level (C,) and sentence-level (C.). During this task, HALVA is prompted with 'Describe the image in detail', allowing for the generation of detailed image deserip- tions. The results in Table demonstrate that HALVA substantially reduces hallucination in image descriptions compared the base variants. For instance, compared L.aVA-vV.55m HALVA reduces C, from 50.0 to 41.4, similarly, compared to VILA-vI 1333/344 HEALVA13B33 reduces C, from 33.0 30.0. Furthermore, HALVAAB outperforms or matches the performance of other hal- lucination mitigation methods, such as OPERA (Huang et al., 2023). EOS (Yue et al., 2024), and HA-DPO (Zhao et al., 2023b). should be noted that our proposed DPA does not negatively impact the language generation ability or expressiveness of MILMs, unlike EOS (Yue et al., 2024), which substantially reduces the average generation length from 100 to 85 and 79 for the 13B and 7B variants, respectively. As discussed earlier in Section 4, such a degree of reduction can lead to missing key details in image descriptions and are undesirable for MLLMs. In contrast, HALVA maintains the same generation length the base model, e.g., 98 100.9 or 182.6 183.4, while effectively reducing hallucination. However, a limitation of CHAIR (Rohrbach et al., 2018) is that it does not consider other key aspects of image descriptions, such coverage of objects and detailedness of

---

Published as a conference paper at ICLR 2025
descriptions, when evaluating ballucination. Therefore, we also evaluate AMBER (Wang et al., 2023b), a more recent object hallucination benchmark, which we discuss later.
MME-Hall We evaluate HALVA discriminative tasks using MME (Fu et al., 2023). Specifically, we utilize the hallucination subset of MME, which consists of four object-related subtasks: existence, count, position, and color, referred to MME-Hall. The full score of each category is 200, making the maximum total score 800. The results presented in Table 2 demonstrate that HALVA substantially improves performance compared to the base model. For instance, HALVA 3a achieves a score of 675.0, resulting in a performance gain of 31.7 points with respect to the base model LLaVA-v1.5 Moreover, as presented in Table 2, existing methods including finetuning (e.g. HA-DPO, EOS) and inference-baseer (e.g.. VCD, Woodpecker) approaches are ineffective in mitigating hallucinations across such broad categories and worsen the performance compared to their base model. The detailed results of MME-Hall are presented in Appendix C.
AMBER. To evaluate performance on both generative and discriminative tasks, we use AMBER (Wang et al., 2023b), which measures hallucination using sev- eral metrics. For generative tasks, AM- BER assesses the frequency of halluci- nated objects in image descriptions, sim- ilar to (Rohrbach et al., 2018). Moreover, AMBER evaluates hallucination in three additional aspects of generative abilities: the number of ground-truth objects cov- ered in the description, the hallucination rate, and the similarity of hallucinations in MI.LMss to those observed in human cognition. Discriminative tasks are cat- egorized into three broad groups: exis- tence, attribute, and relation, each as- sessed using FI scores. For additional details these evaluation metrics, refer the reader to (Wang et al., 2023b).
Table 3: Results on AMBER. Cover:: coverage of ground-
truth objects; Hall.: Hallucination Rate: indicates that the reported values are from (Wang et al., 2023b). RResults are computed by us, using their checkpoint. Red: worse than base model.
Method
Generative
Discriminaative
CHAIR
Cover.
Hall.
(FIT)
Ow (Ye et al., 2023a)
21.6
50.1
76.1
18.9
LLaVA' a (Liu et al,, 2024)
11.5
51.0
48.8
32.7
MinGGPT-1" 1- (Zhu et al., 2023)
13.6
63.0
65.3
61.7
mPLUG-OWI2! in et al., 20236)
10.6
52.0
39.9
78.5
InstricsRI Jp (Dal et at., 2023)
8.8
52.2
38.2
81.7
LLaVA-vL.5t
36.4
74,7
HA-DPO's nu iZhoo al., 20236)
6.7
49.8
30.9
78.1
RDS' is (Yie al. 2024)
22.7
75.6
Wondpecker m (Yin et al., 2023)
6.9
48.9
30.4
67.0
HALVA (Ours)
6.6
53.0/2.
32.2,6.
83.4ra.
RLHF- Vinuu Yu et al. (2023a)
6.8
46.1
27.4
87.1
LLaVA.v1.5pm (Lia et 2023c)
51.9
30.5
73.1
HALVA (Ours)
64,00
52.6,a
30.4
86.5
VILA-vI 5 (1.in et al., 2024)
9.9
63.3
56.1
82.2
HALVA (Ours)
9.
63.9pp
51,2
S7.
GPT-4V (Achiam al., 2023)
4.6
67.1
30.7
87.4
The results presented in Table 3 demonstrate that HALVA outperforms the base model by large margin, in both generative and discriminative tasks. For instance, HALVA7B reduces hallucination in caption generation from 7.8 to 6.6, while increasing the coverage of ground-truth objects in the descriptions from 51% to 53%. This confirms that our method reduces hallucination without compromising the descriptive power of MLLMs. On the other hand, while HA-DPO and EOS report slightly lower hallucination rates, the number of ground-trute -ruth objects covered is reduced to 49.8% and 49.1%, respectively. This indicates a degradation in the overall performance of these MLLMs on general tasks. Similar shortcomings are also noticed when using inference-based correction methods such as Woodpecker (Yin et al., 2023), where the object coverage is reduced by 2. 1% compared to the base model. Woodpecker also performs poorly on discriminative tasks as it fails to capture key concepts from short responses of LLa VA-v which it aims to correct. Morcover, our proposed DPA substantially enhances performance on discriminative tasks, for both 7B and 13B variants. For instance, HALVATB improves the F1-score on the attribute category from 64.6% to 80.0%. Additionally, HALVA138 improves the F1 score on relation based tasks from 45.0% to 73.5%. Overall, HALVA outperforms both HA-DPO and EOS on discriminative tasks by a large margin, achieving a 5.3% and 7.8% higher Fl score respectively. Furthermore, HALVA 13R and HALVA |38//84 perform better or on par with GPT-4V on discriminative tasks, i.e.. Fi-score of 86.5 by HALVA 87.9 by HALVA13B8844 and 87.4 by GPT-4V. The detailed results are in Appendix C.
MMHHll-Benhh We also conduct LLM- hallucination evaluation to rigorously test for potential hallucinations in generated responses that might not be captured when validated against a limited ground-truit truth information, as done in (Rohrbach et al., 2018), 8).. We utilize AMMHal-Bencl (Sun al., 2023), which evaluates hallucination across 12 object-topics, including object attributes. presence of adversarial objects, and spatial relations, among others. Following (Sun et al., 2023), we use GPT-4 (Achiam et al., 2023) as the judge to rate the responses on a scale of 0 to 6, with respect to standard human-gon -ennrrtee answers and other ground-trui information of the images. The results
7

---

Published as a conference paper at ICLR 2025
Table 4: Results on MMHallBennhh. f, and indicate that the reported values from (Sun et al., 2023), (Jiang et al., 2023), and (Yu et al., 2024). Results are computed by us, using their official checkpoint. Red: worse than base model.
Table 5: Results on HallusionBench. t 'indicates that the reported values are from (Liu et al., 2023a).
"Results are computed by us, using their official checkpoint.
Yes/No Bias (--0)
Overall
Method
Score (?)
Overall
Rate (U)
Hall.
Method
Acc. (f)
mPLUG.Owl-vl :.u (Ye et al., 2023a)
0.32
43.93
Kosmos-2* (Pong al., 2023)
1.69
0.68
MiniGPT5' To (Zheng et al., 2023)
0.25
40.30
DEFFCC (Laurengon et al., 2024)
1.89
0.64
MiniGPT4's 16 (2hu et 2023)
0.19
35.78
m (Dai 1n
al, 2023)
2.10
0.58
InstrucBLIP' m (Dai et nl., 2023)
-0.13
45.26
LLaVA' hn (Liu al., 2024)
1.55
0.76
BLIP2 (Li ot ol., 2023c)
0.18
40.48
VCD x (1 Long et ol., 20233
2.12
0.54
mPL.UG__Ol--v2 z (Ye et al., 2023b)
0.25
47.30
OPERA (Huang al., 2023)
2.33
0.50
LRV-Instrrctioon 75 (Lin ct al., 2023b)
0.26
42.78
LUREs (Zhou et al., 2023)
T.L m (Lin al., 2023c)
0.31
47 47.09:t0.14
LLaVA-RLHF [San et 2023)
LLaVA-SFT-u (Sun al., 2023)
2.05
1.76
0.68
0.67
LLaVA-RLIIF 78 Sun
(2023)
42.96
LLaVA-v1.5pp (Lio et 2023c)
2,114905
0.54
HIA-DPO'sa >u (Zhao al., 20236)
0.26
48.36
HACI.zn (Jiang al., 2023)
EOS' is (Yoe et 202,)
0,29
48.72
HA-DPO",, (Zhao et ol., 2023b)
1.97
0.60
HALVAp (Ours)
0.1 0.17,0.14
48 (8.9571..0 19
EOS 1 (Yue ct al., 2024)
2,03
0.59
Qwen VL lax
2023)
39.15
HATXA (Ours)
2.25
0.547700
00
Open Flamingol (Awadalla al. 2023)
0.33
38.44
LL smn (Liu et 2024)
1.11
0.84
BLIP2-T5' us (Li et al., 2023e)
0.08
48.09
IostnweBLIPP ysa (Dai ot al., 2023)
2.14
0.58
RLHI-V 1881445 (Yu et al., 2023a)
0.13
47.47
Vinsss (Yu al,, 2023a)
0.52
LLaVA-1.55 us (Liu al., 2023c)
0,26
46.94
LaVA-SFTTm (Sun al., 2023)
2.43
LLaVA-I.5 12B (Liu al., 2023c)
0.38
46.50
LLaVA-RLBF ya (Sun et al., 2023)
LLaVA-v (Liu et 2023c)
CODE na (Kim ct al.,, 2024)
2.1750002
2.53
2.49
0.50=
0.51
0.57
HAIXA (Ours)
LLaVA-RLHF (se(Sun ot al., 2023)
0.20
0.17
49. 49.10
46.41
HAIXAps (Ours)
2.58
0.45
VILA-v1.55 smass (Lin et al., 2024)
0.19
55, 55.39an000
VILA-v1.5 sooss (Lin et 2024)
2.58 10.02
+9.0L
HALVApRSS4 (Ours)
0.02.o.11 1Y
56.60
is
HALYApaccu (Ours)
2,58 00
0.45
1o
01
GPT4V! (Achiam al., 2023)
65.28
GPT4V (Achiam et al., 2023)
Gemini Pto Vision" (Team al. 2023)
-0.02
36.85
presented in Table 4 demonstrate that HALVA considerably improves performance with respect to LLaVA-v1.5. Furthermore, we observe that our approach is more effective in mitigating hallucination than existing RLHF, SFT, or DPO-based methods. For example, HALVA7 achieves a score of 2.25 surpassing the 7B variants of RLHF, DPO, and SFT -based methods, which report scores of 2.05, 1.97, and 1. 1.76, respectively. Moreover, HALVA 3B reduces the hallucination rate to 0.45, compared to 0.57 for VA-RLHF Note that as L.LVV--RLHE and LLaVA-SFT use the same language and vision encoders as HALVA (Vicuna-V1.5 and ViT-L/14), ensuring a fair direct comparison. The detailed results for the individual categories presented in Appendix C.
4.2 EVALUATION ON HALLUCINATION BENCHMARKS BEYOND OBJECT HALLUCINATION
To further stress-test DPA on other forms of vision- langun-a hallucinations that are not restricted to objects and may occur due to visual illusions, we evaluate performance on HallusionBench (Liu et al., 2023a). The results presented in Table 5 demonstrate that our proposed method directly benefits other forms of vision- languaaee hallucinations as well. HALVAGB. HALVA and HALVA (383844 improve the overall accuracy by 1.86%, 2.16%, and 1.21%. respectively, compared to their base models. Moreover, DPA mitigates Yes/No bias in MILM responses. Specifically, HALVAJ3B088 reduces Yes/No bias from 0.19 to 0.02. Detailed results on HallusionBench are in Appendix C.
4.3 EVALUATION ON NON- HHHHHHLLLLLLLLII BENCHMARKS
We further assess HALVA on general lasion-language tasks using four popular benchmarks: VQA-v2 (Goyal et al., 2017), MM-Vet (Yu et al., 2023b), TextVQA (Singh et al., 2019). MME (Fu et al., 2023), and ALLaVA-Bench-ine -hll--l--te--ll (LLaVA-BW) (Liu et al., 2024). We follow the evaluation protocol mentioned in LLaVA-v1.5 (Liu et al., 2023c). The results presented in Table 6 show that HALVA maintains or improves performance with respect to the base models. For example, HALVA improves on MME, MM-Vet, and LLaVA-BW by 16.3, 1%, and 1.8% respectively, while retaining the same performance on VQA-v2. A similar trend is noticed in the case of HALVA and HALVA13B88- Unlike HALVA existing finetuning methods such HA-DPO, and EOSTg, based on LLaVA-v1.5,, exhibit statistically significant performance drops in general tasks when tuned for hallucination mitigation. We present the details of our statistical analysis in Appendix 1.11.
8

---

Published as a conference paper at ICLR 2025
there flve people
Desanite iragee detail.
The image fectires a large American Adt aleplane parsied on The rumway at an diport. The alipline is nose flmmed tarmin: with nowe Beaning
idouad vchile thi iraege?
No
greeeyy Tiores zeveral poopre vis tie the soonee veog derzert stall on pnceenngaee Stare of theen are slarnligg nobr the paane wif ofieess ane scatiens and accune The peuastpaa of inansilaale assransi preparing
Yes, the dloud the
impreoodion Bim
Has, the cloud in the image whille
The same separtta tra.
lended
the right oningn dide smrller
la the cloud black
Dan the left orange cioke?
the imagge fectiree a lange Arerican dot nee wirphen peroed in sumnag serport wiplere pos tamac. appcors be proparing The
User
Ves: the right orange dirce
by greeay adds atmosphere coene
cleaad tiee
Juan orange sindll
No, the clous image not Blnck Io while cloud
No. Fie right orange ciddee larger
than the left orange cinde
Figure 6: Qualitative comparisons between HALVA [ and LLaVA-v1.5 [+)) J. Our proposed DPA effectively mitigates hallucination under different setups: (A) detail image description, (B) visual crestion-answering (C) Yes-or-No answer, (D) visual illusion. Hallucinations are highlighted in red. More examples, comparing with VAvv1.1. and VILA-v1.5, are in Appendix E.
Table 6: Results general vision-language tasks. "Results are computed by us, using their official checkpoint. Red underline indicates that the performance drop is statistically significant.
Method
VOA!"
MM-Vet, TeaVOA, MME.
LLaVA-BW,
78.5
31.1
58.3
1510.7
65.4
HA-DPOs
27.6.n.
30.7..
56.7
1502.
66.2rux
80S
77.6
31.
55.2
3.
1424
65
Sen
HALVA
78.5
32.1-10
1527.0miaa
67.2-1
Figure 5: Left: Changes in the model state due to
1 800
36.1
61.2
1530.1
72.5
DPA training with varying a. Right: Changes is alignment loss before and after training across all training samples. Default a 0.4 for HALVATB
HALVA
800
37.8
61.2
1591.1119
72.7-oz
VILA-vI 5ps
82.8
41.3
65.0
1569.6
80.8
HALVArg0s
44.3
61.8,02
1575.7
82441
4.4 ABLATION STUDY
Recalling the final DPA objective. which combines the alignment loss (C.) and KL divergence (La), defined as Lapa La + a La, we examine the change in model state with varying a, as depicted in Figure 5 (Left). The y axis represents the extent to which the model diverges from its initial state during DPA training, while the axis shows the change in the relative log- ogg-robbbbility of the hallucinated tokens. Each data point in this figure represents the calculated alignment loss and divergence after training for different values of The figure illustrates that with very low a, e.g. 0.01, the model substantially diverges from its initial state. As a increases, the model tends to retain a state similar to the base model. We empirically find that ===0 0.4 works optimally for HALVA The change in L. before and after DPA training computed over the entire training samples is presented in Figure 5 (Right). In-depth ablation studies on the proposed loss and generative data-augmentation are presented in Appendix C.
4.5 QUALITATIVE ANALYSIS
A qualitative comparison of HALVA to the base model is shown in Figure 6, with additional examples in Appendix E. HALVA consistently provides more accurate image descriptions than LLa VLaV5-11.5. For example, in Figure 6 (A), LLL hallucinates 'people', 'airport staff", 'passengers' in an image of parked airplane. contrast, HALVA accurately describes the image with necessary details. Additionally, our method does not exhibit LLa tendency to answer 'Yes' to most questions, which can contribute to hallucinations. This is shown in Figure 6 (B), where HALVA correctly answers 'Yes' when asked "Is the cloud white in the image?' and responds with 'No' when asked 'Is the cloud black in this image?', whereas LLaVA-vis answers 'Yes' to both cases. In another example, shown in Figure 6 (C), unlike LLaVA-v1.5. HALVA provides the correct answer to the number of people present in the image. Lastly, we present an example of hallucination caused by visual illusion in Figure 6 (D). While HALVA is not explicitly trained for such vision- hallucinations, our approach shows some ability to mitigate it.

---

Published as a conference paper at ICLR 2025
5 RELATED WORK
Multimodal LLM (MLLM). Vision-language models (VLMs) often align image and text features in a shared embedding space, pioneered by CLIP (Radford et al., 2021) and ALIGN (Jia et al., 2021), and others (Yu et al., 2022: Chen et al., 2022; Li et al., 2022; Wang et al., 2022). This alignment is achieved through contrastive learning on large image-text datasets. VLMs show strong generalization across various tasks. Leveraging LLMSs and vision encoders from VLMS like CLIP, recent MILMS (Liu et al., 2024; Zhu et al., 2023; Team et al., 2023; Achiam et al., 2023; Dai et al., 2023; Li et al., 2023c; Peng et al., 2023; Hu et al., 2024; Dai et al., 2023; Bai et al., 2023; Chen et al., 2023b) further enhance visual perception, understanding, and reasoning. While some MLLMs are open-source, others are only accessible through APIS (Achiam et al., 2023; Team et al., 2023; Bai et al., 2023). Among the publicly available MLLMs, LLaVA (Liu et al.. 2024; 2023c) and VILA (Lin al., 2024) are widely used due to their simplicity and the availability of code, models, and training data. This makes them suitable base models for demonstrating applicability of DPA on off-the-shelf MILMMs
Alignment. Reinforcement learning from human feedback (RLHF) (Christiano et al., 2017) aligns models by training a new model via KL--regularized RL. problem using an outcome reward. DPO (Rafailov et al., 2023) and many follow-ups (Azar et al., 2024; Pal et al., 2024; Tang et al., 2024; Amini et al., 2024) emerged as a simple alternative to RLHF that sidesteps reward modeling, Note that all these methods operate at the sequence level, which provides noisy feedback in alignment in all intermediate steps. On the other hand, recent work has focused on token-level aligmment methods (Mudgal et al., 2023; Zeng et al., 2024; Rafailov et al., 2024; Chakraborty et al., 2024) with a process reward (g-function). In contrast, our proposed DPA is an offline alignment method designed overcome two key limitations of existing methods: providing fine- inneggrinedd feedback through phrase-level alignment and restricting divergence by applying a strong token- wise forward KL regularizer.. Notably, the forward KL regularizer helps avoid the seeking behavior of reverse KL-based RL fine-tuning, which may lead to low diversity in generations ( (Wang et al., 2023a).
Hallucination mitigation. Multimodal hallucination generally refers to the misrepresentation of verifiable information in relation to the given input. This phenomenon has been primarily studied in the context of object hallucination (Rohrbach et al., 2018; Bai et al., 2024: Zhou et al., 2023; Sun et al., 2023; Biten et al., 2022). Prior work to mitigate this issue can be categorized into three phases: pretraining, where techniques include using balanced instruction-tume poneiinn--ningggg data with equal positive and negative examples (Liu et al. 2023b) or generating and correcting image-instruction pairs on-the-fly fly (Wang et al., 2024); inference, with methods involving specialized decoding strategies (Leng et al., 2023; Deng al., 2024a; Zhu et al., 2024) or iterative corrections using offline models to detect and correct hallucinations at inference time (Zhou et al., 2023; Yin et al., 2023); and finetuning, with approaches relying on human feedback (Sun et al., 2023; Yu et al., 2023a) to train reward models or employing preference optimization techniques (Zhao et al., 2023b; Yu et al., 2023a; 2024; Pi et al., 2024; Zhou et al., 2024; Deng et al., 2024b). While finetuning methods are more efficient direction as they do not require training from scratch (unlike gretraniigg-asse methods) nor changes in the serving infrastructure (unlike inference-based methods), existing finetuning approaches may deteriorate the performance of the base model on general Aision-language tasks (Figure 2). To address this, we introduce DPA, which effective in mitigating object hallucination on a broad set of vision-language tasks while retaining or improving the general abilities of the base model. In contrast to (Gunjal et al., 2024) that explores training a reward model to provide sub-sequence level feedback for preference optimization training, we introduce fine- ane-grrined objective function that can be directly used to finetune multimodal LLMs for hallucination mitigation.
6 CONCLUDING REMARKS
We introduce data-augmented phrase-level aligmment to mitigate object hallucination in MLLMs. Our approach uses generative data augmentation to create pairs of hallucinated and correct responses by selectively altering ground-truth phrases in the correct responses. These pairs are then used to train MLLMs with our proposed DPA loss, which reduces the relative log-likelihool of hallucinated tokens compared to correct ones. Our extensive study demonstrates the effectiveness of DPA in mitigating various forms of object hallucinations, including those related to existence and attributes, as well hallucinations arising from visual illusions complex charts. Additionally, unlike existing fine-tuning- based solutions, DPA effectively mitigates hallucination across diverse vision- languaee tasks while maintaining or even enhancing performance on general vision-language tasks.
10

---

Published as a conference paper at ICLR 2025
REFERENCES
Josh Achiam, Steven Adler, Sandhini Agarwal, Lama Ahmad, lige Akkaya, Florencia Leoni Aleman, Diogo Almeida, Janko Altenschmidt, Sam Altman, Shyamal Anadkat, et al. Gpt-4 technical report. arXiv preprint (tt1:2333.087774, 2023.
Afra Amini, Tim Vieira, and Ryan Cotterell. Direct preference optimization with an offset. arXiv preprint arXiv:2402. 10571, 2024.
Wenbin An, Feng Tian, Sicong Leng, Jiahao Nie, Haonan Lin, Qian Wang, Guang Dai, Ping Chen, and Shijian Lu. Agla: Mitigating object hallucinations in large vision-language models with assembly of global and local attention. arXiv preprint arXiv:2406.12271 2024.
Rohan Anil, Andrew M Dai, Orhan Firat, Melvin Johnson, Dmitry Lepikhin, Alexandre Passos, Siamak Shakeri, Emanuel Taropa, Paige Bailey, Zhifeng Chen, al. Palm 2 technical report. arXiv preprint arXiv:2305. 10400 2023.
Anas Awadalla, Irena Gao, Josh Gardner, Jack Hessel, Yusuf Hanafy, Wanrong Zhu, Kalyani Marathe, Yonatan Bitton, Samir Gadre, Shiori Sagawa, et al. Openflamingo: An open-source framework for training large autoregressive vision-language models. arXiv preprint arXiv: 013901139, 2023.
Mohammad Gheshlaghi Azar, Zhaohan Daniel Guo, Bilal Piot, Remi Munos, Mark Rowland, Michal Valko, and Daniele Calandriello. A general theoretical paradigm understand learning from human preferences. In International Conference on Artificial Intelligence and Statistics, pp. 4447-4455. PMLR, 2024.
Jinze Bai, Shuai Bai, Shusheng Yang, Shijie Wang, Sinan Tan, Peng Wang, Junyang Lin, Chang Zhou, and Jingren Zhou. Qwen-vl: A versatile vision-language model for understanding, localization, text reading, and beyond. 2023.
Zechen Bai, Pichao Wang, Tianjun Xiao, Tong He, Zongho Han, Zheng Zhang, and Mike Zheng Shou. Hallucination of multimodal large language models: A survey. arXiv preprint arXiv:2404. 18930, 2024.
Assaf Ben-Kish, Moran Yanuka, Morris Alper, Raja Giryes, and Hadar Averbuch-Elor. Mocha: Multi-objective reinforcement mitigating caption hallucinations. arXiv preprint arXiv:2312.03631, 2023.
Ali Furkan Biten, Lluis Gómez, and Dimosthenis Karatzas. Let there be a clock on the beach: Reducing object hallucination in image captioning. In WACV, pp. 1381-1390, 2022.
Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared D Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam. Girish Sastry, Amanda Askell, et al. Language models are few-shot learners. NeurlPS, 13:1877-1901, 2020.
Souradip Chakraborty, Soumya Suvra Ghosal, Ming Yin, Dinesh Manocha, Mengdi Wang, Am- rit Singh Bedi, and Furong Huang. Transfer q star: Principled decoding for Ilm alignment. arXiv preprint Xrxiv:2405. 240449405.. 2024.
Jun Chen, Deyao Zhu, Xiaoqian Shen, Xiang Li, Zechun Liu, Pengchuan Zhang, Raghuraman Krishnamoorthi, Vikas Chandra, Yunyang Xiong, and Mohamed Elhoseiny. Minigpt-v2: large language model as unified interface for vision-language multi-task learning, arXiv preprint arXiv:2310.09478. 2023a.
Keqin Chen, Zhao Zhang, Weili Zeng, Richong Zhang, Feng Zhu, and Rui Zhao. Shikra: Unleashing multimodal 11m's 's referential dialogue magic. arXiv preprint arXiv:2306. /5195, 2023b.
Xi Chen, Xiao Wang, Soravit Changpinyo, AJ Piergiovanni, Piotr Padlewski, Daniel Salz, Sebastian Goodman, Adam Gryener, Basil Mustafa, Lucas Beyer, et al. Pali: A jointly-scaled multilingual language-image model. arxiv preprint arXiv:2209.066. 06794, 2022.
Zhe Chen, Weiyun Wang. Hao Tian, Shenglong Ye, Zhangwei Gao, Erfei Cui, Wenwen Tong, Kongzhi Hu, Jiapeng Luo, Zheng Ma, et al. How far are we to gpt-4v? closing the gap to commercial multimodal models with open-source suites. arXiv preprint arXiv:2404 1682/, 2024.
11

---

Published as a conference paper at ICLR 2025
Wei-Lin Chiang, Zhuohan Li, Zi Lin, Ying Sheng. Zhanghao Wu, Hao Zhang, Lianmin Zheng, Siyuan Zhuang, Yonghao Zhuang, Joseph E. Gonzalez. lon Stoica, and Eric P. Xing. Vicuna: An open-source chatbot impressing gpt-4 with 90%* * chatgpt quality, March 2023. https:
///msys.org/bllog./.o 222200220002 00-0003--000 30 -vi cuna/.
Aakanksha Chowdhery, Sharan Narang, Jacob Devlin, Maarten Bosma, Gaurav Mishra, Adam Roberts, Paul Barham, Hyung Won Chung, Charles Sutton, Sebastian Gehrmann, et al. Palm: Scaling language modeling with pathways. JMLR, 24(240):1-133, 2023.
Paul F Christiano, Jan Leike, Tom Brown, Miljan Martic, Shane Legg, and Dario Amodei. Deep reinforcement learning from human preferences. NeurIPS, 30, 2017.
Yung-Sung Chuang, Yujia Xie, Hongyin Luo, Yoon Kim, James Glass, and Pengcheng He. Dola: Decoding by contrasting layers improves factuality in large language models. arXiv preprint arXiv:2309.03888 03883, 2023.
Wenliang Dai, Junnan Li, Dongxu Li, Anthony Meng Huat Tiong, Junqi Zhao, Weisheng Wang, Boyang Li, Pascale Fung, and Steven Hoi. Instructblip: Towards general- purpose vision-language models with instruction tuning. arXiv preprint arXiv:2305.0653 2023.
Ailin Deng, Zhirui Chen, and Bryan Hooi. Seeing is believing: Mitigating hallucination in large vision-language models via dip-guided decoding. arXiv preprint arxiv:2402.15300, 2024a.
Yihe Deng, Pan Lu, Fan Yin, Ziniu Hu, Sheng Shen, James Zou, Kai- Chang, and Wei Wang. Enhancing large vision language models with self-training on image comprehension. arXiv preprini arXiv:2405. 19716, 2024b.
Chaoyou Fu, Peixian Chen, Yunhang Shen, Yulei Qin, Mengdan Zhang, Xu Lin, Jinrui Yang, Xiawu Zheng, Ke Li, Xing Sun, Yunsheng Wu, and Rongrong Ji. Mme: A comprehensive evaluation benchmark for multimodal large language models. arXiv preprint arXiv:2306. 13394, 2023.
Peng Gao, Jiaming Han, Renrui Zhang, Ziyi Lin, Shijie Geng, Aojun Zhou, Wei Zhang, Pan Lu, Conghui He, Xiangyu Yue, et al. Llama-adapter v2: Parameter-efficient visual instruction model. arXiv preprint arXiv:2304.1 10004/55110 55100, 2023.
Tao Gong, Chengqi Lyu, Shilong Zhang, Yudong Wang, Miao Zheng, Qian Zhao, Kuikun Liu, Wenwei Zhang, Ping Luo, and Kai Chen. Multimodal-gph A vision and language model for dialogue with humans. arXiv preprint arXiv:2305.0477 2023.
Yash Goyal, Tejas Khot, Douglas Summers-Star Dhruv Batra, and Devi Parikh. Making the v in vqa matter: Elevating the role of image understanding in visual question answering. In CVPR, pp. 6904-6913, 2017.
Anisha Gunjal, Jihan Yin, and Erhan Bas. Detecting and preventing hallucinations in large vision language models. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 38, pp. 18135-18143, 2024.
Edward J Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, and Weizhu Chen. Lora: Low-rank adaptation of large language models. arXiv preprint DrXiv:2106.09655 2021.
Mingzhe Hu, Shaoyan Pan, Yuheng Li, and Xiaofeng Yang. Advancing medical imaging with language models: journey from n- n-grams to chatgpt. arXiv preprint arXiv:2304.0m 04920,, 2023.
Wenbo Hu, Yifan Xu, Yi Li, Weiyue Li, Zeyuan Chen, and Zhuowen Tu. Bliva: simple multimodal Ilm for better handling of text-rich visual questions. In AAAI, volume 38, pp. 2256-2264, 2024.
Qidong Huang, Xiaoyi Dong, Pan Zhang, Bin Wang, Conghui He, Jiaqi Wang, Dahua Lin, Weiming Zhang, and Nenghai Yu. Opera: Alleviating hallucination in multi-modal large language models via over-trust penalty and eetrospectionnallocattee allocation. arXiv preprint arXiv:2311.179) 2023.
Chao Jia, Yinfei Yang, Ye Xia, Yi-Ting Chen, Zarana Parekh, Hieu Pham, Quoc Le, Yun-Hsun Sung, Zhen Li, and Tom Duerig. Scaling up visual and vision- langunl representation learning with noisy text supervision. In ICMI., pp. 4904 4916. PMLR, 2021.
12

---

Published as a conference paper at ICLR 2025
Chaoya Jiang, Haiyang Xu, Mengfan Dong. Jiaxing Chen, Wei Ye, Ming Yan, Qinghao Ye, Ji Zhang, Fei Huang, and Shikun Zhang. Hallucination augmented contrastive learning for multimodal large language model. arXiv preprint arXiv:2312.0663 2023.
Junho Kim, Hyunjun Kim, Yeonju Kim, and Yong Man Ro. Code: Contrasting self self-geeneated description to combat hallucination in large multi-modal models. arXiv preprint arXiv:2406.0 00100020, 2024.
Ranjay Krishna, Yuke Zhu, Oliver Groth, Justin Johnson, Kenji Hata, Joshua Kravitz, Stephanie Chen, Yannis Kalantidis, Li-Jia Li, David Shamma, et al. Visual genome: Connecting language and vision using crowdsourced dense image annotations. IJV,, 123:32-73, 2017.
Hugo Laurençon, Lucile Saulnier, Léo Tronchon, Stas Bekman, Amanpreet Singh, Anton Lozhkov Thomas Wang, Siddharth Karamcheti, Alexander Rush, Douwe Kiela, al. Obelics: An open web-scale filtered dataset of interleaved image-text documents. NeurIPS, 36, 2024.
Seongyun Lee, Sue Hyun Park, Yongrae Jo, and Minjoon Seo. Volcano: mitigating multimodal hallucination through self-feedback guided revision. arXiv preprint vrXiv:2311.07360 2023.
Sicong Leng, Hang Zhang, Guanzheng Chen, Xin Li, Shijian Lu, Chunyan Miao, and Lidong Bing. Mitigating object hallucinations in large vision-lant vision-language models through visual contrastive decoding. arXiv preprint arXiv:2311.16922, 2023.
Bo Li, Yuanhan Zhang, Liangyu Chen, Jinghao Wang, Fanyi Pu, Jingkang Yang, Chunyuan Li, and Ziwei Liu, Mimic-it: Multi-modal in-context instruction tunigg. arXiv preprint arXiv:2306.05422 05425, 2023a.
Juncheng Li, Kaihang Pan, Zhiqi Ge, Minghe Gao, Hanwang Zhang, Wei Ji, Wenqiao Zhang, Tat- Seng Chua, Siliang Tang, and Yueting Zhuang. Empowering vision- lision-language models to follow interleaved vision- -annuale instructions. arXiv preprint arXiv:2308. 04152, 2023b.
Junnan Li, Dongxu Li, Caiming Xiong, and Steven Hoi. Blip: Bootstrapping language- image pre- training for unified vision-language understanding and generation. ICML, pp. 12888 12900. PMLR, 2022.
Junnan Li, Dongxu Li, Silvio Savarese, and Steven Hoi. Blip-2: Bootstrapping language- -maaag pre- training with frozen image encoders and large language models. arXiv preprint w:Xiv:2301.1259) 2023c.
Yifan Li, Yifan Du, Kun Zhou, Jinpeng Wang. Wayne Xin Zhao, and Ji-Rong Wen. Evaluating object hallucination large vision-language models. arXiv preprini arXiv:2305. 10355, 2023d.
Ji Lin, Hongxu Yin, Wei Ping, Pavlo Molchanov, Mohammad Shoeybi, and Song Han. Vila: On pre-training for visual language models. In CVPR, pp. 26689-26699, 2024.
Ziyi Lin, Chris Liu, Renrui Zhang, Peng Gao, Longtian Qiu, Han Xiao, Han Qiu, Chen Lin, Wenqi Shao, Keqin Chen, et al. Sphinx: The joint mixing of weights, tasks, and visual embeddings for multi-modal large language models. arXiv preprint arXiv:2311.07575 2023.
Fuxiao Liu, Tianrui Guan, Zongxia Li, Lichang Chen, Yaser Yacoob, Dinesh Manocha, and Tianyi Zhou. Hallusionbench: You what you think? you think what you see? image-context reasoning benchmark challenging for gpt-4v (ision), 1.5, and other multi-modality models. arXiv preprint arXiv:2310. 20..:23//..45000 /4566, 2023a.
Fuxiao Liu, Kevin Lin, Linjie Li, Jianfeng Wang. Yaser Yacoob, and Lijuan Wang. Mitigating hallucination in large multi-modal models via robust instruction tuning. In ICLR, 2023b.
Haotian Liu, Chunyuan Li, Yuheng Li, and Yong Jae Lee. Improved baselines with visual instruction tuning. arXiv preprint arXiv:2310.0372 2023c.
Haotian Liu, Chunyuan Li, Qingyang Wu, and Yong Jae Lee. Visual instruction tuning. NewrIPS, 36. 2024.
13

---

Published as a conference paper at ICLR 2025
Ilya Loshchilov and Frank Hutter. Decoupled weight decay regularization. arXiv preprint arXiv:171..S101. 2017.
Renze Lou, Kai Zhang, Jian Xie, Yuxuan Sun, Janice Ahn, Hanzi Xu, Yu Su, and Wenpeng Yin. Muffin: Curating multi-faceted instructions for improving instruction following. In ICLR, 2023.
Sidharth Mudgal, Jong Lee, Harish Ganapathy, YaGuang Li, Tao Wang, Yanping Huang, Zhifeng Chen, Heng-Tza Tze Cheng, Michael Collins, Trevor Strohman, et al. Controlled decoding from language models. arXiv preprint arXiv:2310. 17022, 2023.
Arka Pal, Deep Karkhanis, Samuel Dooley, Manley Roberts, Siddartha Naidu, and Colin White. Smaug: Fixing failure modes of preference optimisation with dpo-positive. arXiv preprint arXiw:2402.13228, 2024.
Zhiliang Peng, Wenhui Wang, Li Dong, Yaru Hao, Shaohan Huang, Shuming Ma, and Furu Wei. Kosmos-2: Grounding multimodal large language models the world. arXiv preprint arXiv:2306. 14824, 2023.
Renjie Pi, Tianyang Han, Wei Xiong, Jipeng Zhang. Runtao Liu, Rui Pan, and Tong Zhang. Strength- ening multimodal large language model with bootstrapped preference optimization. arXiv preprint arXiv:2403.cm 08730, 2024.
Yao Qin, Chiyuan Zhang, Ting Chen, Balaji Lakshminarayanan, Alex Beutel, and Xuezhi Wang. Understanding and improving robustness of vision transformers through patch-based negative augmentation. NeurIPS, 35:16276-16289, 2022.
Xiaoye Qu, Qiyuan Chen, Wei Wei, Jishuo Sun, and Jianfeng Dong. Alleviating hallucination in large lisiua-language models with active retrieval augmentation. arXiv preprint 005555 2024.
Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et al. Learning transferable visual models from natural language supervision. In ICML, pp. 8748 8763. PMLR, 2021.
Rafael Rafailov, Archit Sharma, Eric Mitchell, Christopher D Manning, Stefano Ermon, and Chelsea Finn. Direct preference optimization: Your language model is secretly reward model. NeurIPS, 36, 2023.
Rafael Rafailov, Joey Hejna, Ryan Park, and Chelsea Finn. From r to g": Your language model is secretly q-function. arXiv preprint 1232v 2024.
Colin Raffel, Noam Shazeer, Adam Roberts, Katherine Lee, Sharan Narang, Michael Matena, Yanqi Zhou, Wei Li, and Peter J. Liu. Exploring the limits of transfer learning with a unified text-to-text transformer. JMLR, 21(140):1-67 2020.
Samyam Rajbhandari, Olatunji Ruwase, Jeff Rasley, Shaden Smith, and Yuxiong He. Zero-infinity: Breaking the gpu memory wall for extreme scale deep learning. In SC, pp. 1-14, 2021.
Jie Ren, Samyam Rajbhandari, Reza Yazdani Aminabadi, Olatunji Ruwase, Shuangyan Yang. Minjia Zhang. Dong Li, and Yuxiong He. Zero-offload: Democratizing billion-scale model training. In USENIX ATC, pp. 551-564, 2021.
Anna Rohrbach, Lisa Anne Hendricks, Kaylee Burns, Trevor Darrell, and Kate Saenko. Object hallucination in image captioning. arXiv preprint arXiv:1809.02I1 2018.
John Schulman, Filip Wolski, Prafulla Dhariwal, Alec Radford, and Oleg Klimov. Proximal policy optimization algorithms. arXiv preprint 1rXiv:1707.063: 2017.
Amanpreet Singh, Vivek Natarajan, Meet Shah, Yu Jiang, Xinlei Chen, Dhruv Batra, Devi Parikh, and Marcus Rohrbach. Towards vqa models that can read. In CVPR, pp. 8317-8326, 2019.
Zhiqing Sun, Sheng Shen, Shengeao Cao, Haotian Liu, Chunyuan Li, Yikang Shen, Chuang Gan, Liang- Yan Gui, Yu-Xiong Wang. Yiming Yang, et al. Aligning large multimodal models with factually augmented rlhf. arXiv preprint arXiv:2309. 14525, 2023.
14

---

Published as a conference paper at ICLR 2025
Yunhao Tang, Zhaohan Daniel Guo, Zeyu Zheng, Daniele Calandriello, Rémi Munos, Mark Rowland, Pierre Harvey Richemond, Michal Valko, Bernardo Ávila Pires, and Bilal Piot. Generalized preference optimization: unified approach to offline alignment. arXiv preprint arXiv:2402.05749, 2024.
Gemini Team, Rohan Anil, Sebastian Borgeaud, Yonghui Wu, Jean-Baptiste Alayrac, Jiahui Yu, Radu Soricut, Johan Schalkwyk. Andrew M Dai, Anja Hauth, et al. Gemini: a family highly capable multimodal models. arXiv preprint arXiv:2312.11805.. 2023.
Hugo Touvron, Thibaut Lavril, Gautier Izacard, Xavier Martinet, Marie-Anne Lachaux, Timothée Lacroix, Baptiste Rozière, Naman Goyal, Eric Hambro, Faisal Azhar, et al. Llama: Open and efficient foundation language models. arXiv preprint arXiv:2302.13991 2023a.
Hugo Touvron, Louis Martin, Kevin Stone, Peter Albert, Amjad Almahairi, Yasmine Babaei, Nikolay Bashlykov, Soumya Batra, Prajjwal Bhargava, Shruti Bhosale, et al. Llama 2: Open foundation and fine-tuned chat models. arXiv preprint wrx.x:2307.09288 2023b.
Bin Wang, Fan Wu, Xiao Han, Jiahui Peng, Huaping Zhong. Pan Zhang, Xiaoyi Dong, Weijia Li, Wei Li. Jiaqi Wang, et al. Vige: Visual instruction generation and correction. In AAAI, volume 38, pp. 02.53309-5 5309-53177,, 2024.
Chaoqi Wang, Yibo Jiang, Chenghao Yang, Han Liu, and Yuxin Chen. Beyond reverse kl: Gen- cralizing direct preference optimization with diverse divergence constraints. arXiv preprint 1IX1::2309.16240, 2023a.
Junyang Wang, Yuhang Wang, Guohai Xu, Jing Zhang, Yukai Gu, Haitao Jia, Ming Yan, Ji Zhang, and Jitao Sang. An I1m- free multi-dimensional benchmark for milms hallucination evaluation. arXiv preprint ar arXiv:2311.0739/ 2023b.
Peng Wang, An Yang, Rui Men, Junyang Lin, Shuai Bai, Zhikang Li, Jianxin Ma, Chang Zhou, Jingren Zhou, and Hongxia Yang. Ofa: Unifying architectures, tasks, and modalities through a simple sequence-ter cooence-to-seeee ro-seuuence learning framework. In ICML, pp. 23318-23340. PMLR, 2022.
Sheng Wang, Zihao Zhao, Xi Ouyang, Qian Wang, and Dinggang Shen. Chatcad: Interac- tive computer-aided diagnosis on medical image using large language models. arXiv preprint arXiv:2302.07257, 2023c.
Junfei Wu. Qiang Liu, Ding Wang, Jinghao Zhang, Shu Wu, Liang Wang, and Tieniu Tan. Logical closed loop: Uncovering object hallucinations in large vision- languaee models. arXiv preprint arXiv:2402. 1622, 2024.
Qinghao Ye. Haiyang Xu, Guohai Xu, Jiabo Ye, Ming Yan, Yiyang Zhou, Junyang Wang, Anwen Hu, Pengcheng Shi, Yaya Shi, et al. mplug-owl: Modularization empowers large language models with multimodality. arXiv preprint arXiv:2304. 2no::233444477 /4/78, 2023a.
Qinghao Ye, Haiyang Xu, Jiabo Ye, Ming Yan, Haowei Liu, Qi Qian, Ji Zhang, Fei Huang, and Jingren Zhou. mwluu-ow12: Revolutionizing mulli-modaa large language model with modality collaboration. arXiv preprint arXiv:2311.04257 2023b.
Shukang Yin, Chaoyou Fu, Sirui Zhao, Tong Xu, Hao Wang, Dianbo Sui, Yunhang Shen, Ke Li, Xing Sun, and Enhong Chen. Woodpecker: Hallucination correction for multimodal large language models. arXiv preprint arXhv:2310.16045, 2023.
Jiahui Yu, Zirui Wang, Vijay Vasudevan, Legg Yeung. Mojtaba Seyedhosseini. and Yonghui Wu. Coca: Contrastive captioners are image-text foundation models. arXiv preprint arXiv:2205.01917. 2022.
Tianyu Yu, Yuan Yao, Haoye Zhang, Taiwen He, Yifeng Han, Ganqu Cui, Jinyi Hu, Zhiyuan Liu, Hai-Tao Zheng, Maosong Sun, et al. Rlhf-v: Towards trustworthy milms via behavior alignment from fine-grained correctional human feedback. arXiv preprint 2220:2312.000844 2023a.
Tianyu Yu, Haoye Zhang, Yuan Yao, Yunkai Dang, Da Chen, Xiaoman Lu, Ganqu Cui, Taiwen He, Zhiyuan Liu, Tat-Seng Chua, et al. Rlaif-v: Aligning milms through opun-source ai feedback for super gpt-4v trustworthiness. arXiv preprint 4405 17220, 2024.
15

---

Published as a conference paper at ICLR 2025
Weihao Yu, Zhengyuan Yang, Linjie Li, Jianfeng Wang, Kevin Lin, Zicheng Liu, Xinchao Wang, and Lijuan Wang. Mm-vet: Evaluating large multimodal models for integrated capabilities. arXiv preprint arXiv:2308. 23t.:233000249900, 2023b.
Zihao Yue, Liang Zhang, and Qin Jin. Less is more: Mitigating multimodal hallucination from an eos decision perspective. arXiv preprint arXiv:2402. 22544222.4455 14545, 2024.
Yan Zeng, Hanbo Zhang, Jiani Zheng, Jiangnan Xia, Guogiang Wei, Yang Wei, Yuchen Zhang, and Tao Kong. What matters in training gpt4-style language model with multimodal inputs? arXiv preprint arXiv:2307.02460 2023.
Yongcheng Zeng, Guoqing Liu, Weiyu Ma, Ning Yang. Haifeng Zhang, and Jun Wang. Token-level direct preference optimization. arXiv preprint wxxi:2244..11999 2024.
Xiaohua Zhai, Basil Mustafa, Alexander Kolesnikov, and Lucas Beyer. Sigmoid loss for language image pre-training In ICCV, pp. 11975-11986, 2023.
Haozhe Zhao, Zefan Cai, Shuzheng Si, Xiaojian Ma, Kaikai An, Liang Chen, Zixuan Liu, Sheng Wang, Wenjuan Han, and Baobao Chang. Mmicl: Empowering vision-language model with multi-modal in-context learning. arXiv preprint arXiv:2309.07915 2023a.
Zhiyuan Zhao, Bin Wang, Linke Ouyang, Xiaoyi Dong, Jiaqi Wang, and Conghui He. Beyond hallucinations: Enhancing Ivlms through dallucinatiinnawwan direct preference optimization. arXiv preprint arXiv:2311. 1.t.:9991,..699, 2023b.
Chenyu Zheng, Guoqiang Wu, and Chongxuan Li. Toward understanding generative data augmenta- tion. NeurlPS, 36, 2024.
Kaizhi Zheng, Xuchai He, and Xin Eric Wang. Minigpt-5: Interleaved vision-and-lange ann-uang generation via generative vokens. arXiv preprint arXiv:2310.02239, 2023.
Yiyang Zhou, Chenhang Cui, Jaehong Yoon, Linjun Zhang, Zhun Deng, Chelsea Finn, Mohit Bansal, and Huaxiu Yao. Analyzing and mitigating object hallucination in large vision-language models. arXiv preprint 10.00:2310000744 2023.
Yiyang Zhou, Chenhang Cui, Rafael Rafailov, Chelsea Finn, and Huaxiu Yao. Aligning modalities in vision large language models via preference fine-tuning arXiv preprint arXiv:2402.11411 2024.
Deyao Zhu, Jun Chen, Xiaoqian Shen, Xiang Li, and Mohamed Elhoseiny. Minigpt-4: En- hancing vasion-language understanding with advanced large language models. arXiv preprint 105V:2230410592, 2023.
Lanyun Zhu, Deyi Ji, Tianrun Chen, Peng Xu, Jieping Ye, and Jun Liu. Ibd: Alleviating hallucinations in large vision-language models via image-biased decoding. arXiv preprint arXiv:2402. 18476, 2024.
Xin Zou, Yizhou Wang. Yibo Yan, Sirui Huang, Kening Zheng, Junkai Chen, Chang Tang, and Xuming Hu. Look twice before you answer: Memory- visual retracing for hallucination mitigation in multimodal large language models. arXiv preprint 10X05:2410003577, 2024.
16

---

Published as a conference paper at ICLR 2025
Appendix
The organization of the appendix is as follows:
Appendix A: Pseudo code
Appendix B: Distinction between ours DPA and DPO-based hallucination mitigation methods
Appendix C: Additional experiments and eesults
Appendix D: Implementation details
Appendix E: Qualitative results
Appendix F: Limitations
A DPA PSEUDO CODE
Our proposed DPA is fairly straightforward implement. Below, we provide PyTorch-based pseudo code. Please note that this is a minimal implementation to present the key steps of our algorithm. Some of the intermediary and rudimentary steps (e.g., ignoring padded inputs during loss calculation) are intentionally omitted for brevity. The code will be made publicly available.
import torch
import torch nn. functional as
def forward (self, *+inputs):
"""x: vision-language input
y_pos: correct response of x
neng hallucinated response of x constructed through gen. data aug. rer,,, y_ref: reference input-cutput pair to calculate divergence ***
batch_size shape [0]
forward pass with correct and hallucinated responses
pos_legits self.model (x, Y_pcs)
neg_logits self.model (x, y_neg)
calculate log-prooaoilities pos pos_logps, pos_labele neg_logps, neg_labels self. _ne__oogits,,.eeeeoeeoeeeeoieo_s y_neg)
sppss_oogiteeosooo y-pos)
accumulate log-probabilities of
correct and hallucinated tckens phrase level
pos wos_logps self.accumulate Logps (pos (pos_logps) neg_logps self. wtt.faaccullaateeegogo.o (neg_logps)
phrasc-level alignment loss
alignment_ion torch. alignment_loss alignment_less, _laseetttlssssteeeeeeeeeee (ctimennt_csss.mmant mean
torch. (neg_logp pos_logps)
forward pass with the reference samples logits self wal.f.moeelff (x_reerr ref, y_ref)
with prrch.no_grad())) Seerence_logite. = self. meee model (x_ret ref, y ref)
calculate probability proba F.softmax (logits, dim=-1)
reference_proba = F. softmax (reference dim=-1)
tcken-wise KL divergence
divergence
ceffrrecc__rrbat creference_probae log() -proba. log )))
divergence divergence sum hbtbchhhhiieeeeee
final loss
loss - al i goseent_ooss_____________ loss aal..aalhaadiverree
17

---

Published as a conference paper at ICLR 2025
return loss
B DISTINCTION BETWEEN OURS DPA AND DPO-BASED HALLUCINATION MITIGATION METHODS
Several existing and concurrent works, such as HA-DPO (Zhao et al., 2023b), RLHF-V (Yu ct al., 2023a), and RIAIFF (Yu et al., 2024), have introduced hallucination mitigation techniques for MI.LMs, that are derived from DPO (Rafailov al., 2023). Following, we discuss the differences hetween our proposed DPA and DPO.
We write both DPA (ours) and DPO (Rafailov et al., 2023) objectives using the same notations, which are as follows: Ty as the model being trained; Teet as the frozen reference model; æ as the input; yf and y² as correct and hallucinated responses; 'D as training samples. We express v as sequence of tokens Th = {t] tt,,. .., f and denote the i-th hallucinated phrase y = Ti. [ ef where si and e are the start and end indices of v with S x S cf < [Ta]. Similarly, y is expressed as a sequence tokens T, = {ti, ...... : .fr.....fr... fr.1). and denote the i-th correct phrase vf To(sf <f]. where sf and ef are the start and end indices of Dt with 1 1 of S of s [Tal. N is the total number of hallucinated phrases in y"; and 8 are loss coefficients to control the influence of the reference model in training. For the sake of simplicity, we assume that {æe, Ue } are reused reference sample in DPA. Therefore, as discussed in Section 2, the final DPA loss can be expressed
I (6)T,,
Edpa (To; Fret) E E( (/)
log
N
no (tgla,))))) + II ne(t)) x t2, j=sf
II
s=mf
phrase-lvell alignmest loos
TT
+
Teer (Gl,,, (2n))
Log Taar (T,n(())),,,)))) log 10g(To(()le,
loken-wise divergence
On the other hand, the training objective of DPO is:
(x) log o log TTr(y)
3 log No (x) Terl (y (x)
Edpo (Te) Teet) -E /
Note that in our proposed DPA (Capa), given {z, y", y²), calculate the phrase-level alignment loss based on the log-put log-robbabillties of the tokens the hallucinated phrases and not on the tokens of sequence. Additionally, the KL-regularizer is applied at the token-level to closely retain the vision-language capabilities of the hase model. In DPO (Coro), however, given v. yf, , the reward margin between the correct and hallucinated responses is maximized to increase the log-likelihood of the correct response while reducing that of the hallucinated response. Despite the fact that the loss formulation of DPO is different from ours DPA, one fundamental difference that their loss is calculated at sequence level. i.e., penalizing all the tokens of a hallucinated response. Intuitively, the training objective of DPA provides more localized and fine-grained feedback unlike DPO (Rafailov et al.. 2023) and other existing alignment techniques (Christiano et al., 2017; Schulman et al., 2017). This makes DPA unique and effective compared to existing and concurrent works.
Accordingly, the nature of the correct and hallucinated responses used in DPO-based methods and our DPA also differ. To illustrate this we present one side-by-side comparison using a training sample from HA-DPO (Zhao et al., 2023b) and ours in Figure S1, which shows that while HA-DPO make changes at the sequence level, we apply changes at the word or phrase-level to construct the negative responses. In particular, unlike, HA-DPO, we selectively alter the ground truth information in the correct description, while keeping the rest of the response intact.
18

---

Published as a conference paper at ICLR 2025
Chosen: choto euoling monont snowboarder ceccuting
juni. sodilaading
Correct: A ancobboander
The anowboorddrr surrounded
veerrnn
hasa a Miue sky in the beciggrond Me has patch of cleae blue sky baiaand fimm. The enowbbartio is doing
safedy octietyy The snowy annaccaee with yoos the bookdhop cots the poono, and knowfbcarder takes center stage, exn bigiing impressive skil and afhieticio as they spar throuhh
hiss lags bent with arms extnnied right
weeving
Rejacted:
The pidium decidd elecattying momantt of antwwbcander eswazing rn cieir Hallucinated: jamping and twri fix wctwnried downeant black and white has has right herd The older a swewenn a white west with track number on the back.
sumcunded
jumn with ancesbrent desety voctle under medeemseth. ancottoonder seertin heiree bety bockshop conter stage, showcool that obdll and oth edoiom they soar through alr.
bchind
setivtyy snowboarder takes
We present training samples from the DPO-based method on the left (from HA-DPO) and ours on the right, highlighting differences in the nature of the negative samples. While HA-DPO makes changes (highlighted in blue) at sequence level, we apply one-to-one changes (highlighted in green and red) at the word or phrase-level to construct the negatives. The positives are referred to as "Chosen' in HA-DPO, while refer to them 'Correct'; and the negatives in HA-DPO, while we refer to them an "Hallucinated", Since there are no overlapping samples of sample that closely resemble each
Figure $1:
referred to 'Reject'
descriptive responses between HA-DPO and our data, other.
19

---

Published as a conference paper at ICLR 2025
C ADDITIONAL EXPERIMENTS AND RESULTS
C.I ABLATION ON LOSS
Recall our final objective function, which is comprised of both alignment loss (Ca), and token-wise KL divergence (Ca) between the ne (the model being trained) and (the reference model that is kept frozen). defined Ldno L. + La. First, study the behavior of HALVA with varying Simply put, lower allows we to diverge more from Tiet, whereas higher a aligns Ty more closely with Met. By default, we initialize both We and Tiat from the same base model. Therefore, a higher a would result in Ty to perform the same as the base model. Following, we analyze the impact of varying a on HALVA and HALVA while tracking their performance observe that for HALVATB the MME-Hall dataset. The results are presented in Figures S2 and S3, We similar to the highest performance. We present qualitative examples in Figure S4, showing the adverse effect of using very low By default, we use 0.4 for HALVARB. a 0.5 for HALVAI3 and 0.2 for HALVAI
of between 0.3 and 0.4 yields better outcome, whereas the model behaves model when a > 0.4. For (ALVA on the other hand, an a the range 0.4 to 0.6 shows
0.500
0.250
660
Phrase levelleee aligrment LOGS
0.540
Token-aise KL Divergence
0.200
8.35.5
2-0.4
570
0.520
2-
MME Subset)
860
0.500
0.150
650
Dew
0.480
0.100
0.400
0.400
0.000
0.400
0.000
100
200
300
100
3.0 0.1 5.2 0.25
0.3
0.35 0.4 0.45 0.5
Steps
Stops
(b) Performance MME-Hal Hall. with varying d.
(a) The training curves with varying a.
Figure S2: The training curves with varying a (a) and their performance on object hallucination (b) are presented. a the range 0.3 to 0.4 achieves optimal performance on the 7B variant.
0.500
0.500
0
-=0
660
Phrage-lar alggmeett La5s
Diwergence
0.200
0.520
0.500
0.150
640
2.000
0.460
6:3
0.050
0.400
0.000
00
0.1
03
0.5
06
0.7
0.8
Steps
Steps
(b) Performance on MME-Hall, with varying a.
(a) The training curves with varying
Figure S3: The training curves with varying a (a) and their performance on object hallucination (b) presented. in the range of 0.4 to 0.6 achieves optimal performance the 13B variant.
20

---

Published as a conference paper at ICLR 2025
Question: Describe the image in detail.
Quest ion: Provide a one-sentence caption for the provided image.
= 0.01: The image is black and white photo of man and little girl sitting together couch. The man is holding the little girl, and they are both smiling.
0.01: painting of house with white house with pink house with large white house with pink house with a large white house with a pink house with a large white house with a pink house with large white house with pink house with a large white house with pink house with large
0.4 (default) The image black and white photo of man and young girl sitting together on couch. The man is wearing a tie, and the little girl is wearing dress. They are both smiling and appear to be enjoying their time together.
a = 0.4 (default): A painting of a house with a white porch and pink flowers.
Figure $4: Qualitative examples demonstrating the impact of DPA training with very low As presented here, training with a very low a of 0.01 may occasionally hurt the language generation ability of an MILM. The example the left side shows instance of degeneration, while the example on the right side shows lack of descriptive power, failing to mention key details in the description, such as 'the man wearing tie' or 'the girl is wearing dress'. The 7B variant used in this study.
21

---

Published as a conference paper at ICLR 2025
C.2 ABLATION ON GENERATIVE DATA AUGMENTATION
We perform ablation study explore the effect of different sampling strategies which have been used in generative data augmentation. As mentioned in Section 2, generate hallucinated responses in three setups: closed-set co-occurrences (9K), open-set co-occurrences (1 (11K), and Yes-or-No No questions (1.5K). We generate total of 21.5K samples that contains 28K unique pairs of correct and hallucinated phrases based on 5K unique hallucinated objects. We study the impact of these categories along with their varying number of samples. tuning
We perform this study on HALVAn and the same training hyperparameters those obtained on the entire data. From the results presented Table SI, three key observations are made. First, open-set hallucinated descriptions show henefits reducing hallucinations generative tasks, as evidenced by superior performance on CHATR. Second, mixing the Yes-or-No hallucinated responses reduces hallucination in discriminative tasks, leading to F1 boost the AMBER dataset. Finally, combining all the splits results in overall improvements competitive performances statistics all the splits Table S2. Figure S5, we present the training curves different generative data augmentations, demonstrating stability during training across various data splits.
broader range of tasks. We present the key
Table S1: Ablation study on sampling strategy used in generative data augmentation. C2 and C, refer to CHAIR at instance and sentence-level; Fl refers to the Fl-scores of all the discriminative tasks and HR refers hallucination rate on generative tasks.
CHAIR
AMBER
MME-Hall
Data Split
C, +
C,
F1
HR
Score
Closed set
12.6
45.0
73.9
34.7
643.3
Open-set
11.2
39.6
73.1
33.3
643.3
Closed set Open-set (50%)
11.7
41.8
79.8
32.0
643.3
Closed set Open-set
12.6
43.6
74.1
34.0
648.3
Closed set Open-set + Y-or-N (50%)
11.8
43.2
82.4
32.2
641.0
Closed set Open-set Y-or-N
11.7
41.4
83.4
32.2
665.0
Table S2: Key statisties of training samples used in DPA training.
# Avg. hallucinated instances per sample
Length (in words)
Data Split
# Samples
Avg/Min./Max.
One-sentence caption
528
2.7
15/6/53
Short description
11573
6.9
42/12/128
Detailed description
8268
11.3
71/32/246
Yes-or-No No (one word answer)
1510
1
1/1/1
Full
21874
8.1
49/1/246
0.65
Closed set
levelleeelllee Alignment Loss
Token-wise KL Divergencee
50% data
data
0.10
0.35
0.08
Phrase-level
0.50
0.45
0.02
0.40
0.00
0
100
200
300
100
200
300
Steps
Steps
Figure 85: Training curves for different generative data augmentations using 0.4.
22

---

Published as a conference paper at ICLR 2025
Table S3: Ablation study on divergence measure using HALVATB- (a) We find that using seen samples as the reference data for divergence measure achieve overall better performance. (b) Our study shows that initializing the reference model and the model being trained from the same checkpoint, achieves optimal performance. C and C. refer to CHAIR at instance and sentence-level; Fl refers to the Fl-scores of the discriminative tasks and HR refers to hallucination rate in the image descriptions.
(a) Ablation study reference data.
(b) Ablation study reference model.
Ref. Data
CHAIR
AMBER
MME-Hall
Ref. Model
CHAIR
AMBER
MME-Hall
C. C.
FIt
HR
Score
C, 4
C, 4
FI
HR
Score t
Unseen data
12.7
47.4
81.7
34.7
668.3
7B
11.7
41.4
83.4
32.2
665,0 640.0
Seen data
11.7
41.4
83.4
32.2
665.0
13B
12.4
45.2
80.1
34.7
C.3 ABLATION ON DIVERGENCE MEASURE
Reference data. We experiment with the reference data that has been used to measure KL divergence with respect to the reference model. We briefly experiment in two setups:
Unseen data: we directly use the vision-language instructions and correct responses the reference samples.
Seen data: we fraction the instruction tuning dataset the base model originally trained on, and use them as reference samples.
We perform this experiment on HALVATB and the results are presented in Table S3 (a). The results demonstrate that using seen samples to measure divergence gives better estimate of model state during training, and accordingly the tuned model overall performs better,
various benchmarks.
Reference model. By default, we initialize the reference model (the model kept frozen) and online model (the model being trained) from the same checkpoint. Additionally, we experiment with initializing the reference model different than the model being trained. In particular, we experiment with training LLaVAn while using LLaVA as the reference model. We find this interesting to explore as both LLaVAnB and LLaVA are originally trained in similar setup, and V performs relatively hetter compared to the L. VArp of the benchmarks (Liu al., 2023c). The results presented Table $3 (b) show that initializing the reference model and the online model from the same checkpoint, achieve optimal performance. We believe this is likely since the reference model initialized from an identical state of the model being trained, gives a true estimate of variety of benchmarks.
divergence and accordingly optimized model performs better
C.4 DETAILED RESULTS OF AME--ALL
In Table S4, present the detailed results of the MME-Hall (Fu al., 2023) benchmark across its four sub-categories: existence, count, position, and color. Our results indicate that DPA mitigates retains the same performance as the base model) object hallucination across different aspects, unlike prior finetuning methods such as HA-DPO (Zhao et al., 2023b) and EOS (Yue et al., 2024), or inference-based methods such as VCD (Leng et al., 2023) and Woodpecker (Yin et al., 2023), which either degrade overall performance show improvement in calegory but suffer in others.
Table S4: Detailed results on MME-Hall.
Object
Attribute (1)
Method
Total (1)
Existence
Count
Position
Color
LLaVA-v1.5t
190.0
155.0
133.3
170.0
648.3
HA-DPOm
190.0
133.3
136.7
158.3
618.3
BOS
190.0
138.3
118.3
160.0
606.7
VCDrR
184.7
138.3
128.7
153.0
604.7
Woodpeckersn
165.0
98.3
56.7
46.7
366.7
HALVA (Ours)
190.0
165.0
135.0
175.0
665.0
LLaVA-v1.5mm
185.0
155.0
133.3
170.0
643.3
HALNAm (Ours)
190.0
163.3
141.7
180.0
675.0
VILA-vI.5mmmus
185.0
170.0
148.3
185.0
688.3
HALVA (Ours)
185.0
173.3
148.3
185.0
691.7
23

---

Published as a conference paper at ICLR 2025
C.5 DETAILED RESULTS AMBER
Table SS, we present the detailed results of AMBER (Wang et al., 2023b). shown, HALVA reduces hallu- cination (e.g., from 7.8 to 6.6) while improving object coverage (from 51% to 53%) in image description tasks, outperforming HA- IA-D, EOS, and Woodpecker. HALVA significantly improves performance on discriminative tasks, achieving Fl score improvements of up 13.4%.
Table S5: Detailed results AMBER.
Generative Task
Discriminative Task (F1")
Method
CHAIR
Coverage (1) Hall, Rate (L) Cognition (L) Existence Attribute Relation Overall
LLaVA-v1.51
7.8
51.0
36.4
4.2
83.3
64.6
65.6
74.7
HA-DPOts
3.3
88.1
66.1
78.1
EOST
5.1
49.1
22.7
2.0
82.8
67.4
69.2
75.6
Woodpeckerya
6.9
48.9
30.4
3.6
81.7
53.5
41.5
67.0
HALNAcH (Ours)
53.0
32.2
3.4
93.3
63.1
83.4
LLaVA--V--Spa
6.6
51.9
30.5
3.3
78.5
70.2
45.0
73.1
HALVA (Ours)
6.4
52.6
30.4
3.2
92.6
81.4
73.5
86.5
VILA-vI.5 iS8084
9.9
63.3
56.1
4.8
87.5
77.8
66.7
82.2
HALVA (Ours)
9.1
63.9
54.2
4.0
93.9
82.6
75.9
87.9
C.6 DETAILED RESULTS OF MMHAL-BENCH
In Table S6, we present the detailed results of MMHal- BenHh (Sun et al., 2023) across its eight sub-categories. Our proposed DPA demonstrates consistent effectiveness mitigating object hallucinations in fallowing types: adversarial, comparison, relation, and holistic on both HALVA and HALVAna Additionally. DPA improves performance in out of 8 subcategories for both the 13B variants. Moreover, recent hallucination miti- gation methods such HA-DPO and EOS prove ineffective in addressing such broad categories of hallucinations, even resulting worsened baseline performance.
Table S6: Detailed results on MMHal-Bende
Overall
Hall.
Score Question Type (f)
Method
Score (1)
Rate
Attribute Adversarial Comparison
Counting Relation Envireament
Holistic
Other
LLaVA-v1.3
2.11
0.56
3.
1.00
1.61
1.97 sa0e 2.36
3.20
2,1420.50
1.53
HA-DPOm
1.97_0.0
1.
1.08
11440
1.89
2.22
3.31
1.42
1.17
EOST
2.03
0.59
2.09
1.78
1.89
1.53
2.09
3.08
1.6720.29
1.53
HALVAcH (Ours)
_o.10 g..4
2.7810.0
1.47
1.97
1.89
3.03
3.20
2.42
1.22 jo:2r
I.I.VAyyl.5g
2. 2.38=0.02
0.50
3,20
2.53
2.55
2.20
1.97
3:332014
1.50+0.22
1.72-20.18
HALVA1и (Ours)
2s
0.46 oz
4.03
2.58
2.66.0
2.08
2.45g0.5
3.36
2.44gn39
2.
VILA-vI 11
2.58 D. 0.46
3.3640.13
1.08
3.39
2.05
2,97
3.11
2.19
2.47.0.0s
HALVApmost (Ours)
0..06
0.45 10.00
3.11
10.0
1.47
2.08
1o00
3.11ga.15
3.19
1.641021
2.58 20.0
C.7 DETAILED RESULTS OF HALLUSIONBENCH
Table S7, we present the detailed results of HallusionBench (Liu al., 2023a), which evaluates MLLMs beyond object hallucination, including those may cause by visual illusions and quantitative analysis form charts or graphs, among others. In addition to improving the overall performance, the results demonstrate the effectiveness of DPA on all the sub-categories (i.c.. easy set, hard set) of HallusionBench as well. For example, find that HALVAR and HALVAIs8 substantially improve performance (4.34%-6 -..9%%% the Hard Set of HallusionBench. which consists human-edited image-question pairs specially crafted elicit hallucinations MLLMs. We note that, in addition to hallucination mitigation, DPA helps MLLMs in reducing Yes/No bias. As discussed earlier, VLa v.5 is prone answering 'Yes', in most cases. Our proposed DPA effectively reduces Yes/No bias from 0.31 0.17 and from 0.38 0.20 HALVAn and HALVA respectively. Morcover, in the case of HALVA the Yes/No bias is reduced from 0.19 to 0.02, with 0 being ideal.
C.8 A CRITICAL ANALYSIS OF OUR PROPOSED DPA
Here, we critically assess whether the performance enhancement observed our proposed DPA attributable to generative data augmentation, the proposed training objective, or their combination. To investigate this, we apply generative data augmentation directly another finetuning-basel hallucination mitigation approach, HA-DPO (Zhao et al., 2023b). In HA-DPO, correct and hallucinated pairs are employed to finetune MI.MM.
24

---

Published as a conference paper at ICLR 2025
Table S7: Detailed results on HallusionBench.
Yes/No Blas
Question Pair
Fig.
Easy Acc.
Hard Acc.
Acc.
Method
Pct. Diff (~~ D)
FP Ratio (~ 0.5)
(gdec)) 1
(fcce)
(Easy adce)
(Hard adce)
(aAce)
LLaVA-v1.5m
0.31 Loce 0.26
0.79 8,00
10.70
19.65
42.34 1o:13
41,47
47.09 1o.ss
HA-DPOys
0.76
11.21
19.08
42.86
44.19
48.36
EOS7s
0.29
0,78
11,21
18.50
43,96
42,09
48.72
HALVApu (Ours)
0.17
0.67=n000
13.85
48
42.71
45.8 45.81
48.95
LLaVA-v1.55mm
0,38
0.85
8.79 8.79
15 15.22:u.17
44.25
35.97
46.50
HAIVAga (Ours)
0,20
0.70=0.00
13.85
20.1320.17
44.47
42.87
49.
05
VILA-v1.5mmmn
0.19 Louce
0.71 .0.00
18.90 10:00
24.86_n.25
$2.38
16.20
55.39 15.39 10.05
HALVApaoss (Ours)
0.02
0,53
22.71 2v.0s
27.65
52.8920.4
46,96a0.25
56,60.0.as
aiming to maximize the reward margin between the correct responses and the hallucinated ones. Accordingly, we train HA-DPO by replacing their data with the output of our generative data augmentation module. We utilize the official code released by (Zhao et al., 2023b) and conduct learning rate) ensure effective training. Subsequently, we evaluate the performance of the newly trained AADD on both hallucination (CHAIR, AMBER, MME-Hall) and non- halllucccinaionn (MME) benchmarks. The results presented in Table S8 indicate that applying our proposed generative data augmentation to HA-DPO does not yield the same level of performance boost as HALVA. This confirms that the performance boost of proposed method stems from combination of the KL-regularized phrase-level alignment objective and the data augmentation setup. Note that since our proposed method necessitates a pair of aligned correct and hallucinated phrases, and the descriptive responses utilized in HA-DPO do not meet this requirement, we are unable to apply DPA directly to their data.
tuning (mainly with varying
Table S8: Effect of generative data augmentation HA-DPO. Here, CHAIR, AMBER, and MME- Hall are ballucination benchmarks, and MME is a general vision- Asionnlanguaae benchmark.
CHAIR (C,) J AMBER F1 MME-Hall MME
HA-DPOOB
11.0
78.1
618.3
1502.6
HA-DPOyR w/
14.6
77.7
631.7
1508.9
Generative Data Aug.
HALVA
11.7
83.4
665.0
1527.0
C.9 RESULTS ON POPE
addition to the hallucination benchmarks in main paper, we also evaluate HALVA using POPE (Li al., 2023d). While POPE is used in prior works, we note few key limitations and find it to be not well suited benchmark for evaluating MLLMs, as listed below. Please note that the similar concerns are also echoed in recent works (Wang al., 2023b; Bai et al., 2024).
First, POPE employs Yes-or-No protocol to check for existence of object, but lacks coverage of other types of object hallucinations, such as object attributes (e.g., color, count) and object relations (e.g., position, environment). Second, the questions are formulated based on only 500 images and include total of 79 unique objects. which fails to capture object hallucinations across diverse visual concepts. Third, POPE does not evaluate hallucinations in descriptive tasks (e.g., image description), where MLLMs tend to hallucinate more. These limitations led to introduction of more comprehensive benchmarks such AMBER and MME among others, which we are used as the primary evaluation benchmarks in this work.
As shown in Table S9, we observe that while models such as GPT-4o and InternVL22 perform considerably better than others on MME and HallusionBench. they are not well-represented by POPE. Despite these shortcomings. able obtain 87.1 and 87.9 97.9 for HALVAT and HALVA using different 0.005.
C. RESULTS ON LINGUISTIC QUALITY
To analyse whether DPA training have an adverse affect on the linguistic quality of the responses generated by MLLMs, we evaluate the responses on four aspects: grammatical correctness, fluency, detailedness, and choice of words. Since there is standard or commonly used benchmark for these tasks, we use randomly selected 100 detailed image descriptions (a subset from the AMBER (Wang al., 2023b) image description task) generated
25

---

Published as a conference paper at ICLR 2025
Table S9: The results on POPE presented. Results are obtained using a different than our default. Added here for reference only, and should not be directly compared with 7B and 13 models, due to the large discrepancy their model sizes.
POPE (FF f>
AMBER
HallusionBench (Acc. 1)
MME-Hall (Score f)
MME (Score f)
Method
(FI f)
LLaVA-v1.5p
85,9
74.7
47.1
684.3
1510.7
LLaVA-RLHFm
81.5
76.3
43.0
493.3
1190.0
HA-DPOTp
86.9
78.1
48.4
15U2.6
EOS
86.0
75.6
48.7
606.7
1424.4
HALVArp (Ours)
84. $4.8/87.1*
83.4
49.0
665.0
1527.0
LLaVA-vI.51p
85.9
73.1
46.5
643.3
1530.1
LLaVA-RLHF
81.9
83.7
46.4
585.0
1367.7
HALVAr (Ours)
84, $4.9/87.9*
86.5
49.1
675.0
1544.0
VILA-v1.51a HALVArmasa (Ours)
82.2
1569.6
87.9
56.6
691.7
1575.7
GPT-4o' (v.0513, dctail-high)
85.6
55.0
2310.3
Intern VL2 (Chen al., 2024)
81.9
56.5
2293.1
by LLaVA 1.5TB and HALVA7.. with GPT-4o-mini as the judge to rate them on a scale of to 10. The template used in evaluation is presented in Figure S6. As shown in Table S10, HALVA7и exhibits the same performance LLaVA 1.57g.
Table S10: Results linguistic qualities of the responses.
Model
Grammatical Correctness
Fluency
Detailedness
Choice of Words
ILLVVA 1,5mB HAL (Ours)
9.90 0.30
9.64 + 0.52 9.51 + 0.50
8.37 + 0.48
8.93 + 0.26
9.99 0.10
8.35 0.48
8.99 + 0.23
Following a detailed image description.
Your task is to assess the response on the following criteria: 1. Grammatical Correctness: Analyze the response for granmar, punctuation, and syntax acsuracy.
2. Fluency: Evaluate whether the response flows smoothly, reads naturally, and maintains coherence throughout.
3. Detailedness: Check if the respense provides sufficient and relevant detail to address the topic comprehensively, without redundancy unnecessary informaticn.
4. Choice Words: Assess if the words used are appropriate, varied, and effectively convey the intended message.
Rate each criterion
scale from 0 to 10, where 0 indicates poor quality and 10 signifies an excellent response.
Here is the image description to evaluate:
(description)
Your respense should be in this format:
Grammatical Correctness: SCORE
Fluency: SCORE
Detailedness: SCORE
Choice of Words: SCORE
Figure S6: The template for evaluating the linguistic quality of the responses.
26

---

Published as a conference paper at ICLR 2025
Table S11: Analysing if the base models are better than the hallucination mitigation methods on general tasks. Here Model is a base model and Model 2 is hallucination mitigation method. Red: performance drop with statistical significance.
Evaluation benchmark
Model 1
Model 2
Adjusted A
Standard Error
# samples
LL HA-DPOg
VQAv2
0,7850
0.7760
0.0090
0.0065
0.0013
107394
MMVet
0.3110
0.3070
0.0040
-0.0574
0.0314
218
TextVQA
0,5820
0.5670
0.0150
0.0013
0.0070
5000
MME
0.7554
0.7513
0.0041
-0.0143
0.0093
2114
LaVA-Bench--n--
0.6540
0.6620
-0.008
-0.1284
0.0614
60
LL vs. EOSBB
VQAv2
0,7850
0.7760
0.0090
0,0065
0.0013
107394
Ve
0.3110
0.3140
+0,0030
-0.0644
0.0314
218
Tex(VQA
0.5820
0.5520
0.0300
0.0163
0.0070
5000
MME
0.7554
0.7122
0.0432
0.0248
0.0093
2114
LLaVA--ench-in-tte--iii
0.6540
0.6580
40.0040
-0.1244
0.0614
LLa LL vs. HALVAp (Ours)
VQAv2
0.7850
0.7850
0.0000
-0.0025
0.0013
107394
MM
0.3110
0.3210
40,0100
-0.0714
0.0314
218
TexIVQA
0.5820
0.5820
0.0000
-0.0137
0.0070
5000
MME
0.7554
0.7635
-0.0081
-0.0265
0.0093
2114
LLaVA-Benan Beneh-in-tte---eeeeeeeeeeeeeeeee
0.6540
0.6720
-0.0180
-0.1384
0.0614
LLa vs. HALVA1 (Ours)
VQAv2
0.8000
0.8000
0:0000
-0.0024
0.0012
107394
MMVet
0,3610
0.3780
-0.0170
-0.0808
0.0325
218
TexIVQA
0,6120
0.6120
0.0000
-0.0135
0.0069
5000
MME
0.7651
0.7720
-0.0070
-0,0250
0.0092
2114
LLaVA-Benche tettiiin-thweeee wild
0.7250
0.7270
-0.0220
-0.1150
0.0576
60
VILA-v1.5 1383344 vs. HALVA13HA3 (Ours)
VQAv2
0.8280
0.8280
0.0000
-0.0233
0.0012
107394
MMVet
0.4430
0.4430
0.0000
-0.0659
0.0336
218
TextVQA
0.6500
0.6480
0.0020
-0.0112
0.0067
5000
MME
0.7848
0.7879
-0.0031
-0,0206
0.0089
2114
TiVA--Bncc--i-ttie wild
0,8080
0.8240
-0.0160
-0.1157
0.0508
60
C.11 STATISTICAL ANALYSIS
We accept the performance improvement or drop in Model compared to Model 2 on given task to be statistically significant if the Adjusted A is greater than 0, where A is the performance difference between Model and Model 2. We consider the Standard Error (SE) with statistical significance 95% confidence. Note that the original results are scaled between mathematical expressions are given helow
where represents the worst represents the best. The
Model x (1 Model 1)
SE
number samples
A Model Model 2,
Adjusted A 1.96 SE.
The results are presented in Tables S11 to S14 show that the existing finetuning- hallucination mitigation methods such as HA-DPO and EOS show statistically significant performance drops general tasks. In contrast, proposed DPA does not exhibit such deterioration. Moreover, observe that HALVAn shows statistically significant improvements CHAIR, AMBER generative, and AMBER discriminative tasks. The same holds true for HA-DPOts and BOS Both our 13B variants (HALVAE and HALVA13894) show improvements across all setups, with the improvements on AMBER discriminative tasks heing statistically significant. Unlike DPA, existing methods, such HA-DPO and EOS, exhibit performance deterioration in 2 out of hallucination tasks compared the base model, where the performance drop for EOS MME-Hall statistically significant.
27

---

Published as a conference paper at ICLR 2025
Table S12: Analysing if hallucination mitigation methods better than the base models general lasks. Here Model is a hallucination mitigation method and Model 2 is base model.
Evaluation benchmark
Model
Model
Adjusted
Standard Error # samples
HA-DPOTB vs. L.La VA-.VA-11.55p
VQAv2
0.7760
0.7850
+0.0090
-0.0115
0.0013
107394
MMVet
0,3070
0.3110
-0.0040
-0,0652
0.0312
218
TextVQA
0,5670
0.5820
-0.0150
-0.0287
0.0070
5000
MME
0.7513
0.7554
-0.0041
-0,0225
0.0094
2114
LLaVA- mench-in-the-weee
0.6620
0.6540
0,008
-0.1117
0.0611
60
EOS vs. VLv
VQAv2
0,7760
0.7850
-0.0090
-0.0115
0.0013
107394
MMVet
0,3140
0,3110
0.0030
-0.0586
0.0314
218
TextVQA
0,5520
0.5820
-0.0300
-0.0438
0.0070
5000
MME
0.7122
0.7554
-0.0432
-0.0624
0.0098
211 2114
LLVA-Benchh-i--tee -hhee--i-tthe-wldd
0.6580
0.6540
0.0040
-0.1160
0.0612
60
HALVAc (Ours) vs. VA-v1511.
VQAv2
0.7850
0.7850
0.0000
-0.0025
0.0013
107394
MMVet
0,3210
0,3110
0.0100
-0.0520
0.0316
218
TextVQA
0,5820
0,5820
0.0000
-0.0137
0.007
5000
MME
0.7635
0.7554
0.0081
-0.0100
0.0092
2114
0.6720
0.6540
0.0180
-0.1008
0.0606
60
HALVA (Ours) LLa VLvV.--1.55
VQAv2
0,8000
0.8000
0.0000
-0.0024
0.0012
107394
MMVet
0,3780
0.3610
0.0170
-0.0474
0.0328
TextVQA
0.6120
0.6120
0.0000
-0.0135
0.0069
5000
MME
0.7720
0.7651
0.0070
-0.0109
0.0091
21 2114
-ivA-Beech-in--the-wild
0.7270
0.7250
0.002
-0.1107
0.0575
60
HALVA 138/344 (Ours) vs. VILA-vl..5
VQAv2
0.8280
0.8280
0.0000
-0.0023
0.0012
107394
MMVet
0.4430
0.4430
0.0000
-0.0659
0.0336
218
TextVQA
0.6480
0.6500
-0.0020
-0.0152
0.0068
5000
MME
0.7879
0.7848
0.0031
-0.0144
0.0089
2114
LLVA--Bemch-in-thh-willl
0.8240
0.8080
0.0160
-0.0804
0.0492
60
28

---

Published as a conference paper at ICLR 2025
Table S13: Analysing if hallucination mitigation methods are better than the base models on halluci- nation tasks. Here Model a hallucination mitigation method and Model is base model. Green: performance improvement with statistical significance.
Evaluation benchmark
Model
Model 2
A
Adjusted Standard Error
samples
HA-DPOqs vs. LLaVA-v1.5t
CHAIR
0.6180
0.5000
0.1180
0.0754
0.0211
500
MME-Hall
0.7729
0.8104
-0.0375
-0.0905
0.027
240
AMBER-Generaine (Hall. Rate)
0.6910
0.6360
0.0550
0.0264
0.0146
1004
AMBBRR-Diseriminai
0.7810
0.7470
0.0340
0.0272
0.0035
14216
AlMHal-Bench (Hall. Rate)
0.4000
0.4600
-0.0600
-0.1580
0.0500
96
Hallusion-Bees Bench
0,4836
0.4709
0.0127
-0.0165
0.0149
1129
BOSm LLa
CHAIR
0.5980
0.5000
0.0980
0.0550
0.0219
500
MME-Hall
0.7584
0.8104
-0.0520
-0.1062
0.0276
240
AMBER-Generativ (Hall. Rate)
0.7730
0.6360
0.1370
0.1111
0.0132
1004
Diser-Disermminntive
0.7560
0.7470
0.0090
0.0019
0.0036
14216
MMHal-Benne (Hall. Rate)
0.4100
0.4600
-0.0500
4.. 0.1484
0.0502
96
Hallusion-Bencr Bench
0.4872
0.4709
0.0163
+0.0129
0.0149
1129
HALVApB (Ours) vs. LLaVA-v1.5mm
CHAIR
0.5860
0.5000
0,0860
0.0428
0.0220
500
MME-Hall
0.8313
0.8104
0.0209
+0.0265
0.0242
240
(Hall, Rate)
0.6780
0.6360
0.0420
0.0131
0.0147
1004
0.8340
0.7470
0.087
0.0809
0.0031
14216
MMHal-Bened (Hall. Rate)
0.4600
0.4600
0.0000
+0.0977
0.0509
96
Hallusion- Bench
0.4895
0,4709
+0.0106
0.0149
13 (Ours) vs. LLL
CHAIR
0.5460
0.5280
0.0180
-0.2266
0.0223
500
MME-Hall
0.8438
0.8041
0.0396
+0.0063
0.0234
240
-eBER-GGeeretivee (Hall, Rate)
0.6960
0.695
0.0010
+0.0275
0.0145
AMBERR-iscrimingg
0.8650
0.7310
0.1340
0.1284
0.0029
14216
MMHal-Berne (Hall, Rate)
0.5500
0.5000
0.0500
+0.0495
0.0508
Hallusion- Bench
0.4910
0.4650
0.0260
+0.0322
0.0149
1129
HALVA13W38 (Ours) vs. VII.A-vI.5 p3wa24
CHAIR
0.7000
0.6700
0.0300
+0.0102
0.0205
500
MME-Hall
0.8646
0.8604
0.0043
+0.3900
0.0221
240
AMBER-Gencrativ (Hall. Rate)
0.4580
0.4390
0.0190
-0.0118
0.0157
1004
AMBER-Disee SiBER-Discriminatiee
0.8790
0.8220
0.0570
0.0516
0.0027
14216
MMHal-Benedl Bench (Hall. Rate)
0.5500
0.5400
0.0100
-0.0895
0.0508
96
Hallusion Bench
0.5660
0.5539
0.0121
+0.0168
0.0148
1129
29

---

Published as a conference paper at ICLR 2025
Table S14: Analysing if base models are better than the hallucination mitigation methods on halluci- nation tasks. Here Model is a base model and Model 2 is a hallucination mitigation method. Red: performance drop with statistical significance.
Evaluation benchmark
Model
Model 2
A
Adjusted A
Standard Error
# samples
LLaVA-v1.5yp vs. HA-DPOO
CHAIR
0.5000
0.6180
-0.1180
-0.1618
0.0224
500
MME-Hall
0.8104
0.7729
0,0375
-0.0121
0.0253
240
AMBER AMBER-Generativ (Hall. Rate)
0.6360
0.6910
-0.0550
-0.0848
0.0152
1004
0.7470
0.7810
-0.0340
-0.0411
0.0036
14216
MMHal-Benal (Hall. Rate)
0.4600
0.400
0.0600
+0.0397
0.0509
96
Hallusion- Bencc
0.4709
0.4836
-0.0127
+0.0418
0.0149
1129
LLVAA-v1.57 vs. EOST
CHAIR
0.5000
0.5980
-0.0980
00.1418
0.0224
500
MME-Hall
0.8104
0.7584
0.0520
0.0024
0.0253
240
AMBER AMBER-Generative (Hall. Rate)
0.6360
0.7730
-0.1370
00.1668
0.0152
1004
AiBER-Discriminative
0.7470
0.7560
-0.0090
+0.0161
0.0036
14216
MMHal-Benn (Hall. Rate)
0.4600
0,4100
+0.0497
0.0509
Hallusion- Benhhh
0.4709
0,4872
-0.0163
+0.4554
0.0149
LLaVA-v1.5m vs. HALVA (Ours)
CHAIR
0.5000
0.5860
-0.0600
0.12998
0.0224
500
MME-Hall
0.8104
0.8313
-0.0209
+0.0705
0.0253
240
GenerrGGene (Hall, Rate)
0.6360
0,6780
-0.0420
+0.0718
0.0152
AMBER-Discrining
0.7470
0.8340
-0.0870
-0.0941
0.0036
14216
MMHal-Bench (Hall. Ratc)
0.4600
0.4600
0.0000
-0.0997
0.0509
Hallusion-
0.4709
0.4895
-0.0186
-0.0477
0.0149
1129
VA-v..155 vs. HALVA (Ours)
CHAIR
0.5280
0,5460
-0.0180
0.0618
0.0223
500
MMF:-Hall
0.8041
0.8438
+0.0396
+0.0898
0.0256
240
AMBER-GGneratte (Hall. Rate)
0.6950
0.6960
-0.0010
-0.0295
0.0145
1004
AMBER- DoBER-Disermiintiive
0.7310
0,8650
-0.1340
-0.1413
0.0037
14216
MMHal-Bench (Hall. Rate)
0.5000
0.5500
-0.0500
-0.1500
0.0510
96
Hallusion-Bench
0.4650
0.4910
-0.0260
+0.0551
0.0148
1129
VILA-v1.5 HALVA
(Ours)
CHAIR
0.6700
0.7000
-0.0300
0.0712
0.0210
500
MMF-Hall
0.8604
0.8646
-0.0043
4.0481
0.0224
240
(Hall. Rate)
0.4390
0.4580
-0.0190
-0.0497
0.0157
1004
AMBEER-iscrimiiatie
0.8220
0.8790
-0.0570
-0.0633
0.0032
14216
MMHal-Bench (Hall. Ratc)
0.5400
0.5500
-0.0100
-0.1097
0.0509
96
Hallusion-Bench
0.5539
0.5660
-0.0121
-0,011
0.0148
1129
30

---

Published as a conference paper at ICLR 2025
D IMPLEMENTATION DETAILS
D.1 TRAINING HYPERPARAMETERS
The details of training hyperparameters used in DPA training is presented in Table S15,
Table S15: Details of training hyperparameters used in DPA training.
HALVArB
HALVA
HALVA 138334
Base model
LLaVA-v1.55 1,57
LLaVA-v 1.
VILA-vI.5
LLM
Vicuna-v1.s
Vicuna-vI.55
Vision encoder
CLIP ViT-Lg36/14
SigL.I-LL440MM
Trainable module
LoRA in LLM and everything clse is kept frozen
LoRA sctup (Hu ct al., 2021)
rank=128, alpha=256
Learning rate
5c-6
2.5c-5
Learning rate scheduler
Cosine
Optimizer
AdamW (Loshchilov & Hutter, 2017)
Weight decay
0.
Warmup ratio
0.03
Epoch
(342 steps)
Batch size per GPU
Batch size (total)
64
(loss coefficient)
0.2 7cro stage (Ren al., 2021; Rajbhandari al., 2021) 1.5
0.4
0.5
Memory optimization
Training time
hrs.
hrs.
D.2 LICENSES OF EXISTING ASSETS
For images, we use publicly- available Visual Genome dataset (Krishna et al., 2017). This dataset can be downloaded from hst / //homsso.... CS washi ngtton www.onnooooooooonooooooo.oo edu/-ranjar jay/viiiii.. sual genome/ /appp...... html and is licensed under Creative Commons Attribution 4.0 International License.
For the base MLLM, publicly available and its Apache license can be found at https: // /////tt rithub. com/ com/baot wwwt ian- liu/ LLaVA/bicb www.mmaiimmaaaae LICNNS. VILA-v1.5 is publicly available and its Apache license 2.0 can be found at https:tpsa / git.t..thhhhe.eg.e cow/Nooal/ssaa/coooooo.a LA.bll http////bbb/mmmmm/////////aaaaaa CENSE. The weights used in this work are
LLaVA-v1.5 (Liu al., 2023c) and VILA-v1.5 (Lin al., 2024). LLaVA-v1.5
available as follows:
LLaVA-v1.5gg http://hwws.gggggoeeeeeee////aaa
LLaVA-vI.Syar http:///ws.hugggaee...////oooola
VILA-v1.5:: http:////wccp/..oo........o///ea 5-13bb
D.3 GENERATIVE DATA AUGMENTATION SETUP
We present the prompt templates that are used to prepare correct and hallucinated descriptions in Figures $7, S8, and S9. The full list of instructions used in generating image descriptions is presented in Figure S10. We leverage Gemini Vision Pro (gemini nim -1.-1.. O-pro-vvisiie depicting the pipeline generating correct descriptions, closed-set hallucinated descriptions, and open-set descriptions are presented in Figures $11, $12, and $13. We present additional examples of training samples for one sentence image caption, short image description, detailed image description, and Yes-or-No questions in Figures $14, S15, S16, and $17, respectively.
in preparing the responses. Complete examples
31

---

Published as a conference paper at ICLR 2025
# Input
## Image
<Image>
## Text
Here are the region descriptions of the given image.
<Region description 1> <Region description 2>
<Region description 3>
The descriptions the ground truth information for the image.
Based on the given region descriptions,
write response for the following question.
Question:
cinstruction>
The respense must correct and has strong readability.
Do NOT add any new information or additional details. # #Output
<Correct description>
Figure S7: The template for generating the correct image descriptions.
32

---

Published as a conference paper at ICLR 2025
Input
## Text
The given text is description of image. <Correct description>
Please rewrite the given text by replacing the mentioned words with those from the given options.
Please choose the replacement that sounds the most apprepriate.
Replace the word: <ground-truth object 1> with word from the given options: <list of hallueinated objects 1> Replace the word: kground-truth object 2> with word from the given options: <list of hallucinated objects 2>
The description should cogically make sense, the style of the new text should be the same as the original text, and has strong readability.
Please make sure to NOT include the following words the description: <list of ground-truth objects>.
Your respense should only include the new description and nothing else.
Output
<Hallucinated description>
Figure S8: The template for generating the closed-set hallucinated descriptions.
33

---

Published as a conference paper at ICLR 2025
# Input
## Text
The given text is description of an image. <Correct description>
Please rewrite the given text by replacing the mentioned object with ancther object of similar types or categories.
For example, an animal can be replaced with another animal
type of vehicle vehicle and so on.
be replaced by another type
The description should logically makes sense, the style of the new text should be the same as the original text, and has strong readability.
Your respense should only include the new description and nothing else.
The follewing objects need to be replaced:
<list of ground-truth objects>.
# Output
<Hsllucinated description>
Figure S9: The template generating open-set hallucinated descriptions.
34

---

Published as a conference paper at ICLR 2025
# Instructions for one sentence caption:
Provide one-sentence caption for the provided image.
# Instructions for short description:
Describe the image concisely.
Provide a brief description of the given image. Offer a succinct explanation of the picture presented. Summarize the visual content of the image.
Give a short and clear explanation of the subsequent image.
Share concise interpretation of the image provided.
Present a compact description of the photo's s key features.
Relay a brief, clear account of the picture shown. Render clear and concise summary of the photo.
Write terse but informative surmary of the picture. Create compact narrative representing the image presented. Please provide a short description of this image.
# Instructions for detailed description:
Give an elaborate explanation of the image you see. Share comprehensive rundown of the presented image. Offer thorough analysis the image. Provide a detailed description of the given image.
Explain the various aspects of the image before you. Clarify the contents of the displayed image with great detail. Characterize the image using a well-detailed descriptien.
Break down the elements of the image in detailed manner.
Walk through the important details of the image.
Portray the image with rich, descriptive narrative.
Narrate the contents of the image with precision.
Analyze the image in comprehensive and detailed manner.
Illustrate the image through descriptive explanation.
Examine the image closely and share its details.
Write an exhaustive depiction of the given image. Write detailed description of the given inage.
Figure S10: Instructions for different types of image descriptions. These instructions are directly taken from (Liu al., 2024). list them here for the sake of completeness.
35

---

Published as a conference paper at ICLR 2025
# Input
## Image
## Text
Here are the region descriptions the given image.
man jumping on a skateboard skate park with colorful graffiti mountains man in white shirt and blue jeans skating ramp with graffiti markings
white clcuds
Skateboarder making a jump
45 more
The descriptions are the ground truth information for the image.
Based the given region descriptiens, write response for the following question.
Question: Walk through the important details of the image.
The respense must be correct and has strong readabllity.
Do NOT add any new information or additional details.
# Output
young man in white shirt and blue jeans skateboarding in skate park. The skate park has colorful graffiti on the walls and ramps. There are stairs and a fence behind the skate park. There are also some trees and buildings in the background. The sky is blue with clouds.
Figure S11: A complete example of generating the correct image deseriptions.
36

---

Published as a conference paper at ICLR 2025
# Input
## Image
PECOLEy
Fooi
SCOLEA
## Text
The given text is description of an image.
Correct description:
The image shows a variety of fruits on wooden tables at farmer's market. There are pears, apples, oranges, and pineapples. The fruits are arranged in boxes and crates.
There a price sign on some of the boxes.
Please rewrite the given text by replacing the mentioned words with these from the given options.
Please choose the replacement that sounds the most appropriate.
Replace the word: fruit with word from the given optiens: plate, leaf, food, basket, vegetable
Replace the word: apple with a word from the given opticns: table, banana, reot, bowl, shirt
Replace the word: pear with word from the given opticns:
tomato, gummed label, container, writing, hair
The description should logically make sense, the style of the new text should be the same the original text, and has strong readability.
Please make sure to NOT include the following words in the description: apple, box, crate, fruit, ground, letter, line, pear, tree trunk, wall, word.
Your response should only include the new description and nothing else.
Output
The image displays an assortment of vegetables on wooden tables at farmer's s market. There tomatoes, bananas, oranges, and pineapples. The vegetables are organized containers and signs. There is a price tag on some of the containes ners.
Figure S12: A complete example of generating closed set hallucinated descriptions is provided. The image is not fed to Gemini in generating hallucinated response, shown here for reference.
37

---

Published as a conference paper at ICLR 2025
Input
## Image
## Text
The given text is description of an image.
Correct description:
A young man in a white shirt and blue jeans is skateboarding in a skate park. The skate park has colerful graffiti on the walls and ramps. There are stairs and fence behind the skate park. There the background. The sky is blue with some clouds.
also some
and buildings in
Please rewrite the given text by replacing the mentioned object with ancther object of similar types categories. For example,
animal can be replaced with another animal or one type of vehicle can be replaced by another type of vehicle and so on.
The description should logically makes sense, the style the new text should be the same as the original text, and has strong readability.
Your respense should only include the new description and nothing else.
The follewing objects need to be replaced:
advertisement, arm, baseball cap, building, cloud, desiga, distance, fence, head, jean, leaf, man, mountain, park, pole, ramp, shadow, sign, skate park, skateboard, skating, sky, stair, t-shirt, tree, trick, wheel.
# Output
young weman a black dress and red sneakers rollerblading in a roller rink. The roller rink has vibrant murals on the walls and floors. There are benches and railing behind the roller rink. There are also some plants and shops in the background. The ceiling white with some lights.
Figure S13: complete example of generating open-set hallucinated descriptions provided. The image is not fed to Gemini in generating hallucinated response, shown here for reference.
38

---

Published as a conference paper at ICLR 2025
GOCTURTE PAERR
CLANS
APWSIT OR CHERRY DA
Un TIEE DINOSAL
Quest ion; Provide cne-sentence caption for the provided image.
Question: Provide one-sentence capt for the provi ded image.
Correct: There are three people holding and using their black smartphones.
The image shows variety of donuts metal shelves in donus shop.
Correct:
Fa !ucinated: There are three people holding using their black tablets.
Hallucinated: The image depicts assortment of cupcakes on wooden: she ves in a cupcake shop.
Question:
Provide one-sentence caption for the provided ded mage.
Question: Please provide short description of this image.
The image shows a blonde woman wearing a pink dress with red bow in her hair.
Correct:
Correct: : man is snowboarding down a snowy slope at night.
mallucinated:: A person is snowboarding down snowy hill at night.
The image displays a blonde woman wearing pink gown with red hat her head.
Hallucinated:
Figure S14 Examples of one sentence image captions used in DPA training.
39

---

Published as a conference paper at ICLR 2025
Question: Write terse but informative aummary the picture.
Question: Provide a brief
description the giver image.
Correct: The image night view of baseball game. There are two baseball playera, one is the batter and the other the catcher.
Correct: A black cat is lying on the ground next to a small brown rd. The cat has its eyes open and is looking at the bird.
Ballucinated: The image night view of hockey game. There are two hockey players, one is the shocter and the other is the goalie.
Lallucinated: black dog is lying on the grass next to small brown lea The dog has its eyes open and is looking the leaf.
Questions Provide brief eeseription of the gi ver image.
Two boys are standing in baseball field. They are wearing baseball fniforms and holding baseball mitts. The boy the left is wearing green and wh white uni form and the boy on the right is wearing a black and white uniform.
Correct:
Summari ze the visual content of the image.
Question:
Correct: A brown and white bulldog is standing on a blue surf bbord in ocean. The bulldog is looking at the camera with overbite. There is a big splash of water in front
surfboard.
Ea Lucinated: Two children are stancing in soccer field. They are wearing soccer uniforms and holding soccer balls. The child on the left is wearing blue and white uni form and the chi Id the right is wearing red and black uni form.
Hallucinated: A gray and white cat standing on a yellow skateboard in the snow. The cat is looking the camera with snaggletooth. There a big pile of snow in front of the skateboard.
Figure S15: Examples of short image descriptions used in DPA training.
40

---

Published as a conference paper at ICLR 2025
Question:
Clari fy the contents of the displayed image with great detail.
Question: iilustrate the image
through descriptive explanation.
There are few notercycles parked in parking lot. There is of the motorcycles. He is looking at the motoreycle, The motorcycle is orange and black, It has chrome exhaust pipe. There are some trees and bui buildings in the background.
Correct:
Correct: A yellow container house is placed on the sidewalk. The house has red and low sign on the front. There are some buckets in front of the house, is squatzing on the sidewalk next to the house. There are green bushes and brick sidewalk.
standing behind
Talluciiateed There are a few trucks parked in parking lot There is person standing behind one of the trucks. He is looking the truck. The truck is orange and black. It has chrome license plate. There are some plants and houses in the backgrouns.
Hallucinated: A yellow trailer placed the grass, home has a blue and yellow flag on the front. There are some barrels in
the home. woman is kneeling the grass next the home There are red flowers and a path.
Figure $16: Examples of detailed image descriptions used in DPA training.
41

---

Published as a conference paper at ICLR 2025
See fering 3al En Vestang a herry rand myy Stary Suisty dear Ser
Just see and its Free!!
Question: Describe the image in detail.
Correct: : woman sitting behind table in park. There is s1 on the table e that says just see the results and its freev The woman wearing blue jean jacket and beaded neck ace. There is stack of pamphlets on the table. The table is covered with red tablecloth. The ground is covered with brown leaves. There is a large tree in the background.
Quest ion: Explain the vari ous aspects of the image before you.
Correct: This image shows a stall, horse standing
The horse has a wh bl blaze its forehead and white Bocks on its back legs. The stall is made cinder blocks and has metal gate. There is a pile of manure in the stall. The horse is standing on dirt. There are green hills background.
Hallucinated: A man is sitting beh nd chair in garden. There is a poster on the chair that says Just see
Hal fallucinatedd This image depicts black cow standing in pen. The cow has bl alak spet on its forshead and black socks on its front legs. The pen is made of wooden planks and has wooden gate. There is pile of hay in the pen. The cow is standing on straw. There are brown hills in the background.
outcome and its frees The man is wearing black leather coat and a golden chain. There is pile of leaflets an the chair, The chair covered with blue sheet. The floor is covered with green grass. There is a tall building in the background.
Figure S16 (Continued): Examples of detailed image descriptions used in DPA training.
42

---

Published as a conference paper at ICLR 2025
Question:
Are there total of nine benches visible in the scene? Please in Yes
Question: Is there nightstard on the left side of the bed? P Please word Yes No.
Correct: No
Correct: Yes
Fall Lucinated: Yes
nallucinated: No
Quest lon: Is there person located on the left side the image? Please answer in one word Yes or No.
Question: Is the cup located on left side of the table? F Flease answer in word Yes or No.
Correct:
Correct: Yes
Eal Fallucinated: Yes
Hallucinated: No
Figure S17: Examples of Yes-or-No questions used in DPA training.
43

---

Published as a conference paper at ICLR 2025
E
QUALITATIVE RESULTS
E.1 QUALITATIVE COMPARISON BETWEEN HALVA AND LLAVA-V1.5
Describe the image in detail.
User
The image features a dirty, old toilet sitting in dirt-covered area, possibly an outhouse or an abandoned building. The toilet is positioned in the center of the scene, surrounded by dirt and debris. There are several carrots scattered around the area, with some close the toilet and others further away. The overall scene appears to be quite unclean and neglected.
The image features a dirty, old toilet sitting in a dirt-covered room. The toilet is positioned in the middle of the room, surrounded by pile of dirt and debris. The walls of the room are made of cinder blocks, giving the space a somewhat unfinished appearance.
Figure S18: Comparing detailed image descriptions between HALVA and its base model LLa v1.5 @ Unlike HALVA, LLaVA-v1.5 hallucinates 'carrot" in the image description. Hallucinations are highlighted in red.
44

---

Published as a conference paper at ICLR 2025
The image captures busy street at night with car driving down the road. The car is posilioned in the middle of the scene and its headlights are on, illurnintting the street. There are several traffic lights visible in the image, with one located near the center and another towards the right side of the scene
In addition to the car. Ihere are two other people in the scene, one standing
Gloser to the left side and the other near the center. handbag can be seen
he ground, possibly belonging one
pedestrians. The overall image bustling and lively, with the car and people
atmosphere
havigating the busy street at night
image captures busy street at night with car driving down the road. The street is filled with various signs, including traffic light and street sign, which are both lit up. The traffic light positioned near the center scene, while the street sign located on the right side.
Describe the image in detail.
There also two other visible the scene, behind the main and another further down the road. The street appears indicated by the presence of foreign signs.
8
foreign country,
User
Figure S19: Comparing detailed image descriptions between HALVA [33] and its base model
VLv [M]] LLaVA-v1.5 wrongfully describes the position of the 'cars' and 'traffic lights'. Moreover, it hallucinates the presence of 'people', 'pedestrian', and 'handbag'. In contrast, HALVA correctly identifies the relative positions of the 'traffic lights' and "cars", while accurately describing the image. Hallucinations are highlighted in red.
45

---

Published as a conference paper at ICLR 2025
Describe the image in detail.
User
The image captures baseball game in progress, with pitcher standing on the pitcher's mound, preparing to throw the ball. The pitcher is wearing red shirt and is in the process of winding up for the pitch.
In the scene, there are several other people present, including batter, catcher, and an umpire. The batter is holding baseball bat, ready to swing the incoming pitch. The catcher is positioned behind the batter, wearing baseball glove, and the umpire standing nearby, observing the game closely
The baseball is visible in the air, close to the pitcher, the pitcher prepares to release it towards the batter
The image captures a baseball pitcher in the middle of pitch on baseball field. The pitcher is wearing red shirt and is in the process throwing ball. is standing on the pitcher's mound, which is a raised area in the center of the field.
The baseball can be seen in the air, close to the pitcher's hand, he releases The pitcher's baseball glove is also visible, positioned near his other hand. The scene is set against a backdrop of lush green field, showcasing the baseball diamond and the pitcher's skillful performance.
Figure S20: Comparing detailed image descriptions between HALVA v1.5 [D]. LLaVA-v1.5 exhibits object hallucination that may happen due object co-occurrence in the training data, such as the occurrence of 'pitcher' with 'batter', 'catcher', and 'umpire'. HALVA effectively mitigates such hallucination and accurately describes the image with necessary details. Hallucinations are highlighted in red.
and its base model LLaVA-
46

---

Published as a conference paper at ICLR 2025
Is there direct contact between the dog and beach?
User
Are there three people in this image?
Yes, the dog is running
m
directly on the beach, with Its paws touching the sand.
User
Yes, there three people in this image.
Yes, the dog is running directly on the beach, with Its paws touching the sand.
Yes, there are three people in this image.
The same image given in separate trial.
image given separate
there direct contact between the dog and sea?
Are there four people in
8
this image?
8
User
User
Yes, the dog is running
the beach near the water which suggests that
Yes, there are four people in
Tis image
direct contact
sea
No, dog directly touching the sea. running on the beach near the water, but there is gap between the dog and the water.
No, there are three people in this image.
Figure S21: HALVA + does not exhibit tendency answer 'Yes" for any questions asked, like LLaVA-v1.5 +
Moreover, HALVA can accurately lay out the details of the image to further support the answer. As presented in the right side example, HALVA answers with "the dog is not directly touching the sea. It is running on the beach near the water, but there is a gap between the dog and the " Hallucinations are highlighted in red.
47

---

Published as a conference paper at ICLR 2025
the top longer than bottom line?
Is the top line shorter than the bottom line?
the top same size the bottom line?
8
8
User
Yes, the top is longer than the bottom line.
Yes, the top line is shorter
Yes, the top line is the same
han the bottom
ADD
bottom
Yes, the line longer than the bottom line.
No. the top line longer
No. the top line longer
than the bottom line.
than the bottom line.
Figure S22: Examples illustrating the effectiveness of our proposed DPA in mitigating hallucinations that arise due to visual illusions. HALVA [ [ does not exhibit tendency to answer 'Yes' to any questions asked, like LLaVA-v1.5 @ Moreover, HALVA exhibits consistency in its responses unlike LLa Hallucinations are highlighted red.
According medal number chart of 2008 beijng olympic, does USA have the most gold metals, followed by and Russia?
Gold
Silver
Bronze
28
United States
36
Russian Fed.
23
28
User
Great Britain
19
15
Germany
16
Australia
14
17
Yes, according to the meta
Korca
13
10
8
humber chart of the 2008
Japan
10
Beijing Olympic Games, the
United States won the most
Italy
gold medals, followed by China and Russia The chart
France
17
shows the number gold, silver, and bronze medals won by each country during the games. The United States won total of 36 gold medals, followed by China with 37 gold medals, and Russia with 23 gold medals.
No, the chart shows that China has the most gold medals, followed by the United States Russia.
Figure S23: Examples illustrating the effectiveness of our proposed DPA in mitigating hallucinations that are beyond object centric. HALVA [$] accurately answers to this chart-based question unlike LLaVA-v1.5 @.. Hallucinations are highlighted in red.
48

---

Published as a conference paper at ICLR 2025
E.2 QUALITATIVE COMPARISON BETWEEN HALVA AND VILA-V1.5
Is there direct contact between the dog and skate?
Is there one knife in this image?
8
User
User
Yes, there is knife the image, which man holding.
1TA
Yes, the dog is standing top of the skateboard, its paws on skateboard.
Yes, the dog is standing on the skateboard. with its paws on the skateboard.
Yes. there is knife the image.
The same image given separate trial.
The same image given in separate trial.
Is there direct contact between the dog and skate?
Are there two knives this image?
8
User
User
Yes, there are two knives in the image. One
Yes, the dog is standing the skateboard which in direct contact with the ground
knife is being held by the man, and the other knife is sticking out of the coconut
No, the dog is not directly touching the ground. is standing on skateboard, which placed the ground.
No, there is only one knife in the image, which the man is holding.
Figure S24: HALVA does not exhibit tendency answer 'Yes' for any questions asked, like VILA-v1.5 t
Moreover, HALVA can accurately lay out the details of the image to further support the answer. As presented in the right side example, HALVA answers with "the dog not directly touching the ground. It is standing on a skateboard, which is placed on the ground.". Hallucinations highlighted in red.
F LIMITATIONS
In this work, we focused on mitigating object hallucinations in MLLMs. However, MLLMs also suffer from other forms of hallucinations that may
due modality misalignment over-reliance language while ignoring other input modalities, among others. While we showed some promising results on generalization to other forms hallucination, a rigorous exploration of those directions is left for future work. Finally, we believe our method may have applications in other areas as well. For example, might be adapted to mitigate bias and harmful language generation, among others. We leave this exploration for future research.
49

---

Published as a conference paper at ICLR 2025
Is there direct contact between the giraffe and grass?
Are there two this image?
User
User
Yes, the giraffe walking through grass, means is direct contact with the grass.
Yes, are two horses this image.
Yes, the giraffe is walking through the grass, which means is in direct contact with the grass.
Yes, there are two horses in this image.
The same image given separate trial.
The same image given in a separate trial.
Is there direct contact between the tree and giraffe?
Are there three horses this image?
8
User
User
Yes, there direct contact between
are three horses image
the giraffe, as the giraffe standing under tree.
No, there direct contact between tree and giraffe. The giraffe standing the grassy field, and tree is located the background.
No, there are two horses in the image.
Figure S25: HALVA does not exhibit tendency answer 'Yes" for any questions asked, like base model VILA-v1.5 Moreover, HALVA exhibit consistency its response unlike VILA- v1.5, shown in the left example, HALVA confirms the presence of two horses in both the time. Hallucinations are highlighted red.
50

---

Published as a conference paper at ICLR 2025
World War Deaths
Soeat
53%
Canera
Alle Hillary
Nilaary daste
carta onabirdd
According to the image. did the United States suffer the greatest number of total deaths in World War II, with China having the second-highest?
8
User
Yes, according to the image, the United
States suffered the greatest number of lotal deaths in World War with China having the second-highest.
No, according to the image, the Soviet Union suffered the greatest number of total deaths in World War with China having the second-highest.
Figure S26: Examples illustrating the effectiveness of our proposed DPA in mitigating hallucinations that are beyond object centric. HALVA ] accurately answers to this chart-based question unlike VILA-v1.5 1.5 ( Hallucinations are highlighted in red.
51


Reviews:
[{"review_id": "98jGzQopnw", "reviewer": "Reviewer_Qf9P", "summary": "This paper proposes Data-augmented Phrase-level Alignment (DPA), a novel loss function designed to reduce object hallucinations in multimodal large language models (MLLMs) while preserving their general vision-language capabilities. By generating pairs of hallucinated and correct responses through data augmentation, DPA trains MLLMs to distinguish hallucinated phrases from correct ones. Experimental results show that MLLMs fine-tuned with DPA achieve significant improvements, reducing hallucination rates and enhancing performance on visual question-answering and image description tasks.", "questions": "Please see the weakness part.", "limitations": "", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "* The DPA loss function is an innovative approach that mitigates object hallucinations by targeting phrase-level distinctions, offering a focused solution for multimodal hallucination issues.\n\n* The data augmentation method is straightforward yet effective, generating hallucinated-correct response pairs that enable the model to learn nuanced differences with minimal complexity.\n\n* DPA demonstrates significant performance gains", "weaknesses": "The core idea of the paper is to generate correct-hallucinated data pairs through data augmentation. However, I have three questions about this process.\n1. In hallucination-related datasets, object hallucination does not frequently occur, raising questions about the validity of replacing every possible object and attribute with hallucinations. Since models seldom generate such hallucinations, this augmentation strategy might introduce excessive \"non-realistic\" hallucination cases, leading to a mismatch between training and real-world distributions, potentially impacting the model's generalization.\n2. The method’s effectiveness may be limited by the diversity of data augmentation. Since hallucinated data generation relies on a finite set of replacements, it may not fully cover the types of hallucinations that could appear in practical applications, limiting the model’s ability to handle unseen hallucinations.\n3. The data augmentation strategy itself lacks independent experimental evaluation. The experiments mainly focus on improvements in model performance across different benchmarks, without assessing the augmentation strategy’s generalization effect and impact on model training stability across tasks.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "YyxyKlO6NN", "reviewer": "Reviewer_EoXH", "summary": "This paper presents a Data-Augmented Phrase-Level Alignment (DPA) approach aimed at reducing object hallucinations in Multimodal Large Language Models (MLLMs). The method centers on generating paired “correct” and “hallucinated” responses using data augmentation, which are then used to train a phrase-level alignment loss that reduces the probability of hallucinated tokens. The authors strive to maintain the model’s overall vision-language capabilities while minimizing hallucinations. Experimental results across multiple benchmarks indicate that DPA effectively mitigates hallucinations and may even improve detailed object coverage in generated descriptions.", "questions": "1. How does DPA perform on other types of hallucinations beyond object hallucination, such as attribute or location hallucinations?\n2. The reported results for some baseline methods, such as VCD, differ from those in the original papers. Did you directly test VCD, or were the results extracted from their papers? If the latter, the comparison may not be entirely fair, as it mixes experimental results with reported findings from other sources.\n3. This paper augments the training data with hallucinated responses by substituting terms in both open-set and closed-set cases. However, simple substitution could potentially impact fluency and grammatical accuracy. Could this approach compromise data quality and, in turn, affect model performance?", "limitations": "", "rating": 6, "confidence": 5, "soundness": 2, "presentation": 3, "contribution": 2, "strengths": "1. This paper proposes an effective alignment method that successfully mitigates object hallucinations, showing improved scores across hallucination benchmarks in both discriminative and generative tasks.\n2. DPA reduces object hallucinations without impacting the overall performance on VQA tasks, achieving comparable or even higher scores on VQA benchmarks.\n3. The paper provides a variety of quantitative results across multiple benchmarks, including both generative and discriminative tasks. This breadth of evaluation provides some evidence of the DPA’s effectiveness in certain contexts.", "weaknesses": "1. The DPA approach offers limited novelty beyond existing finetuning and data augmentation techniques. While phrase-level alignment is applied in a new way here, it basicly builds on existing concepts and does not significantly advance the field of hallucination mitigation research.\n2. Although the results include some competitive baselines, such as HA-DPO and EOS, several relevant and recent methods are omitted. A more comprehensive comparison would strengthen the evaluation. Additionally, some detailed results for LLaVA-13B and VILA are missing, and the selection of methods across different benchmarks lacks consistency.\n3. While the paper asserts that DPA preserves general vision-language capabilities, the supporting evidence is limited. A broader evaluation across diverse benchmarks would help determine whether this approach impacts overall performance.\n4. The authors highlight the limitations of existing methods, noting they “require massive training data.” However, the proposed DPA also introduces additional training requirements, which suggests a tradeoff between efficiency and effectiveness.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "JUsL7yITUh", "reviewer": "Reviewer_mbRw", "summary": "This paper introduces a novel method called Data-augmented Phrase-level Alignment (DPA) to mitigate object hallucinations in multimodal large language models (MLLMs) for vision-language tasks. DPA generates pairs of “hallucinated” and “correct” responses to fine-tune the model, reducing the generation of hallucinated phrases. A KL divergence regularization term is added to retain the model’s general capabilities. Experimental results demonstrate that models trained with DPA exhibit significant improvements in hallucination mitigation and maintain strong performance on general tasks across multiple benchmarks.", "questions": "1. VILA is also based on LLaVA-1.5 SFT. Is DPA equally effective on other architectures, such as Qwen?", "limitations": "", "rating": 8, "confidence": 4, "soundness": 3, "presentation": 4, "contribution": 4, "strengths": "1. The writing of this paper is excellent and very detailed. The experiments are comprehensive, covering most of the popular benchmarks.\n\n2. I really like this paper. Many works that use DPO-like methods to reduce hallucinations experience a decrease in VQA capabilities. The authors identified this issue and proposed a specialized loss to maintain the model’s performance while penalizing hallucinated phrases. Additionally, the authors validated their method separately on non-hallucination benchmarks, such as VQA-v2 and TextVQA, demonstrating its effectiveness. I believe this work makes a significant contribution to reducing hallucinations in MLLMs.", "weaknesses": "1. DPA relies on the quality of the generated “hallucinated-correct” response pairs. If these generated data lack accuracy or diversity, it may affect the model’s training effectiveness and generalization capability.\n2. Although the experimental results demonstrate the effectiveness of DPA, the paper lacks a fine-grained analysis of hallucination types (such as objects, attributes, actions). Such analysis could provide a deeper understanding of the method’s performance across different types of hallucinations.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "WUhMvcMnpF", "reviewer": "Reviewer_WxnC", "summary": "In this paper, the authors propose data-augmented phrase-level alignment to mitigate object hallucination in MLLMs. The method mainly involves the generation of negative hallucinated responses and phrase-level fine-tuning. The hallucination issue of models fine-tuned with the proposed method is alleviated and the general performance is maintained.", "questions": "Minors:\n1. The construction of dataset is not introduced with details. Details in appendix should be at least provided via cross-reference.\n2. Figure 5 right is not illustrated in the main text. I find it not easy to understand the information it's trying to convey.", "limitations": "", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 2, "contribution": 2, "strengths": "1. It is critical to study the issue of hallucination in MLLMs as well as the trade-off phenomenon in the mitigation of hallucination.\n2. The proposed method is simple yet effective.\n3. Extensive experiments verify the effectiveness of the method.", "weaknesses": "1. The motivation to mitigate hallucination at phrase level is not clearly addressed. There is no illustration on why the phrase-level loss can retain the original performance on general multimodal tasks. It seems straightforward that the constraint on the KL divergence can prevent the fine-tuned model from diverging too far. If it is the only reason, the contribution of the method is greatly weakened.\n\n2. The explanation of Figure 2 is likely to be overclaimed. On line 260, it says \"_EOS achieves a slightly lower hallucination rate_\", but the figure shows that the hallucination rate of EOS is around 5.0 and that of HALVA is around 6.5. This difference is noticeable enough to me as the gap of this metric between HALVA and LLaVA-1.5 is similar. Meanwhile, Figure A right demonstrates that HALVA has a much higher F1 score on AMBER, which is natural because neither EOS nor HA-DPO is trained with Yes/No questions.\n\n3. The presentation of results in experiments is inconsistent. The model lists are different in different tables. For instance, EOS-13B is only shown in Table 1, which makes the verification of the effectiveness less convincing.\n\n4. There are other work mitigating hallucinaton with sub-sequence level training [1]. It is recommended to discuss the difference.\n\n[1] Gunjal, Anish et al. “Detecting and Preventing Hallucinations in Large Vision Language Models.” AAAI Conference on Artificial Intelligence (2023).", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)