Paper: HallE-Switch: Rethinking and Controlling Object Existence Hallucinations in Large Vision-Language Models for Detailed Caption (ICLR-2024)

Reviewer comment (Evaluation):
Although it is interesting to argue that not all hallucination is bad, I don't think the authors successfully supported the argument with examples showing when hallucination is beneficial. With that said, more visualizations like example captions may help better explain the hallucination behavior.

Author response:
We added a section A.1 in Appendix page 1 to support our argument around benefits from parametric knowledge which may also be called “hallucination”. In our study, we define "object existence hallucination" to be a phenomenon where a description makes reference to objects that are not present in the image. However, such hallucinations, when properly harnessed, can be regarded as instances of imagination. Human beings frequently use imagination to successfully accomplish tasks, often without even realizing it.

Reviewer comment (Reproducibility):
There could be more specific illustrations on how the training data was generated using GPT-4.

Author response:
From our understanding, the reviewer needs an illustrations for the data generation process in our method. We update Figure 2 in page 6 to show how we use an open vocabulary detection model to filter objects in ground truth, and recreate detailed caption data based on the filtered objects with GPT4. For how to use GPT4 to generate detailed caption data, we followed a method from LLaVA, by using objects, corresponding bounding boxes, and short captions to generate detailed captions.

Reviewer comment (Novelty):
Related work section doesn't really provide any useful information connecting existing work and the proposed work. For example, some references are missing such as https://arxiv.org/pdf/2110.01705.pdf.

Author response:
We rewrite the related work and notably cite the paper you mentioned. We find this paper proposes the that label distribution may influence hallucination in image caption, which is much earlier than POPE [2] and LRV [3]. It reduces caption hallucination by balancing ground truth object labels, and adding detection labels into language models, which is really impressive to raise the idea before LLMs were widely used. Our work looks into another aspect which is the alignment issue - hallucination happened in detail captions may be caused by the perceivable-vision and language misalignment. Our finding does not contradict with this previous work that balances the labels distribution to the hallucination in captions. As we can treat balancing objects as a reduction of parametric knowledge, which makes the imagination not so clear and confident. Therefore, it is really nice to see, our work’s findings and experiments can echo previous works and provide new ways to understand the reason for hallucination.

[1] Biten, Ali Furkan, Lluís Gómez, and Dimosthenis Karatzas. "Let there be a clock on the beach: Reducing object hallucination in image captioning." Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision. 2022.

[2] Li, Yifan, et al. "Evaluating object hallucination in large vision-language models." arXiv preprint arXiv:2305.10355 (2023).

[3] Liu, Fuxiao, et al. "Aligning Large Multi-Modal Model with Robust Instruction Tuning." arXiv preprint arXiv:2306.14565 (2023).

Reviewer comment (Experiments):
The overall story is not very coherent. First, the details of CCEval are not very clearly described. Then, analysis is conducted on two or three methods with some conclusions drawn. However, the observation mentioned in the paper seems not to have a specific relation with the proposed Hallu-Switch method. The technique is also only evaluated on CCEval, but previous benchmarks are discussed and used in this paper. The reviewer would expect more insights or explanations about why Hallu-Switch works.

Author response:
**1. The overall story is not very coherent. First, the details of CCEval are not very clearly described. Then, analysis is conducted on two or three methods with some conclusions drawn. However, the observation mentioned in the paper seems not to have a specific relation with the proposed Hallu-Switch method. The technique is also only evaluated on CCEval, but previous benchmarks are discussed and used in this paper.**

We frame our overall story as defining, analyzing, and addressing the problem.
1. _How defining:_ Benchmarks that are based on VQA cannot reflect object hallucination for detailed caption. Models did a great job in MME, POPE, does not promise detailed captions without hallucination. We propose a new evaluation metric, CCEval, to measure object existence hallucination reliability. 

2. _How defining leads to our analysis:_ Using CCEval, we can evaluate LVLMs by controlling components to find out the cause of hallucination; _How analysis:_ We analysis language decoder, resolution, and data size to conclude that misalignment between perceivable object in image and object caption form parametric knowledge, which is hallucination.

3. _How analysis leads to our approach:_ We can decrease the amount of parametric knowledge within the model to reduce hallucination; _How our approach addresses the problem:_ We use HallE-Switch to control the contextual and parametric knowledge.


**2. Why technique is only evaluated on CCEval, but previous benchmarks are discussed and used in this paper?**

Technique is only evaluated on CCEval because benchmarks based on VQA cannot reflect object hallucination for detailed caption. Models did a great job in MME, POPE (benchmark for evaluating object existence hallucination by using VQA task), does not promise detailed captions without hallucination. Previous benchmarks are compared with CCEval for showing that model performance on VQA-based benchmark is not correlated to performance on CCEval in the analysis part, so that we can only adhere to CCEval when it comes to our technique, which controls hallucination specifically for detailed captions.


**3. The reviewer would expect more insights or explanations about why HallE-Switch works.**

We provide theoretical explanations on why HallE-Switch works in A.9 in Appendix page 5 and 6. In the new version, we updated the section for readability. Here is a short summarization of why it works:

Assuming LLM is good enough to represent an equivalent distribution with Hidden Markov Chain, then there exists an matrix $W$, so that after transferring word embedding $E$ to $WE$, the LLM's originally simulate the text distribution starting with the initial state $\pi$ will turn out to be equivalent to a different initial state. Corresponding to our analysis, the switch method changes the model from a language model with parametric knowledge to a model cutting the parametric knowledge.

Reviewer comment (Experiments):
The study mainly focuses on LLaVA and InstructBLIP and draws some conclusions for large vision-language models. It might be better to study more models to verify the findings.

Author response:
**4. The study mainly focuses on LLaVA and InstructBLIP and draws some conclusions for large vision-language models. It might be better to study more models to verify the findings.**

We mainly focus on LLaVA and InstructBLIP because they provide multiple language decoder and vision encoder checkpoints, as well as the training data, so that we can break down each component easily to control variables. To evaluate other LVLMs, it would require us to train their models with different language decoder, vision encoders with different image resolution. One challenge is that it is not possible to do so without them releasing the data. Another challenge is that we would incur unfairness if we re-train the models. 

We try to find some models which release intermediate checkpoints and have different architecture from LLaVA for comparison. So far, we only find MiniGPT4_v1 provides instruction data. 

| Models        | CHAIR_s           | CHAIR_i | Coverage | Avg. Length | Avg. Object | 
| ------------- |:-------------:| -----:| -----:| -----:| -----:|
| MiniGPT4-7B     | 91.00 | 32.82 | 36.43 | 98.31 | 11.70 | 11.7 |
| MiniGPT4-13B      | 89.00     |  34.28 | 34.66 | 125.25 | 12.63 |
| MiniGPT4-7B w/ ind. | 85.00    |   27.99 | 32.03 | 151.94 | 14.11 |

From the table, we can see MiniGPT4-7B has better performance compared with 13B. This is aligned with our finding that increasing language decoder does not always lead to performance improvement. The reason of this may be that MiniGPT-4 only tunes Q-former using small amount instruction data. Then, we test our finding on the vision and data side - whether aligning objects grounded by vision encoder and objects in training captions can help reduce hallucination? In MiniGPT4-7B w/ ind., we put undetected objects in "[ ]" for training data. The hallucination is reduced and coverage is maintained, which supports our finding.

Reviewer comment (Writing):
There are many typos in sentences that hinder the reading and understanding. The paper needs careful revision to fix these issues:  
- 'We additionally record **and and** balance the average number of objects and the average length of captions across all cases' in the last third paragraph of page 4  
- 'We find **infeasible** to comparing object hallucinations is impractical when there is a significant disparity in average sentence length and the number of objects.' in the last fourth paragraph of page 4  
- Table 4, the second column for CCEval should be 'finetuning data' rather than 'model'  
- 'The learned M can be regarded as the transformation from a generic word space to the object sensitive word space' in the first paragraph of Sec. 3.2. It seems this describes $W$ rather than $M$

Author response:
**Thanks for your feedback! We have fixed the typos and double checked the paper again. We also updated the related work section to discuss your suggestion on additional LVLMs. Here, we address the major concerns as following:**

Reviewer comment (Novelty):
Small issue that does not affect the rating. Some LVLMs can also be discussed:  
- Chatspot: Bootstrapping multimodal llms via precise referring instruction tuning  
- GPT4RoI: Instruction Tuning Large Language Model on Region-of-Interest  
- MultiModal-GPT: A Vision and Language Model for Dialogue with Humans

Author response:
**Thanks for your feedback! We have fixed the typos and double checked the paper again. We also updated the related work section to discuss your suggestion on additional LVLMs. Here, we address the major concerns as following:**

Reviewer comment (Experiments):
The paper mainly discusses LLaVA and InstructBLIP; what if more models are analyzed? Do these findings still hold somehow?

Author response:
**4. The study mainly focuses on LLaVA and InstructBLIP and draws some conclusions for large vision-language models. It might be better to study more models to verify the findings.**

We mainly focus on LLaVA and InstructBLIP because they provide multiple language decoder and vision encoder checkpoints, as well as the training data, so that we can break down each component easily to control variables. To evaluate other LVLMs, it would require us to train their models with different language decoder, vision encoders with different image resolution. One challenge is that it is not possible to do so without them releasing the data. Another challenge is that we would incur unfairness if we re-train the models. 

We try to find some models which release intermediate checkpoints and have different architecture from LLaVA for comparison. So far, we only find MiniGPT4_v1 provides instruction data. 

| Models        | CHAIR_s           | CHAIR_i | Coverage | Avg. Length | Avg. Object | 
| ------------- |:-------------:| -----:| -----:| -----:| -----:|
| MiniGPT4-7B     | 91.00 | 32.82 | 36.43 | 98.31 | 11.70 | 11.7 |
| MiniGPT4-13B      | 89.00     |  34.28 | 34.66 | 125.25 | 12.63 |
| MiniGPT4-7B w/ ind. | 85.00    |   27.99 | 32.03 | 151.94 | 14.11 |

From the table, we can see MiniGPT4-7B has better performance compared with 13B. This is aligned with our finding that increasing language decoder does not always lead to performance improvement. The reason of this may be that MiniGPT-4 only tunes Q-former using small amount instruction data. Then, we test our finding on the vision and data side - whether aligning objects grounded by vision encoder and objects in training captions can help reduce hallucination? In MiniGPT4-7B w/ ind., we put undetected objects in "[ ]" for training data. The hallucination is reduced and coverage is maintained, which supports our finding.

Reviewer comment (Novelty):
Some related methods have not been reviewed, such as ``Evaluation and Analysis of Hallucination in Large Vision-Language Models''

Author response:
Thanks for your feedback! We updated the related work section to reflect your suggestions. HaELM is a LLM-based hallucination evaluation framework, which uses LLM trained on human annotations, and can evaluate a hallucination in image captions, with low cost. This work can make our CCEval  GPT-4 free. I think this work considers realistic things for academic researchers. In the future, we will consider to follow this work to decrease the cost of CCEval.

Reviewer comment (Presentation):
Presentation needs polishing. The paper is very dense in text and requires some effort for reading. Also, please address the points in "Questions" section.

Author response:
**- Presentation needs polishing. The paper is very dense in text and requires some effort for reading. Also, please address the points in "Questions" section.**

Please refer to the Questions section.

Reviewer comment (Presentation):
The first part and second part of the paper looks more like two separate and condensed papers. Due to this problem, especially the first part fails to deepen our insight about the root cause of the problem. I would expect a deeper treatment on each part.

Author response:
**- The first part and second part of the paper looks more like two separate and condensed papers. Due to this problem, especially the first part fails to deepen our insight about the root cause of the problem. I would expect a deeper treatment on each part.**

We refer the reviewer to revised Section 2 Conclusion in page 6 where it summarizes the entire first part and proposes an explanation on the root cause of the problem. We also revised Section 3 Intro in page 6 to better connect with Section 2. In short, the root cause is the misalignment between perceivable vision and language data, developing contextual and parametric knowledge, which is hallucination. We also refer the reviewer to our first reply for reviewer DU9c for a run through of the overall story.

Reviewer comment (Evaluation):
Results are mostly quantitative. It would be better to show more qualitative examples of hallucination.

Author response:
**- Results are mostly quantitative. It would be better to show more qualitative examples of hallucination.**

We add a section A.8 in Appendix page 5 to show the caption differences with different switch values on 5 images. The qualitative results align with the quantitative ones.

Reviewer comment (Writing):
Section 3.2 is not very clear to me. Does W correspond to "Projector" in Fig 2? According to Fig 2, W comes after LLM. However, equation says the opposite. Is the epsilon parameter applied on word-by-word basis or sentence-by-sentence basis? I may have misunderstood something because I'm not familiar with the LM-Switch work. Regardless, I believe a good paper should be self-contained and can be readable to general audience in the community.

Author response:
**2. Section 3.2 is not very clear to me. Does W correspond to "Projector" in Fig 2? According to Fig 2, W comes after LLM. However, equation says the opposite. Is the epsilon parameter applied on word-by-word basis or sentence-by-sentence basis?**

To explain, $W$ is a projector and it is inserted between LLM backbone and LLM head. Therefore, we have to pass the projector into $M$, where $M$ is LLM backbone + head, resulted in the equation $M’ = M(eps*W)$. We update the Figure 2 and Section 3.2 in page 6 and 7. In the new version, we separately represent LLM backbone as $B$ and LLM head as $H$ in both equation and figure, so it should be clearer. 

The epsilon parameter is the same value for every word in one caption and multiplied with the projector W. Therefore, the answer is word-by-word basis since the model processes each word, but it is the same for every word in one caption.

Reviewer comment (Presentation):
In page 2, object relationship hallucination is mentioned but this concept does not seem to appear again later pages, in metrics or methods presented in the paper. Did I misunderstood?

Author response:
**3. In page 2, object relationship hallucination is mentioned but this concept does not seem to appear again later pages, in metrics or methods presented in the paper. Did I misunderstood?**

In this work, we intentionally only focus on object existence hallucination as mentioned in Introduction. We consider object attributes and relationship hallucination as our future works. The reason is that - it is not surprising that existing LVLMs fail on expressing object attribute and relationship, because MSCOCO dataset does not include this information, and existing LVLMs’ caption data heavily rely on this dataset. However, it is surprising that object existence hallucination is still an issue for these models, because current detailed caption training data is accurate in object existence. Therefore, we decided to tackle this problem first.

Reviewer comment (Evaluation):
Do you observe any downsides or limitations of using this method that were not expressed in the result Table?

Author response:
**4. Do you observe any downsides or limitations of using this method that were not expressed in the result Table?**

One of the limitations is that the method reaches the best performance with larger image resolution, which aligns with our finding. However, increasing image resolution inevitably leads to computation cost.

Another limitation is the extrapolation ability of the switch. Since we train with epsilon +1 and -1, it makes sense to interpolate well when epsilon is between +1 and -1 during inference, and our Table 8 result on CCEval meet this expectation. However, we thought it would be very interesting to test the extrapolation ability of the switch. Specifically, given a switch is trained using epsilon +1 and -1, can we use epsilon < -1 during inference? It would be very nice if the hallucination continued to decrease for epsilon < -1. Unfortunately, hallucination did not decrease further with -1.2, -1.5. In addition, the model starts to repeat and loss language ability if epsilon is outside of range [1, -1]:

_Example captions with switch value -1.2:_ The scene unfolds on a dining table, which is neatly arranged with a variety of food items. There are two plates of food, one containing a sandwich and the other with a piece of cake. The sandwich is located towards the left side of the table, while the cake is positioned towards the right. In addition to the food, there are several utensils scattered across the table. A knife is located towards the left side of the table, while a fork is positioned towards the right. There are also two spoons, one near the center of the table and the other towards the right side. (The table is not just for food; there are also several books scattered across the surface. One book is located towards the left side of the table, while another is positioned towards the right. The third book is located towards the center of the table.) x N times

_Example captions with switch value -1.5:_ The scene unfolds on a bustling city street, where a man is walking across the street. He is wearing a backpack and appears to be in a hurry. There are several other people scattered around the scene, each engaged in their own activities. Starting from the left side of the scene, there's a car parked on the side of the street. Moving towards the right, there's another car, followed by a truck. Further to the right, there's a traffic light, indicating the direction of the flow of traffic. Starting from the top of the scene, there's a person standing near the car on the left side. Moving down, there's another person standing near the car on the right side. Further down, there's a person standing near the truck. (Starting from the top of the scene, there's another person standing near the car on the left side. Moving down, there's another person standing near the car on the right side.) x N times