Volcano: Mitigating Multimodal Hallucination through Self-Feedback Guided Revision (2024)

Abstract: Large multimodal models suffer from multimodal hallucination, where they provide incorrect responses misaligned with the given visual information. Recent works have conjectured that one of the reasons behind multimodal hallucination is due to the vision encoder failing to ground on the image properly. To mitigate this issue, we propose a novel approach that leverages self-feedback as visual cues. Building on this approach, we introduce Volcano, a multimodal self-feedback guided revision model. Volcano generates natural language feedback to its initial response based on the provided visual information and utilizes this feedback to self-revise its initial response. Volcano effectively reduces multimodal hallucination and achieves state-of-the-art on MMHal-Bench, POPE, and GAVIE. It also improves on general multimodal abilities and outperforms previous models on MM-Vet and MMBench. Through qualitative analysis, we show that Volcano’s feedback is properly grounded on the image than the initial response. This indicates that Volcano can provide itself with richer visual information through feedback generation, leading to self-correct hallucinations. We publicly release our model, data, and code at https://github.com/kaistAI/Volcanogithub.com/kaistAI/Volcano

## introduction recent large multimodal models (lmms) use substantial image-text or video-text pairs to help instruct-tuned large language models (llms) comprehend visual features produced by vision encoders @xcite @xcite @xcite @xcite @xcite @xcite . with the introduction of fine-tuning methods such as visual instruction tuning @xcite , lmms are now evolving into assistants capable of understanding the world through multiple channels, akin to humans. ## volcano volcano is a single lmm to generate initial responses, feedback, and revisions, as well as decisions to accept revisions. it follows a sequential procedure of an iterative critique-revision-decide loop. in section 3.1, we introduce the process by which volcano self-revises its responses iteratively. section 3.2 describes the collection of multimodal feedback and revision data used to train r @xmath0: return r best volcano. finally, section 3.3 provides detailed information about the models and data used in our study. the overall process is explained in algorithm 1 and illustrated in figure 2 . ## conclusion in our work, we suggest a novel approach that utilizes feedback as visual signals to direct the model to refine responses that do not accurately reflect the image. building on this approach, we present volcano, a multimodal self-feedback guided revision model. volcano has not only achieved state-of-the-art results on a multimodal hallucination benchmark but also demonstrated its effectiveness by improving performance compared to base-line models on multimodal understanding benchmarks. through qualitative analysis, we demonstrate that the feedback produced by volcano is well-grounded on the image, and providing the model with rich visual information helps reduce multimodal hallucination. we hope our model and data open new pathways for strategies to mitigate multimodal hallucination and uncover the fundamental cause of the issue.