EFUF : Efficient Fine-Grained Unlearning Framework for Mitigating Hallucinations in Multimodal Large Language Models (2024)

Abstract: Multimodal large language models (MLLMs) have attracted increasing attention in the past few years, but they may still generate descriptions that include objects not present in the corresponding images, a phenomenon known as object hallucination. To eliminate hallucinations, existing methods manually annotate paired responses with and without hallucinations, and then employ various alignment algorithms to improve the alignment capability between images and text. However, they not only demand considerable computation resources during the finetuning stage but also require expensive human annotation to construct paired data needed by the alignment algorithms. To address these issues, we propose an efficient fine-grained unlearning framework (EFUF), which performs gradient ascent utilizing three tailored losses to eliminate hallucinations without paired data. Extensive experiments show that our method consistently reduces hallucinations while preserving the generation quality with modest computational overhead. Our code and datasets will be publicly available.

## introduction in the burgeoning field of artificial intelligence, the advent of multimodal large language models (mllms) has opened new frontiers in humancomputer interaction, data processing, and automated content generation @xcite @xcite . these sophisticated models, capable of understanding both text and images, have significantly advanced our ability to automate complex tasks. however, an intriguing and critical phenomenon known as "hallucination" in these models poses unique challenges for current research. hallucination in mllms refers to the generation of inconsistent responses that are not grounded by the please describe the image in detail. the image features a large air france airplane flying through a cloudy sky. the airplane occupies a significant portion of image, stretching from the left to the right side of the frame. the airplane's landing gear is down, indicating that it is either preparing to land or has recently taken off. the sky is filled with clouds, creating a dramatic backdrop for the airplane's flight. multimodal context @xcite . for example, as shown in figure 1 , the caption includes the object "landing gear", but in fact it does not appear in the image. such hallucinations will lead to misinformation, potentially undermining user trust in numerous downstream applications. recent methods for mitigating multimodal hallucination can be divided into two categories: inference-based methods @xcite @xcite @xcite @xcite and finetuning-based methods @xcite @xcite @xcite . inference-based methods correct or restrict generated content through external expert review, self-reflection or decoding strategies during inference stage. however, they usually require additional inference steps with increased costs and delay @xcite . furthermore, each task demands specific procedure or prompt @xcite , adding to the complexity of implementation. overcoming these drawbacks, finetuning-based approaches are proposed to adjust the model directly through specialized datasets and preference alignment algorithms. these algorithms, including rlhf @xcite , dpo @xcite @xcite and contrastive learning @xcite , enhance the congruence between text and images, leading to improved alignment. although they have achieved good performance, two critical issues emerge: first, their data demands are substantial, as they require a comprehensive set of paired positive and negative samples for effective finetuning. the alignment algorithms they employed demand paired hallucinated and non-hallucinated responses for each query. acquiring such specific and varied response sets for each query presents a significant challenge. recent methodologies in this field predominantly rely on human labor to annotate the output from the mllm, requiring specialized expertise and incurring considerable expenditure of time and financial resources. second, the finetuning of mllm utilizing these alignment algorithms usually demands considerable computational resources. most of these techniques are sophisticated and necessitate the simultaneous operation of multiple models to execute preference alignment, thereby escalating the overall cost significantly. to tackle the above issues, we propose the efficient fine-grained unlearning framework (efuf), which offers the advantage of not necessitating paired data and being more efficient during the finetuning phase. our method, grounded in the principles of unlearning, mainly relies on performing gradient ascent on negative samples to mitigate hallucinations, eliminating the need for costly manually-annotated paired data. additionally, it consumes considerably fewer computational resources. unlike traditional alignment algorithms that require simultaneous operation of multiple models to execute preference alignment, efuf operates without this requirement. the key to applying the unlearning algorithm is how to curate positive and negative samples, i.e., distinguish between real and hallucinated objects, in a manner that is both cost-effective and reliable. intuitively, the similarity between objects and their corresponding images can act as an indicator for hallucinations, since the image contains real objects but not the hallucinated ones. inspired by @xcite , we propose to utilize the clip model @xcite to evaluate text-image congruence. trained on a vast corpus of text-image pairs, clip stands as a robust tool to help identify hallucinations. after ascertaining the capability of clip through a preliminary experiment, we curate our dataset manually-free by utilizing clip scores, before applying our unlearning-based method to mllms. this process enables us to harness the power of unlearning, offering a potent and efficient approach for mitigating hallucinations in mllms. our contribution can be summarized as follows: 1) to the best of our knowledge, we provide a new perspective to utilize unlearning to mitigate multimodal hallucination in mllms. 2) we propose an efficient fine-grained unlearning framework efuf, which can obtain positive and negative examples separately in a cost-effective and reliable manner. 3) efuf has good compatibility and can be easily extended to existing mllms. experiments conducted across a range of mllms validate the effectiveness of our method. ## related work in this section, we review the existing studies on hallucination mitigation for mllm and unlearning algorithm. ## preliminary experiment the initial phase of our research involves confirming the hypothesis that text-image congruence can serve as a reliable indicator of hallucination occurrences. to this end, we designed a preliminary study aimed at validating this premise. below, we detail the methods and findings of this experiment. ## conclusion in this paper, we find that text-image similarity is helpful for identifying multimodal hallucinations, and propose a novel unlearning framework to mitigate hallucinations in mllm. specifically, we first curate different samples utilizing the imagerelevance score derived from clip similarity, and then design three distinct losses to perform unlearning on the curated samples. extensive experiments on different baselines show that our method effectively reduces multimodal hallucinations while retaining the general performance of the model.