RULE : Reliable Multimodal RAG for Factuality in Medical Vision Language Models (2024)

Abstract: The recent emergence of Medical Large Vision Language Models (Med-LVLMs) has enhanced medical diagnosis. However, current Med-LVLMs frequently encounter factual issues, often generating responses that do not align with established medical facts. Retrieval-Augmented Generation (RAG), which utilizes external knowledge, can improve the factual accuracy of these models but introduces two major challenges. First, limited retrieved contexts might not cover all necessary information, while excessive retrieval can introduce irrelevant and inaccurate references, interfering with the model’s generation. Second, in cases where the model originally responds correctly, applying RAG can lead to an over-reliance on retrieved contexts, resulting in incorrect answers. To address these issues, we propose RULE, which consists of two components. First, we introduce a provably effective strategy for controlling factuality risk through the calibrated selection of the number of retrieved contexts. Second, based on samples where over-reliance on retrieved contexts led to errors, we curate a preference dataset to fine-tune the model, balancing its dependence on inherent knowledge and retrieved contexts for generation. We demonstrate the effectiveness of RAFE on three medical VQA datasets, achieving an average improvement of 20.8% in factual accuracy.

## introduction artificial intelligence (ai) has showcased its potential in medical diagnosis, including disease identification, treatment planning, and recommendations @xcite @xcite @xcite . in particular, the recent development of medical large vision language models (med-lvlms) has yes, this fungus image shows suspected glaucoma. ## preliminaries in this section, we will provide a brief overview of med-lvlms and preference optimization. ## methodology in this section, as illustrated in figure 2 , we will introduce rule as an efficient solution for improving factuality of med-lvlms. specifically, our approach consists of three main modules that work together to optimize the model's performance. first, we apply the retrieval strategy to med-lvlms, enhancing the model's ability to leverage retrieved information. second, we implement a statistical method to control the factuality risk through calibrated selection of retrieved contexts. third, we develop a preference optimization method to balance the model's reliance on its own knowledge and the retrieved contexts. next, we will detail these three key modules in detail as follows: ## experiment in this section, we evaluate the performance of rule, aiming to answer the following questions: ## related work factuality in med-lvlms. the rapid development of large vision and language models (lvlms) @xcite @xcite @xcite has begun to impact medical diagnosis. a series of med-lvlms @xcite @xcite , represented by llava-med, have emerged, demonstrating impressive performance across various medical image modalities. however, med-lvlms still exhibit significant factual errors, producing medical responses that conflict with the visual medical information @xcite . this could potentially lead to misdiagnoses or missed diagnoses. recently, several benchmarks @xcite have been established to evaluate the accuracy of med-lvlms in tasks such as vqa or report generation. beyond evaluating factuality, improving the factual accuracy of med-lvlms remains an underexplored area. retrieval augmented generation. rag has recently been recognized as a promising solution @xcite . it enhances the model's ability to generate accurate facts by incorporating contextual information from external datasets. in medical multimodal analysis, the rag approach has been applied to various tasks such as medical vqa @xcite and report generation @xcite @xcite . however, in med-lvlms, applying rag-based approaches overlook two critical issues: the number of retrieved contexts and whether the model overly relies on these reference. these factors can significantly affect the model's performance and may even degrade it. in rule, we systematically address these challenges and enhance the factuality of med-lvlms. ## conclusion in this work, we aim to enhance the factuality of med-lvlm by addressing two key challenges in medical rag. specifically, we first introduce a provably effective strategy for controlling factuality risk through the calibrated selection of retrieved contexts. second, we develop a preference optimization strategy that addresses errors stemming from the model's excessive dependence on retrieved contexts, aiming to balance its intrinsic knowledge and the retrieved information. experiments on three medical imaging analysis datasets demonstrate the effectiveness of rule.