MMed-RAG: Versatile Multimodal RAG System for Medical Vision Language Models
Authors: Peng Xia
Kangyu Zhu
Haoran Li
Tianze Wang
Weijia Shi
Sheng Wang
Linjun Zhang
James Zou
Huaxiu Yao
Venue: ICLR 2025 Poster (2025)
Keywords: medical vision-language model
retrieval-augmented generation
Primary area: applications to physical sciences (physics, chemistry, biology, etc.)

Abstract: Artificial Intelligence (AI) has demonstrated significant potential in healthcare, particularly in disease diagnosis and treatment planning. Recent progress in Medical Large Vision-Language Models (Med-LVLMs) has opened up new possibilities for interactive diagnostic tools. However, these models often suffer from factual hallucination, which can lead to incorrect diagnoses. Fine-tuning and retrieval-augmented generation (RAG) have emerged as methods to address these issues. However, the amount of high-quality data and distribution shifts between training data and deployment data limit the application of fine-tuning methods. Although RAG is lightweight and effective, existing RAG-based approaches are not sufficiently general to different medical domains and can potentially cause misalignment issues, both between modalities and between the model and the ground truth. In this paper, we propose a versatile multimodal RAG system, MMed-RAG, designed to enhance the factuality of Med-LVLMs. Our approach introduces a domain-aware retrieval mechanism, an adaptive retrieved contexts selection, and a provable RAG-based preference fine-tuning strategy. These innovations make the RAG process sufficiently general and reliable, significantly improving alignment when introducing retrieved contexts. Experimental results across five medical datasets (involving radiology, ophthalmology, pathology) on medical VQA and report generation demonstrate that MMed-RAG can achieve an average improvement of 43.8% in factual accuracy in the factual accuracy of Med-LVLMs.

Published as a conference paper at ICLR 2025
MMED-RAG: VERSATILE MULTIMODAL RAG SYS- TEM FOR MEDICAL VISION LANGUAGE MODELS
Peng Xia', Kangyu Zhu?, Haoran Li Tianze Wang', Weijia Shi Sheng Wang",
Linjun Zhang', James Zou", Huaxiu Yao! JUNC- Hill, Brown University, Carmegie Mellon University, 'Rutgers University, University of Washington, Stannford University {pxia, huaxiu//css enc
ABSTRACT
Artificial Intelligence (AI) has demonstrated significant potential in healthcare, particularly in disease diagnosis and treatment planning. Recent progress in Med- ical Large Vision-Language Models (Med-LVLMs) has opened up new possibil- ities for interactive diagnostic tools. However, these models often suffer from factual hallucination, which can lead to incorrect diagnoses. Fine-tuning and retrieval- -ariteee-- generation (RAG) have emerged methods address these issues. However, the amount of high-quality data and distribution shifts between training data and deployment data limit the application of fine-tuning methods. Al- though RAG is lightweight and effective, existing RAG-based approaches are not sufficiently general to different medical domains and can potentially cause mis- alignment issues, both between modalities and between the model and the ground truth. In this paper, we propose a versatile multimodal RAG system, MMed-RAG, designed to enhance the factuality of Med-LVLMs. Our approach introduces domain-awand aware retrieval mechanism, an adaptive retrieved contexts selection, and provable RAG-based preference fine-tuning strategy. These innovations make the RAG process sufficiently general and reliable, significantly improving align- ment when introducing retrieved contexts. Experimental results across five med- ical datasets (involving radiology, ophthalmology. pathology) on medical VQA and report generation demonstrate that MMed-RAG can achieve an average im- provement of 43.8% in the factual accuracy of Med-LVLMs. Our data and code are available in hhttps///ittuubb. https://gthub..om//charr ceeeerddpeeg---eee xia/MMMe--RAG
INTRODUCTION
Artificial Intelligence (AI) has already transformed healthcare and still has a lot of potential for fur-
ther advancements (Täjann et 2021; Wang et al.) [019 2025 Ye al., [202]] Tu et al.] [024]
Xia et al.
2024b Hu et al. 2024 Ju al. (0224). Recently, Medical Large Vision-Language Mod-
els (Med-LVLM)
have shown great promise for advancing interactive and intelligent diagnosis (Li
ett al. 2023a Moor et al.) 2023 Zhang al. 2023b Wu et al. 2023b). Despite this potential LLi
ctt al.
2023b Wu et al. 2023at Shi et al. 22024). current Med-LVLMM still face significant reliabil-
ity issues, particularly their tendency generate non-factual medical responses (Xia et al. 2024a Royer et al. 2024) Chen et al. 2024a Jiang et al. (024)) making them unreliable in critical medical applications. These factuality issues raise serious concerns when deploying such models in clinical settings, where even small diagnostic errors could lead to severe consequences for patient care.
Recently, researchers have begun focus on improving the factuality of Med-LVLMs through var- ious techniques, including fine-tuning (Li et al. 2023a Moor et al. 2023 Thawkar et al., 2023 Zhang et al., 2023b, Chen et al. 2024b) and retrieval -uggeeeeed generation (RAG) (Xia et al., 2024c; He et al.) 2024; (unn et al. 2024b). Fine-tuning is a direct method to improve model per- formance, but faces several limitations in the medical field. First, there a lack of sufficient high-quality labeled data for fine-tuning in the medical domain. Additionally, a distribution gap often exists between the training data and the real-world deployment data (Schrouff et al. 2022). leading to significantly worse model performance during deployment. Hence, RAG has emerged as a viable alternative by providing external references during the inference stage, enhancing the factuality of Med-LVLMs (Wu et al. 2023c Gao et al., 2023). However, despite its advantages, rent RAG implementations in Med-LVLMe have significant limitations. First, these methods tend to be dataset-specific, reducing their generalizability across various medical domains. Second, these

---

Published as a conference paper at ICLR 2025
models are still facing misalignment issues that lead to factuality problems. This misalignment may arise from the impact of adding RAG on the original Med-LVLMs' cross-modality alignment, as well as on the overall alignment between the model and ground truth.
To address these challenges, we propose a versatile factual Multimodal Medical RAG system called MMed-RAG, Specifically, MMed-RAG first introduces domain-aware retrieval mechanism, de- signed to handle different domains of medical images more effectively. Here, we design a domain identification module adaptively select a corresponding retrieval model given the input medical image. Secondly, we include a adaptive calibration approach for selecting the number of retrieved contexts. Lastly. MMed-RAG incorporates RAG-based preference fine-tuning enhance cross- modality alignment and overall alignment with ground truth. The preference pairs are designed to achieve two goals: first, to improve cross-modality alignment by encouraging the model to avoid generating responses without utilizing input medical images, even the responses are correct; sec- ond, to improve overall alignment by encouraging the model to understand retrieved contexts when unsure, while avoiding interference from irrelevant retrieved information.
The primary contribution of this paper is MMed-RAG, a versatile multimodal RAG system designed specifically for Med-LVLMs to generate more factual responses. Under mild assumptions, our the- oretical analysis demonstrates that MMed-RAG mitigates both cross-modality misalignment and overall misalignment with ground truth. Furthermore, empirical results five medical multimodal datasets, covering three medical image modalities (radiology, pathology, and ophthalmology), show that MMed-RAG significantly improves the factual accuracy of Med-LVLMMs, achieving improve- ments of 18.5% and 69. 99.1% % on Medical VQA and report generation tasks, respectively, compared to the original Med-LVLM. These empirical findings further demonstrate the effectiveness of our proposed components and support the theoretical analysis in addressing misalignment issues.
2 PRELIMINARIES
In this section, we will provide a brief overview of Med-LVLMs and preference optimization.
Medical Large Vision Language Models. Med-LVLMs bridge LLMs with medical visual mod- ules, allowing the model to take medical image z. and clinical query ia as input x, and autoregres- sively predict the probability distribution of the next token. The text output is denoted as y.
Preference Optimization. Preference optimization has achieved remarkable results in LLM align- ment. Give an input x, a language model policy To can produce a conditional distribution no (y x) with as the output text response. The recently popular DPO [Rafailov et al. 2023) utilizes preference data achieve objective alignment in LLMs. The preference data is defined as D (5) and (8) represent preferred and dispreferred responses given an in- put prompt x. The probably of obtaining each preference pair is p(ye > y) = ar((,, yu )-) r(x, v(),, where (--) )) is the sigmoid function. In DPO, the optimization can be formulated as classification
(a(9), Ve Vi hi=1, where yu
(F) (4)
loss over the preference data as:
Lopo mez) = Ez 1ND [log a (a log ne(vela) (e) a log (x())=>) [at(vuls)
(I)
where Ty represents the reference policy, which is the LLM fine-tuned through supervised learning.
3 MMED-RAG: A VERSATILE MEDICAL RAG SYSTEM
In this section, as illustrated in Figure T. we will propose MMed-RAG, a versatile RAG system for improving the factuality of Med-LVLMs. Specifically, MMed-RAG consists of three complemen- tary modules. First, we design a domain-aware retrieval mechanism to select the optimal retriever by feeding each given medical image to the domain identification module. Second, to select an optimal number of retrieved contexts and filter out low-quality information, MMed-RAy adopts a adaptive method by filtering out low-quality information using the similarity scores during the RAG phase. Lastly, we use a RAG-based preference fine-tuning approach to improve the cross-modality alignment and the overall alignment between groundtruth. We detail these steps as follows:
3.1 DOMAIN-AWARE RETRIEVAL. MECHANISM
In MMG, we introduce a domain-aware retrieval mechanism to efficiently handle medical images from different sources (e.g., radiology, pathology, ophthalmology). Specifically, first
2

---

Published as a conference paper at ICLR 2025
meaor Retrieval
Mechanise Se
Adeptive Rebrieved Context Selection
Out MMC Untre
H Radiology
Medical Image
Med-LVLM
local siricace
there
Question ppaacly present??
Donntracted Proforence Fairs
logg?)
Doman n Label
Iictiniiationn
Redinlogy
Adapiiveok
Repporti
k Pathology
Rebriaver Couthologs)
TOp Pepprts
Simblaityy Scores
RAG BeseBBeeed Preference Fine-Turing
T Med-LVUM
www.
Proference Data
Med
BAG
Cannot Solve Proslers by Self
Originll
How
Copy Hamswork Fror
Fine-tuning
Oriinal
Med LALM
Med-LVUM
DILM
Irag
RAA
Copted OMOP'
eeergar Med-LMLM
Avoid Interference Froe Incorrect Homeoook
Figure I: Overview of MAMe--RAG, a versatile factual multimodal RAG system designed to enhance the reliability of Med-LVLMs. It introduces domain- aware retrieval mechanism that effectively handles different domains of medical images by selecting suitable retrieval models. Additionally, uses adaptive context selection approach determine the optimal number of retrieved contexts and employs preference fine-tuning to improve both cross-modality and overall alignment.
employ a domain identification module that assigns a domain label to each input medical image. To achieve this, create small dataset with medical images as inputs and their corresponding domain labels as outputs, using this dataset to fine-tune the BiomedCLIIP model (Zhang et al. (023a)) improve its domain awareness. Formally, for a given medical image Tu. we predict its domain d = Fzz,). Based on the assigned domain label d, the image By is fed into the corresponding multimodal retriever Ra(-)) for knowledge retrieval.
Here, each multimodal retriever Ra(-)) for each domain d is trained through contrastive learn- ing (Radford et al., (02T). Specifically, the visual and textual information Xamg- Xtzt are pro- cessed by their corresponding encoders (-m (-)g())) E-ze()) to generate textual and visual embeddings Vist = Etze( (Xtz),, Ving = (mmmg)))) Contrastive learning loss is then applied to maximize similarity between text and image embeddings representing the same example, while minimizing the similarity between embeddings representing different examples, as defined below:
Limg + Lezt where Limg
1 N
exp(Sis) Σ log N Ei=1 exp(S Lest = Σ log exp(Si.s
N
EN exp(S,s)
(2)
where S E N represents the similarity matrix between image and text modalities, calculated (Mm V where each element S.j represents the similarity between the image
as: S Vimol
Vimu
representation of example and the text representation of example j.
Finally, for the input image 2,, after feeding into the corresponding multimodal retriever R.(()), the multimodal retriever will retrieves the top-k most similar reports for the image. These retrieved re- ports T = RR(x)) are then provided to the Med-LVILM M(.) references to guide the generation.
3.2 ADAPTIVE RETRIEVED CONTEXT SELECTION
Val nalbattonArsii Azruway Teet srly
CLI Suon
Similerity Score (%)
Accurwey (%))
Following the domain- aware retrieval mechanism, the next step is to determine the optimal amount of context to retrieve. Retrieving too much or too little information can result in hal- lucinations (Xia et al. 2024c). Current RAG methods applied to Med-LVLMs generally rely on empirical results or fixed values based validation sets to seleet the optimal value of the number of retrieved contexts & (Xia et al. [2024c] He et al.) 2024 Sun et al. 2024b). However, the distribution of simi-
3 80
Top-k Reports
Figure 2: Relations between se-
lected contexts and similarity score.
3

---

Published as a conference paper at ICLR 2025
larity scores varies depending the complexity of the image and its alignment with the textual information from the data source. These fixed-k methods do not guarantee optimal performance on target data, as they overlook the similarity scores generated during the retrieval process. To address this, we propose an adaptive method that dynamically selects l based on the similarity scores of the retrieved contexts. Specifically, during the domain-aware retrieval mechanism phase, the retrieved information is denoted as z, (k) = Rz( where k represents the number of retrieved con- texts, and the corresponding similarity scores are denoted as Sk. For simplicity, when there is no ambiguity, we will refer to Jr, (R) as Zy--
As illustrated in Figure 2 our method is based on a key observation: the similarity scores (CLIP score in this case) between retrieved contexts often exhibit a sharp decline after a certain number of results (nearly top-9 this case). This suggests that lower-quality information can still be included among the top-k retrieved contexts when using fixed-k strategy, especially in cases where the fixed value of I is too large. These lower-quality retrievals introduce noise and irrelevant information, which can significantly impair the model's ' ability to generate factual and coherent responses. To mitigate this issue, we draw inspiration from the Gap statistic method used in clustering (Tibshirani et al., 2001) and extend this concept to RAG for Med-LVLMs. Specifically, after retrieving the top-k contexts, we perform an additional round of & optimization by analyzing the similarity ratios between consecutive retrievals. These similarity ratios are denoted as u, log(S,/SS /S5++))) for 0 < i S k. where S; represents the similarity score of the i-th retrieved context. When u, exceeds a predefined threshold y. this indicates a substantial drop in relevance, suggesting that the remaining retrievals are less likely to contribute preferredly to the model's 's output. At this point i, we truncate k. effectively discarding the less relevant retrievals that follow. This adaptive truncation mechanism ensures that only the most relevant contexts are retained for generating the final response, reducing the risk of hallucination and improving the factual accuracy of the outputs.
Although the threshold is fixed, this approach provides a adaptive way to balance the bias and variance in retrieved contexts. By adapting to the characteristics of each input des our method enhances the robustness of the retrieval process and ensures that the selection of l is tailored to the specific data at hand, thereby improving overall performance across diverse contexts and tasks.
3.3 RAG-BASED PREFERENCE FINE-TUNING
After context selection, MMed-RAG supplies Med-LVLM with reliable retrieved information as external knowledge to aid in generating factual responses. However, incorporating this retrieved knowledge may potentially disrupt the original alignment within the existing Med-LVLM, a concern we will elaborate on below:
Alignment Analysis. In the alignment analysis, aim to explore how incorporating retrieved con- text impacts the original alignment in Med-LVLMMs focusing on two key aspects: (1) cross-modalin alignment and (2) overall alignment with the ground truth. To evaluate cross-modality alignment, we conduct two tests on LLaVA-Med-1.5 (Li et al. 2023a) using the Harvard-FairV (Luo) let al., 2024) dataset. First, when replacing the original image with a highly noisy image associated with a different ground truth, the original model gives incorrect answers (the ground truth being the response for the original image). After incorporating RAG, where context is retrieved based on the original image. 55.08% of these cases return correct answers. This indicates that the model directly references the retrieved knowledge without considering the input image, highlighting signif- icant cross cross-modal misalignment issues. Furthermore, 43.31 % of the questions that were originally answered correctly are answered incorrectly after incorporating RAG, suggesting interference from incorrect retrieval information, which leads to overall misalignment with the ground truth.
To address mross-modality misalignment and the overall misalignment introduced by incorporating retrieved knowledge, shown in Algorithm I we propose based preference fine-tuning (RAG-PT) approach to fine-tune the target Med-LVLM M()) Specifically, RAG-PT constructs two types of preference pairs designed mitigate both categories of misalignment.
Preference Pairs for Cross-Modality Alignment. We first construct preference pairs aimed at improving cross- modallty alignment. In this dataset, we select samples from D = (=55) ( ) where Tes 24, and y represent the input medical image, clinical query, and ground-truth answer, respectively. For simplicity, we omit the sample index (t) in the following sections A model's correct response using retrieved knowledge, i.e., M (It, y, is considered a preferred
4

---

Published as a conference paper at ICLR 2025
Algorithm 1: Versatile Multimodal RAG System (MMed-RAG)
Input: D (arb), 22(0) " Dataset; na: Parameters of the Med-LVLA LVLM; Med-LVLM: M(..):
Domain Identification: FF):: Retriever: R(-); Noisy Funetion: I(-).
Output: Ted. Parameters of the reference model.
b Training Stage
Initialize Dem with an empty set
foreach (I,,
do Generate retricved contexts with an assigned domain label xr +- RF((x) (xt)
Generate the noisy image æ,, 4- [(w.)
I Cruss-Modality Alignment
if M(xe, (2ra, (x)) = y and M(xi, (2t, then
Select the preferred response Vw.o1 t- y, dispreferred response Vi.01 + (2
Put {Z x. v1), Vuu,oi, V1..1 into Dem
Overall Alignment
Initialize Doo and D with empty set
11
if M(xe, (2t, x.)) y and Mx.. 24) y then
13
Select the preferred response Vw.o2 V, dispreferred response Vt,o2 + M(z
Put It), o2, Vi.02) into D
15
if MMLa.... [() = y and M(Z.. (#x, (.)) æ then
16
Select the preferred response Vww.o3 + y, dispreferred response yi.oa + MMFFv, (Ie, x,,)
Put ((to, It), 03 M1.03) into D
17
18 Det Dem UD.. Doa Doa U D
19
foreach ((Zv, R.., Ic H... ow (... ) E Dut do
20
Compute the losses Lpt following equation and update Tie
Inference Stage
foreach sample (e)) do
23 Select top-k retrieved contexts with an assigned domain label or t- (F(
24
Get the predictions of the model w/ RAG-PT p + MME,, (21,
response ps, where T, is the retrieved information. A dispreferred response selected from cases where the model makes a correct inference based on an unrelated image, i.c., MMx xx) # y. but M 24 2,) y. reflecting the model's reliance images x. are generated through a two-step process: first, we use the retriever to select an image x' with the lowest similarity to the target image; then, introduce diffusion noise into the selected unrelated image. We define the noise step as 8, and the noised image at step is expressed as:
the retrieved knowledge. The unrelated
z = V 2' E
(3)
where &. o & and &. E 10, is a hyperparameter. The preference pairs constructed in this stage denoted as Dem- By comparing the preferred and dispreferred responses in Dem- encourage the model to prioritize the input medical image when generating responses.
Preference Pairs for Overall Alignment. Second, construct preference pairs to improve overall alignment, focusing on enhancing the model's ability to effectively leverage retrieved knowledge when generating responses. The preference pairs in this stage are constructed from two subsets. The first subset, D is designed to strengthen the model's comprehension and reasoning abilities regarding the retrieved knowledge. Preferred responses are selected where the model correctly an- swers based on both the original image and the retrieved information, i.e., MM 4y + x,)) = y. while dispreferred responses represent cases where the model answers incorrectly based on the im- age without using retrieval, i.c., MMI,,, xT.) * Comparing these preferred and dispreferred re- sponses enhances the model's understanding of the retrieved information and improves the overall effectiveness of RAG. In the second subset, D2 the goal is to mitigate interference from the re- trieved knowledge. Preferred responses are selected where the model correctly answers based solely on the original image without using retrieved knowledge, i.e., MMI,, responses occur when the model answers incorrectly using both the image and retrieved informa- tion. i.e., M(rx, De + 2r) # y. This helps the model learn when rely on its internal knowledge versus retrieved knowledge. Finally, we combine the first and second subsets to form the second set of preference pairs, Drm D1 2
y. while dispreferred
5

---

Published as a conference paper at ICLR 2025
Finally, merge the first and second preference set and denote the preference dataset Dpt (8) ) Dem UDo. (x(0) _010,, yuuo, Mi,o Ti=., where (t) Vi.o represented as preferred and dispreferred responses, respectively; x² denotes the noisy data. Based on the curated preferences, we fine-tune
Med-LVLMM using direct preference optimization (Rafailov et al. 2023) with the following loss:
Let -( [log a (a log
ma (e( (V(
To(u)al=") log s.(0),a
(4)
THEORETICAL ANALYSIS
In this section, we provide theoretical analysis of the model obtained from equation Jn examine how the image input and retrieved context influences the model. Recall that 2u, y, 2,, I, denotes imput medical image, groundtruth answer, question, and retrieved information, respectively.
4.1 THE IMPROVEMENT ON CROSS-MODALITY ALIGNMENT
We first consider the loss for cross-modality alignment,
sollwoole) Lem Bw.. o) [log (a log Tolbw.olo)
log (00
(5)
where (Tews ow) ~ qw (Tws Uw Fe, xip) and (mi, Vt,o) ~ q( M1 my, xr,)) represent distributions of the preferred responses and dispreferred responses on Dems respectively. Let denote (I Ir, 2x)
Definition 4.1 Define the weight of vo with respect log to(y)>>) (x) as
8 Ox.
wi(ro, Ta) = E (y
log To (u)
(6)
Definition 4.1 describes how log no (y)z) changes with respect to Tus and the weight is always non-
dispreferred. We demonstrate that this is reasonable definition through Lemma 4.1
Lemma 4.1 For linear model y = x + 02æx + such that ~ Assumption 4.1 Let hxx, y), abbreviate as h, be
N (0, 1), wi(Kes To) = 03
= Σ To(y)a))
qae (y|=z) (0(y)xr)
Assume that wI(K,, To) < c. where
0 Ox, Assumption 4. requires that x., has a small weight in log T. (v)z). A model no (y(()) independent of I. could satisfy Assumption 4.1 In this case, the reference model generates answers without using information from the image.
VTooullt)
/
To To(y)a))
VTo(ula)
ox, h
dy
ax
Theorem 4.1 Suppose that Assumption
4.1 holds, cross-modality loss increase the weight of xv-
(9) Theorem 4.1 indicates that when the weight of 2, is too small in the initial model (T(2))), the
wf(2e, me) > wi(Ze,
mross-modality loss function adjusts the model to place greater emphasis on images, informed by the retrieved data. Intuitively, for any sample (z, y). generating unrelated images causes the policy to rely less on images. By using samples from this distribution as negative samples, the new model diverges from the initial model, increasing its reliance images.
4.2 THE IMPROVEMENT ON OVERALL ALIGNMENT
In this section, we analyze the improvement on overall alignment. Let qu (Zo, you k xry) and (I (t,oxx)) represent distributions of the preferred responses and dispreferred responses on Di. respectively; q (Iv. Vve,o (2t) and of (Zv. (1,. represent distributions of the preferred responses and dispreferred responses D2 respectively. Overall loss is defined by
Lon - -E(z.vu on..)) log a ( log No(sw.als) log [x( (.(
(10)

---

Published as a conference paper at ICLR 2025
Consider и as the generative distribution underlying M, construction of D and D indicate that there is significant gap between т Ty Tt, Ø4) and π ( Rus Tt, Øx) for 2, generates true answer while 2, generate a false one.
Assumption 4.2 Assume that (y) Fr, Ir, Iz) 2 + is L-lipschitz continuous on H, for all (Fo, Lt. y) such that [y] Fr, Fr, - π Tl Tes Ie. xe, Br)l S I. (T Br); where d2 is any distance metric on the text space.
Based on Assumption 4.2 3, can be viewed as being far from the meaningful retrieved information Ty, resulting in different weight in the model. Then, claim in the following theorem that the overall loss in equation 10 can effectively leverage retrieved knowledge while training.
Assumption 4.3
Let (N Pt, Fr, y). abbreviate as h1, be
h1 = Σ (v(l))))
qau (ul 21 x.) + (N 1.
Ou (ylan, wi, v., + qe Jin xi)
1
9f (1) Te: (/(11) (,) te, Je,
q (yl me) + 7 (T, TT.
(11)
Assume that wi( To) oj and wi( Ro) By where
ah der
/
0h1
To dy
th1 DEr
VTaa
Sxr
l11
2
12
(12)
ViT
Oh 0ā,
Ule
Ne h1
+
ha dy + TT
Ola 0i,
as,
To
12
Theorem 4.2 Suppose that Assumption 4.3 holds, then overall loss increase the weight of a, and decrease the weight of Zy.
wi(Kr, mo) > wi( no), w((a:: mo) < wi( mo)
(13)
Theorem suggests that the model tend to improve the overall alignment. When 7, generates false answer, the training procedure tends to reduce the reliance on ă,. resulting in a decrease the weight assigned to H,. Conversely, if is helpful for generating the true answer, T)(y)z) tend to enhance its use of Iz.
EXPERIMENT
In this section, we evaluate the performance of MMed-RAG, aiming to answer the following questions: (1) Can MMed-RAG effectively improve the factuality of Med-LVLMMs compared to decoding-baan -based and RAG-based baselines? (2) How effective each proposed component on per- formance? (3) What is the effect of preference data for different alignment goals? and (4) Does MMed-RAG actually improve cross-modality alignment and overall alignment?
5.1 EXPERIMENTAL SETUPS
Implementation Details. We use LLaVA-Med-1.5 7B (Li et al. During the preference fine-tuning process, we adapt LoRA fine- tuning (Hu et al. 2021). For the training of retriever, the vision encoder bio Nho-BioClinicaBBERR (Alsentzer et al. 2019). We use the Adam W optimizer with learning rate of 10-3 weight decay of 10 and a batch size of 32. The model is trained for 360 epochs. For more detailed information on training hyperparameters and training data, please see Appendix A.I.I
2023a)
the backbone model.
ResNet-50 (He et al. 2016), and the text encoder is
Baseline Methods. We compare MMed-RAG with two types of LVLM hallucination mitigation methods that show promising results in natural image understanding. 1) Decoding based methods, including Greedy Decoding, Beam Search (Sutskever et al., 2014), DoLa (Chuang et al.) 2023 OPERA (Huang al.) 2023), VCD (Leng et al. (024). These methods manipulate the logits of the model's 's output tokens to enhance factual accuracy. 2) Multimodal RAG- methods, including (Sun al. (024b), RULE (Xia et al. 2024c). Furthermore, we compare the performance with other open-source Med-LVLMs. including -laaminoo (Moor
MedDr (He et al. 2024,, FactMM-RA
al. 2023). MedVIn'T (Zhang al. 2023b), RadFM (Wu et al. 2023b).

---

Published as a conference paper at ICLR 2025
Table I: Model performance (%) of different methods based on LLaVA-Med--.5 on medical VQA task. Notably, we report the accuracy, F1 score and AUROC. The best results and second best results highlighted in red and blue respectively.
Radiology
Ophibalmooooy
Pathology
Models
IU-XXay
MIMIC-CXR
Harvard-FairV
Quili-M
PMC--A (Pathology)
Ace
F1
AUC
Acc
F1
AUC
Acc
F1
AUC
Acc
F1
AUC
Azc
F1
AUC
LLaVA-Med-15
75.47
67.16
75.79
68.88
63.05
72.90
60.03
59.25
51.19
4 Greedy
76.88
65.59
68.74
78.32
86.75
71.13
82.54
85.98
70.09
64.72
70.12
58.75
58.61
70.42
53.
+ Beam Search
76.91
66.06
68.77
81.55
86.36
73.79
80.93
88.08
68.94
63.52
69.33
57.65
56.29
69. 84
52.89
Dolla
78.00
72.19
72.73
67.10
63.47
57.58
70.27
52.95
+ OPERA
70.59
61.54
63.22
69.34
76.66
62.46
71.41
81.37
65.59
60.51
66.32
$1.79
55.32
68.30
51.86
+ VCD
68.90
54.35
61.08
70.89
75.57
64.61
65.88
77.20
64.16
61.43
67.39
53.72
55.10
67.94
51.62
MediDe
83.33
77.15
64.15
67.01
57.01
ReeMMM-RAG
84.51
77.07
77. 77.58
81.86
70.00
83.67
72.20
09:25
73.62
68.15
57.31
+ RULEE
87.84
78.00
85.78
83.9
87.49
83. 44
87.12
92.89
77.08
68.97
73.8
68.13
61.41
70.36
58.91
MMed-RAG
89:54
80.72
87.13
83.57
88.49
08
87.94
78
80.81
72.95
76.35
72:25
64.54
73.09
61.42
Table 2: Model performance (%) of different methods on report generation task. Notably, we report the average BLEU score, ROUGE-L, METEOR. For detailed BLEU score, see Appendix A.6.8
Radiology
Ophthalmology
Models
IU-Xray
MIMIC-CXR
Harvard-
BLEU
ROUGE-L
METEOR
BLEU
ROUGE-L
METEOR
BLEU
ROUGE-L
METEOR
LLaVA-Med-1.5
9.64
12.26
8.21
12.11
13.05
11.16
18.11
11.36
10.75
Greedy
11.47
15.38
12.69
16.63
14.26
14.19
17.98
11.49
13.77
Beam Search
12.10
16.21
13.17
16.97
14.74
14.43
18.37
12.62
14,50
DoLa
11.79
15.82
12.72
17.11
14.89
14.81
18.26
12.51
14.51
OPERA
10.66
14.70
12.01
15.40
12.52
13.72
16.59
11.47
13.63
VCD
10.42
14.14
11.59
15.18
12.30
13.38
16.73
11.38
13.89
ModDr
12.37
16.45
13.50
18.59
15.72
16.77
19.82
13.72
15.40
FactMM-RAG
14.70
15.92
18:71
15.84
16.82
20.82
14.17
15.31
RULE
27.53
23.16
27.99
18.61
15.96
17.42
22.35
14,93
17.74
MMed-RAG
31.38
25.59
32.43
23.25
12.34
20.47
24.82
16.59
19.85
Evaluation Datasets. We utilize five medical generation tasks, i.e., MIMIC-CXR (Johnson et al.) 2019). IU-Xray (Demner-Fushmal et al. 2016), Harvard-FairVL.N. (Luo al. (2024), PMC-OA (Lin al., 2023a) (we only select the pathology part) and Quilt-IM (Iezzogwo et al. 2024). These datasets cover radiology, ophthalmology, and pathology. To construct the VQA benchmarks, following (Xia al. 2024a), we generate question- answer pairs from medical reports using GPT-4 (OpenAll 2023), with answers formatted as yes or no. Pathology images are excluded from the report generation task due to their brief and insufficient descriptions. The detailed dataset descriptions are provided in the Appendix A.2
datasets for medical VQA and report
Evaluation Metrics. Following (Jing et al. 2017 Lin al. 2023b), we use Accuracy, F1 Score and AUROC for evaluating medical VQA task, and BLEU Score (Papineni et al. (2002), ROUGE-L. (Lin 2004) and METEOR (Banerjee & Lavie 2005)
for evaluating report generation task.
5.2
MAIN RESULTS
In this section, we provide a comprehensive comparison with various bascline methods and other open-source Med-LVLMs medical VQA and report generation tasks.
Comparison with Baselines. We compare MMed-RAG with baseline methods on medical VQA and report generation tasks, with the results presented in Table I and Table respectively. Overall, MMed-RAG outperforms all baselines across nearly all metrics and datasets. Specifically, MMed- RAG demonstrates significant performance boost, improving by 18.5% and 69.1% over the orig- inal Med-LVLM in medical VQA and report generation tasks, respectively. When compared to baseline methods, MMed-RAG surpasses seccoddin-bbseedd approaches, achieving improvements of 11.5% and 44.2% in the two tasks. Furthermore, recent RAG-based methods show substantial im- provements over earlier techniques. yet our approach still outperforms RAG-based baselines by 2.8% and 16.1% in the medical VQA and report generation tasks, respectively. This indicates that MMed-RAS RAG effectively mitigates misalignment issues introduced by RAG. Notably, MMed-RAC achieves more pronounced gains in report generation, likely due to the higher complexity of the task and the greater influence of retrieved contexts in guiding open-ended generation.
8

---

Published as a conference paper at ICLR 2025
Comparison with Other Med-LVLMs. To provide com- prehensive comparison, we evaluate MMed-RAG against other open-source Med-LVLMs to demonstrate the superiority of our approach. We assess the performance of these models across different medical image modalities, reporting the average re- sults for medical VQA and report generation tasks in Table [ (see Appendix A.6 for detailed results). Our findings show that MMed-RAG significantly outperforms Med-LVLMs pre-trained on large-scale datasets across various domains. This reinforces the generalizability and effectiveness of our approach across di- verse image domains and medical multimodal tasks.
Table 3: Performance compar-
ison with several Med-LVLMs. Rad: Radiology, Opt: Ophthalo- mology, Pat: Pathology.
Model
Rad
Opt
Pat
Med- Mod-Flomingo
27.42
22.50
29.11
McdVinT
33.17
29.40
25.33
RadI'M
35.82
27.07
24.82
miniGPT-Man Med
36.66
25.28
23.16
MMed-RAG
56.94
56.38
54.10
5.3 ANALYSIS
In this section, we provide detailed analysis of each module's performance, along with a series of analytical experiments, to better understand the performance gains of MMed-RAG, Additionally, we demonstrate the compatibility of our method in Appendix A.6 including its application to generalist and domain-specific Med-LVLMs.
Ablation Studies. We conduct series of ablation experiments to evaluate the impact of each compo- nent in MMed-RAG. The results for both medical VQA and report generation tasks on the IU-Xray and Harvard-Fair FairVLMedd datasets are summarized in Ta-
Table 4: Ablation results on two datasets covering different domains. RG: report gen- eration, Fai: Harvard-Fairds Faiaard-Fai-MMeeeeee
4 According to the results, we can see that: (1)
Model
IU-Xray
Laid
The domain-aware retrieval mechanism (DR) sig- nificantly improves the factuality of Med-LVLM. with an average performance increase of 17.9% and 16.1% on the IU-Xray and Fair datasets, re- spectively. Here, the retrieved knowledge aids the model in generating more factual responses. (2) Building this, the introduction of adaptive trieval context selection (RCS) further filters out unreliable retrieved contexts, yielding an additional performance boost of 19.3% and 6.3% on the IU-Xray and Fair datasets. (3) The inclusion of RAG-based preference fine-tuning (RAG-PT) enhances the model's 's understanding of the retrieved knowledge, leading substantial performance gains of 37.1% and 16.9% on the respective datasets. This demonstrates that RAG-PT effectively addresses misalignment issues.
VQA
RG
VQA
RG
-LaVA-Med.11.
68.99
10.04
66.63
13.41
+DR
77.12
13:23
72.69
15.89
+RCS
79.56
17.92
75.74
17.22
+RAG-PT (Ours)
85,80
29.80
87.18
20.42
Impact of the Preference Data in RAG-PT. To better understand how RAG-PT mitigates the mis- alignment issue and improves performance, we con- ducted a detailed study the training preference data composition of RAG-PT, As described Sec- tion 3.3 the RAG-PT data is designed to address both cross modality alignment and overall align- ment objectives, with the latter focusing on hanced understanding of retrieved knowledge and minimizing retrieval interference. The detailed experimental results in Table 5 demonstrate that the preference data tailored for different alignment objectives positively impacts the model's perfor-
Table 5: Performance using RAG-PT based on subsets of preference data.
Model
IU-Xray VQA
Faid
RG
VQA
RG
LLaVA-Med-1.5
68.99
10.04
66.63
13.41
+RAG-PT
80.19
19.38
79.42
18.37
+RAG-PT2
80.27
20.16
79.35
18.66
+RAG-PT3
81.30
19.43
80.07
18.92
mance, showing the effectiveness of RAG-PT. Additional ablation analysis on preference data can be seen in Appendix A.6.6
How Effective is MMed-RAG in Mitigating Misalignment Issues? To gain more intuitive un- derstanding of the effectiveness of MMed-RAG in addressing misalignment issues: 1) we calculate the proportion of errors caused by RAG and compare it to the proportion after incorporating MMed- RAG. 2) We visualize the attention maps of image and text tokens with and without RAG-PT. First, as mentioned Section 3.3 the model may directly copy reference information, referred to as Copy-Reference (CR) rate. After applying MMed-RAG, as shown in Figure] 3 the CR rate drops to 28.19%. Additionally, the proportion of errors affected by RAG interference, referred to as Over- Reliance (OR) rate, which is initially 43.31% decreased to 8.38% after incorporating MMed-RA. -AG. Furthermore, as shown in Figure[4] the original Med-LVLM tends to rely more heavily on text while ignoring visual information. When retrieval information is introduced, the original Med-LV fo-

---

Published as a conference paper at ICLR 2025
cused more the retrieved answers, even if the content is incorrect. After RAG-PT, the model significantly increases its attention to visual information and reduces the interference RAG, thus better aligning the model's knowledge with the fundamental facts.
Image Tokens
Tokans
Basokra
Miees Dune) U.nnk-led-
Can focal arpppce aecieaatiin
Goen on image?
Merkeai Imaage
UN The soeeri
Reference
Ratio
Thuro Esposss
local
consol lnatonn connet
Figure 3: Aligoment analysis with and without RAG. OR:
Figh aide the king
be sean on the image
Over-Reliance CR: Copy-
Visualization of attention map. The red box region is labeled with the attentions that can be enhanced by MMed-RAG.
Figure 4:
Reference.
RELATED WORK
Factuality in Med-LVLMs. The rapid advancements in Large Vision-Language Models
(LVLMss) (Liu et al. 2024a b) are beginning to influence the field of medical image analysis. Several Med-LVLMs (Li al. [2023a] Moor et al. [023 Zhang al.] 20236 [Wu al. [222b Zhu et al.,
2024). have emerged, showing remarkable performance across different medical imaging modalities.
Despite these advances, Med-LVLMs continue to present notable factual hallucination (Xia al.)
2024a Royer et al., 2024), generating textual outputs that contradict medical visual information.
This raises concerns about potential misdiagnoses or overlooked conditions. Recently, benchmarks have been developed to assess the accuracy of Med-LVLMs in tasks such as visual question answer- ing (VQA) and report generation (Xia et al. 2024a Royer et al. 2024). However, research aimed at enhancing the factual accuracy of Med-LVL.M LMs remains relatively unexplored.
Retrieval Augmented Generation in Med-LVLMs. Retrieval-Augmented Generation (RAG) has proven to be a powerful technique for enhancing factual accuracy in language modeling (Gao al.)
2023 Wu et al. 2023c; [Cen et al. 2024c: Qu et al. 2024] Sun et al.] (024a). In the biomedi- cal domain, RAG leverages external knowledge to guide the generation of Med-LVLMs, offering clear advantages in tasks such as medical VQA and report generation Yuuan et al. 2023 Kumar
& Marttinen] 2024| Tao et al. 2024] He al. 2024 Sun al. (024b). However, these works mainly focus on enhancing the relevance of the retrieved contexts without considering the model's understanding of retrieved knowledge. There are several recent work RAG fine-tuning in LLMs. DPA-RAG (Dong et al., 2024) addresses the alignment issues between the external reranker and the internal LLM through supervised fine-tuning. Then RAG-DDR (Li al. 2024b) leverages rolling method to generate perturbed responses, further mitigating conflicts between parameter memory and external knowledge. In the biomedical domain, RULE (Xia al., 2024c) proposed to use preference fine-tuning to reduce the model's 's over-reliance on retrieved contexts. However, it still overlooks misalignment issues caused by RAG, as well as the generalizability of the retriever given the diverse domains of input images. In response, we propose MMed-RAC to mitigate these risks, enhancing the factuality of Med-LVLMs by addressing these overlooked factors. This can lead to a better cross-modality and overall alignment to enhance the understanding of retrieved knowledge and visual information, ensuring more consistent and reliable performance across tasks.
CONCLUSION
This paper introduces MMed-RAG, versatile multimodal RAG system designed to address the critical issue of factual hallucination in Med-LVI.Ms. MMed-RAG employs domain-aware re- trieval mechanism, adaptive calibration for selecting the optimal number of retrieved contexts, and RAG-based preference fine-tuning to improve both cross-modality alignment and overall alignment with the ground truth. These enhancements significantly boost the factual accuracy of Med-LVLM, Experimental results demonstrate the effectiveness of MMed-RAL -AAG enhancing factual accuracy across various imaging domains, underscoring its potential for reliable use in healtheare. Our find- ings underscore the importance of incorporating robust multimodal RAG mechanism to ensure that Med-LVLMMs serve dependable tools in clinical settings.
10

---

Published as a conference paper at ICLR 2025
ETHICS STATEMENT
This paper presents novel RAG-based approach to enhancing the factuality of Med-LVLMs. We have followed best practices in data collection, model design, and evaluation, ensuring adherence to privacy and ethical standards. All datasets used are sourced from publicly available medical datasets or collected with appropriate ethical considerations, including patient data anonymization. We adhere to principles of research integrity and transparency, and comply with all relevant regula- tions. We hope that our research will contribute to safer, more reliable Al-assisted medical tools and advance healthcare technology responsibly.
REPRODUCIBILITY STATEMENT
We have taken significant steps to ensure that our work is reproducible. All details regarding our pro- posed multimodal RAG system, including the domain-aware retrieval mechanism, adaptive retrieved context selection, and RAG-based preference fine-tuning strategy, are described comprehensively in Section
3 We provide the hyperparameter settings and experimental configurations used in our eval- uations Section 5.1 and Appendix Additionally, we have included detailed pseudocode for the proposed algorithms Algorithm| and an in-depth explanation of the data processing steps for each medical dataset used in Appendix A.1.1 and Appendix A.2
ACKNOWLEDGEMENT
This work is partially supported by Cisco Faculty Research Award.
REFERENCES
Asma Alkhaldi, Raneem Alnajim, Layan Alabdullatef, Rawan Alyahya, Jun Chen, Deyao Zhu, Ahmed Alsinan, and Mohamed Elhoseiny. Minigpt-med: Large language model as a general interface for radiology diagnosis. arXiv preprint arXiv:2407.04100 044400 2024.
Emily Alsentzer, John Murphy, Willie Boag. Wei-Hung Weng. Di Jin, Tristan Naumann, and Matthew McDermott. Publicly available clinical bert embeddings. arXiv preprint 1eXiv:1904.055 2019.
Jinze Bai, Shuai Bai, Shusheng Yang, Shijie Wang, Sinan Tan, Peng Wang, Junyang Lin, Chang Zhou, and Jingren Zhou. Qwen-vl: A versatile vision-language model for understanding, local- ization, text reading, and beyond. 2023.
Satanjeev Banerjee and Alon Lavie Metcor:
An automatic metric for mt evaluation with improved correlation with human judgments. In Proceedings of the acl workshop on intrinsic and extrinsic evaluation measures for machine translation and/or summarization, pp. 65-72, 2005.
Jiawei Chen, Dingkang Yang, Tong Wu, Yue Jiang, Xiaolu Hou, Mingcheng Li, Shunli Wang. Dongling Xiao, Ke Li, and Lihua Zhang. Detecting and evaluating medical hallucinations in large vision language models. arXiv preprint arXiv:2406.10185, 2024a.
Junying Chen, Ruyi Ouyang, Anningzhe Gao, Shunian Chen, Guiming Hardy Chen, Xidong Wang, Ruifei Zhang, Zhenyang Cai, Ke Ji. Guangjun Yu, et al. Huatuogpt-vision towards injecting medical visual knowledge into multimodal 1lms at scale, arXiv preprint arXiv:2406. 2024b.
Zhanpeng Chen, Chengjin Xu, Yiyan Qi, and Jian Guo. Milm is strong reranker: Advancing multimodal vetrieval-auggeene generation via knowledge-enhanced reranking and noise-injected training. arXiv preprint 1.XXi:22407.214399 2024c.
Zhe Chen, Weiyun Wang. Hao Tian, Shenglong Ye, Zhangwei Gao, Erfei Cui, Wenwen Tong, Kongzhi Hu, Jiapeng Luo, Zheng Ma, et al. How far are we to gpt-4v? closing the gap to com- mercial multimodal models with open-source suites. arXiv preprint 1iXi::2404.16822 2024d.
11

---

Published as a conference paper at ICLR 2025
Zixiang Chen, Yihe Deng, Huizhuo Yuan, Kaixuan Ji, and Quanquan Gu. Self-play fine-tuning converts weak language models to strong language models, 2024e. URL https://arxii... org/ abs/2401.0133 01335
Clément Christophe, Praveen K Kanithi, Tathagata Raha, Shadab Khan, and Marco AF Pimentel. Med42-v2: A suite of clinical llms. arXiv preprint arXiv:2408.06 00..2408.00612..
Yung-Sung Chuang, Yujia Xie, Hongyin Luo, Yoon Kim, James Glass, and Pengcheng He. Dola: Decoding by contrasting layers improves factuality in large language models. arXiv preprint arXiv:2309.03883, 2023.
Dina Demner-Fushmanl Mare D Kohli, Mare B Rosenman, Sonya E Shooshan. Laritza Rodriguez, Sameer Antani, George R Thoma, and Clement J McDonald. Preparing a collection of radiol- ogy examinations for distribution and retrieval. Journal of the American Medical Informatics Association, 23(2)304--10,, 2016.
Guanting Dong, Yutao Zhu, Chenghao Zhang, Zechen Wang, Zhicheng Dou, and Ji-Rong Wen. Understand what IIm needs: Dual preference alignment for retrieval-augmenned generation. arXiv preprint arXiv:2406. 18676, 2024.
Yunfan Gao, Yun Xiong, Xinyu Gao, Kangxiang Jia, Jinliu Pan, Yuxi Bi, Yi Dai, Jiawei Sun, and Haofen Wang- setrieval-augmente generation for large language models: A survey. arXiv preprint arXiv:2312.10997, 2023.
Pierre Gravel, Gilles Beaudoin, and Jacques A De Guise. A method for modeling noise in medical images. IEEE Transactions on medical imaging, 23(10):2211-2333 2004.
Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recog- nition. Proceedings the IEEE Conference on Computer Vision and Pattern Recognition, pp. 770-778, 2016.
Sunan He, Yuxiang Nic, Zhixuan Chen, Zhiyuan Cai, Hongmei Wang, Shu Yang, and Hao Chen. Meddr: Diagnosis-guided bootstrapping for large-scale medical vision-language learning. arXiv preprint 1tXiv:2404.15127, 2024.
Robbie Holland, Thomas RP Taylor, Christopher Holmes, Sophie Riedl, Julia Mai, Maria Patsia- manidi, Dimitra Mitsopoulou, Paul Hager, Philip Müller, Hendrik PN Scholl, et al. Specialist vision-language models for clinical ophthalmology. arXiv preprint arXiv:2407.0844
2024.
Edward Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang. Lu Wang, and Weizhu Chen. Lora: Low-rank adaptation of large language models. arXiv preprint arXiv:2106.09685, 2021.
Ming Hu, Peng Xia, Lin Wang, Siyuan Yan, Feilong Tang, Zhongxing Xu, Yimin Luo, Kaimin Song, Jurgen Leitner, Xuelian Cheng. et al. Ophnet: A large-scale video benchmark for ophthalmic surgical workflow understanding. In European Conference on Computer Vision, pp. 481-500. Springer, 2024.
Qidong Huang, Xiaoyi Dong, Pan Zhang, Bin Wang, Conghui He, Jiaqi Wang, Dahua Lin, Weiming Zhang, and Nenghai Yu. Opera: Alleviating hallucination in multi-modal large language models via over-trust penalty and retrospection-allocatioo arXiv preprint arXiv:231 31. 17911, 2023.
Wisdom Ikzzogo, Saygin Seyfioglu, Fatemeh Ghezloo, Dylan Geva, Fatwir Sheikh Mohammed, Pavan Kumar Anand, Ranjay Krishna, and Linda Shapiro. Quilt- 1m: One million image-text pairs for histopathology. Advances in neural information processing systems, 36, 2024.
Yue Jiang, Jiawei Chen, Dingkang Yang, Mingeheng Li, Shunli Wang, Tong Wu, Ke Li, and Lihua Zhang. Medthink: Inducing medical large-scale visual language models to hallucinate less by thinking more. arXiv preprint arXiv:2406.1 1451, 2024.
Baoyu Jing, Pengtao Xie, and Eric Xing. On the automatic generation of medical imaging reports. arXiv preprint arXiv: 101.1711..00 1711118195. 195, 2017.
12

---

Published as a conference paper at ICLR 2025
Alistair EW Johnson, Tom Pollard, Nathaniel R Greenbaum, Matthew P Lungren, Chih-ying Deng, Yifan Peng, Zhiyong Lu, Roger G Mark, Seth Berkowitz, and Steven Horng. Mimic-cxr-jpee a large publicly available database of labeled chest radiographs. arXiv preprint arXiv:1901.0704. 2019.
Lie Ju, Yukun Zhou, Peng Xia, Daniel Alexander, Pearse Andrew Keane, and Zongyuan Ge. Ex- plore langone model with hierarchical information for multiple retinal disease recogni- tion. Investigative Ophthalmology & Visual Science 65(7):1593-1595, 2024.
Yogesh Kumar and Pekka Marttinen. Improving medical multi-modal contrastive learning with expert annotations. arXiv preprint arXiv:2403. 10153, 2024.
Sicong Leng. Hang Zhang, Guanzheng Chen, Xin Li, Shijian Lu, Chunyan Miao, and Lidong Bing. Mitigating object hallucinations in large vision-language models through visual con- trastive decoding. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13872-13882, 2024.
Chunyuan Li, Cliff Wong, Sheng Zhang, Naoto Usuyama, Haotian Liu, Jianwei Yang, Tristan Nau- mann, Hoifung Poon, and Jianfeng Gao. Llava-med: Training large language-and-vision assis- tant for biomedicine in one day. In Thirty-seventh Conference on Neural Information Processing Systems Datasets and Benchmarks Track, 2023a.
Haoran Li, Junqi Liu, Zexian Wang, Shiyuan Luo. Xiaowei Jia, and Huaxiu Yao. Lite: Modeling environmental ecosystems with multimodal large language models. arXiv preprint arXiv:2404.01 165, 2024a.
Xinze Li, Sen Mei, Zhenghao Liu, Yukun Yan, Shuo Wang, Shi Yu, Zheni Zeng, Hao Chen, Ge Yu, Zhiyuan Liu, et al. Rag-ddr: Optimizing retrieval- -augmeeed generation using differentiable data rewards. arXiv preprint arXiv:2410. 13509, 2024b.
Yingshu Li, Yunyi Liu, Zhanyu Wang, Xinyu Liang, Lingqiao Liu, Lei Wang. Leyang Cui, Zhaopeng Tu, Longyue Wang, and Luping Zhou. A comprehensive study of gpt-4v's multimodal capabilities in medical imaging. arXiv preprint arXiv:2310.2033 2023b.
Chin-Yew Lin. Rouge: A package for automatic evaluation of summaries. In Text summarization branches out, pp. 74-81, 2004.
Weixiong Lin, Ziheng Zhao, Xiaoman Zhang, Chaoyi Wu, Ya Zhang, Yanfeng Wang, and Weidi Xie. Pme-clip: Contrastive language-image pre-training using biomedical documents. In International Conference on Medical Image Computing and Computer-Assise Assisted Intervention, pp. 525-536.
Springer, 2023a.
Zhihong Lin, Donghao Zhang, Qingyi Tao, Danli Shi, Gholamreza Haffari, Qi Wu, Mingguang He. and Zongyuan Ge. Medical visual question answering: A survey. Artificial Intelligence in Medicine, 143:102611, 2023b.
Haotian Liu, Chunyuan Li, Yuheng Li, and Yong Jae Lee. Improved baselines with visual in- struction tuning. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. -2336,, 2024a.
Haotian Liu, Chunyuan Li, Qingyang Wu, and Yong Jae Lee. Visual instruction tuning. Advances in neural information processing systems, 36, 2024b.
Yan Luo, Min Shi, Muhammad Osama Khan, Muhammad Muneeb Afzal, Hao Huang, Shuaihang Yuan, Yu Tian, Luo Song, Ava Kouhana, Tobias Elze, et al. Fairclip: Harnessing fairness in vision-language learning. arXiv preprint urXiv:2403.11999 2024.
Fanqing Meng, Jin Wang, Chuanhao Li, Quanfeng Lu, Hao Tian, Jiaqi Liao, Xizhou Zhu, Jifeng Dai, Yu Qiao, Ping Luo, et al. Mmiu: Multimodal aulti-image understanding for evaluating large vision-language models. arXiv preprint arXiv:2408.0211 027118, 2024.
Michael Moor, Qian Huang, Shirley Wu, Michihiro Yasunaga, Yash Dalmia, Jure Leskovee, Cyril Zakka, Eduardo Pontes Reis, and Pranav Rajpurkar. fed-fflaingo: a multimodal medical few- shot learner. In Machine Learning for Health (ML4H), pp. 353-367. PMLR, 2023.
13

---

Published as a conference paper at ICLR 2025
OpenAl. Gpt-4 technical report, 2023. http:://arakv.org/aa///BB/
Kishore Papineni, Salim Roukos, Todd Ward, and Wei-Jing Zhu. Bleu: a method for automatic evaluation of machine translation. In Proceedings of the 40th annual meeting of the Association for Computational Linguistics, pp. 311-318, 2002.
Xiaoye Qu, Qiyuan Chen, Wei Wei, Jishuo Sun, and Jianfeng Dong. Alleviating halluci- nation large vision-language models with active retrieval augmentation. arXiv preprint arXiv:2408.0055 2024.
Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agar- wal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, Gretchen Krueger, and Ilya Sutskever. Learning transferable visual models from natural language supervision, 2021.
Rafael Rafailov, Archit Sharma, Eric Mitchell, Christopher D Manning, Stefano Ermon, and Chelsea Finn. Direct preference optimization: Your language model secretly a reward model. In seieth Conference Neural Information Processing Systems, 2023.
Machel Reid, Nikolay Savinov, Denis Teplyashin, Dmitry Lepikhin, Timothy Lillicrap, Jean- baptiste Alayrac, Radu Soricut, Angeliki Lazaridou, Orhan Firat. Julian Schrittwieser. et al. Gem- ini 1.5: Unlocking multimodal understanding across millions of tokens of context. arXiv preprint arXiv:2403.05530, 2024.
Carlos Riquelme, Joan Puigcerver, Basil Mustafa, Maxim Neumann, Rodolphe Jenatton, André Susano Pinto, Daniel Keysers, and Neil Houlsby. Scaling vision with sparse mixture of experts. Advances in Neural Information Processing Systems, 85955338595,,, 2021.
Corentin Royer, Bjoem Menze, and Anjany Sekuboyina. Multimedeval:: A benchmark and toolkit for evaluating medical vision-language models. arXiv preprint arXiv:2402.09262, 2024.
Ma Guadalupe Sanchez, Ma Guadalupe Sánchez, Vicente Vidal, Gumersindo Verdu, Gumersindo Verdúú Patricia Mayo, and Francisco Rodenas. Medical image restoration with different types of noise. In 2012 Annual International Conference of the IEEE Engineering in Medicine and Biology Society, pp. 4382-4385. IEEE, 2012.
Jessica Schrouff, Natalic Harris, Sanmi Koyejo. Ibrahim M Alabdulmohsin, Eva Schnider, Krista Opsahl-Ong, Alexander Brown, Subhrajit Roy, Diana Mincu, Christina Chen, et al. Diagnosing failures of fairness transfer across distribution shift in real-world medical settings. Advances in Neural Information Processing Systems, 35: 13304-99931,8 2022.
Mehmet Saygin Seyfioglu, Wisdom O Ikezogwo, Fatemeh Ghezloo, Ranjay Krishna, and Linda Shapiro. Quilt-llava: Visual instruction tuning by extracting localized narratives from open-source histopathology videos. In Proceedings of the IEEE/CVF Conference Computer Vision and Pattern Recognition, pp. 13183-13192, 2024.
Noam Shazeer, Azalia Mirhoseini, Krzysztof Maziarz, Andy Davis, Quoc Le, Geoffrey Hinton, and Jeff Dean. Outrageously large neural networks: The sparsely-gated mixture-of-experts layer. arXiv preprint arxiv: 1701 106588638, 2017.
Congzhen Shi, Ryan Rezai, Jiaxi Yang, Qi Dou, and Xiaoxiao Li. A survey on trustworthiness in foundation models for medical image analysis. arXiv preprint arXiv:2407.15851 2024.
Jiashuo Sun, Jihai Zhang, Yucheng Zhou, Zhaochen Su, Xiaoye Qu, and Yu Cheng. Surf: Teach- ing large vasion-language models to selectively utilize retrieved information. arXiv preprint arXiv:2409, 14083, 2024a.
Liwen Sun, James Zhao, Megan Han, and Chenyan Xiong. Fact-aware multimodal retrieval aug- mentation for accurate medical radiology report generation. arXiy preprint MxXi:24407.15666 2024b.
Ilya Sutskever, Oriol Vinyals, and Quoc V Le. Sequence to sequence learning with neural networks. In Advances in neural information processing systems, pp. 3104-3112, 2014.
14

---

Published as a conference paper at ICLR 2025
Yitian Tao, Liyan Ma, Jing Yu, and Han Zhang. Memory- cross- moos- semantic alignment network for radiology report generation. IEEE Journal of Biomedical and Health Informatics, 2024.
Alexandra- Maria Täufan, Bogdan Ionescu, and Emiliano Santarnecchi. Artificial intelligence in neu- rodegenerative diseases: A review of available tools with a focus on machine learning techniques. Artificial Intelligence Medicine, 1 11::102081, 2021.
Omkar Thawkar, Abdelrahman Shaker, Sahal Shaji Mullappilly, Hisham Cholakkal, Rao Muham- mad Anwer, Salman Khan, Jorma Laaksonen, and Fahad Shahbaz Khan. Xraygpt: Chest radio- graphs summarization using medical vision-language models. arXiv preprint arXiv:2306.07971, 2023.
Robert Tibshirani, Guenther Walther, and Trevor Hastie. Estimating the number of clusters in data set via the gap statistic. Journal of the Royal Statistical Society; Series B (Statistical Methodology), 63(2):111 1--411 2001.
Hugo Touvron, Louis Martin, Kevin Stone, Peter Albert, Amjad Almahairi, Yasmine Babaei, Niko- lay Bashlykov, Soumya Batra, Prajjwal Bhargava, and Shruti Bhosale. Llama Open foundation and fine-tuned chat models. arXiv preprint arXiv:2307.0922 099288, 2023.
Tao Tu, Shekoofeh Azizi, Danny Driess, Mike Schaekermann, Mohamed Amin, Pi-Chuan Chang, Andrew Carroll, Charles Lau, Ryutaro Tanno, Ktena, al. Towards generalist biomedical ai. NEJM AI, (3a:23o23001138 2024.
Chunhao Wang, Xiaofeng Zhu, Julian Hong, and Dandan Zheng. Artificial intelligence in radio- therapy treatment planning: present and future. Technology in cancer research & treatment, 18: 1533033818873922, 2019.
Xiyao Wang, Yuhang Zhou, Xiaoyu Liu, Hongjin Lu, Yuancheng Xu, Feihong He, Jachong Yoon, Taixi Lu, Gedas Bertasius, Mohit Bansal, et al. Mementos: A comprehensive benchmark for mul- timodal large language model reasoning over image sequences. arXiv preprint arXiv:2401.10529, 2024.
Zhen Wang, Mingxiao Li, Peng Xia, Chao Jiang, Ting Shen, Jiaming Ma, Yu Bai, Suhui Zhang, Yiwei Lai, Sitong Li, et al. Screening cognitive impairment in patients with atrial fibrillation: deep learning model based on retinal fundus photographs. Heart Rhythm O2, 2025.
Zhepei Wei, Wei-Lin Chen, and Yu Meng. Instructrag: Instructing retrieval- aurieval-auggented generation with explicit denoising. arXiv preprint arXiv:2406. 2024.
Chaoyi Wu, Jiayu Lei, Qiaoyu Zheng, Weike Zhao, Weixiong Lin, Xiaoman Zhang, Xiao Zhou, Ziheng Zhao, Ya Zhang, Yanfeng Wang, et al. Can gpt-4v (ision) serve medical applications? case studies on gpt-4v for multimodal medical diagnosis. arXiv preprint arXiv:2310.09900 2023a.
Chaoyi Wu, Xiaoman Zhang, Ya Zhang, Yanfeng Wang, and Weidi Xie. Towards generalist foun- dation model for radiology. arXiv preprint mXXv:2308.022663 2023b.
Yuanhao Wu, Juno Zhu, Siliang Xu, Kashun Shum. Cheng Niu, Randy Zhong. Juntong Song, and Tong Zhang. Ragtruth: A hallucination corpus for developing trustworthy oetrieall-augment language models. arXiv preprint arXiv:2401 .,002690,.,036,, 2023c.
.00396,
Peng Xia, Ze Chen. Juanxi Tian, Yangrui Gong, Ruibo Hou, Yue Xu, Zhenbang Wu, Zhiyuan Fan, Yiyang Zhou, Kangyu Zhu, et al. Cares: comprehensive benchmark of trustworthiness in medical vision language models. arXiv preprint arXiv:2406.06007 2024a.
Peng Xia, Ming Hu, Feilong Tang, Wenxue Li, Wenhao Zheng, Lie Ju, Peibo Duan, Huaxiu Yao, and Zongyuan Ge. Generalizing unseen domains diabetic retinopathy with disentangled representations. In arXiv preprint wrXiv:2406.06334 2024b.
Peng Xia. Kangyu Zhu, Haoran Li. Hongtu Zhu, Yun Li, Gang Li, Linjun Zhang, and Huaxiu Yao. Rule: Reliable multimodal rag for factuality in medical vision language models. arXiv preprint arXiv.2407.05131. 2024c.
15

---

Published as a conference paper at ICLR 2025
Qing Ye, Chang- Yu Hsich, Ziyi Yang, Yu Kang, Jiming Chen, Dongsheng Cao, Shibo He, and Tingjun Hou. A unified drug target interaction prediction framework based on knowledge graph and recommendation system. Nature communications, 12(():6775, 2021.
Zheng Yuan, Qiao Jin, Chuanqi Tan, Zhengyun Zhao, Hongyi Yuan, Fei Huang, and Songfang Huang. Ramm: Retreeval-augmented biomedical visual question answering with multi-modal pre-tree vre-training. In Proceedings of the 3 3sst ACM International Conference on Multimedia, pp. 547- 556, 2023.
Jihai Zhang, Xiaoye Qu, Tong Zhu, and Yu Cheng. Clip-moe: Towards building mixture of experts for clip with diversified multiplet upeycling. arXiv preprint arXiv:2409.19291 2024.
Sheng Zhang, Yanbo Xu, Naoto Usuyama, Hanwen Xu, Jasprect Bagga, Robert Tinn, Sam Pre- ston, Rajesh Rao, Mu Wei, Naveen Valluri, et al. Biomedclip: multimodal biomedical foundation model pretrained from fifteen million scientific image-text pairs. arXiv preprint arXiv:2303.00915, 2023a.
Xiaoman Zhang, Chaoyi Wu, Ziheng Zhao, Weixiong Lin, Ya Zhang, Yanfeng Wang, and Weidi Xie. Pmc-vqa: Visual instruction tuning for medical visual question answering. arXiv preprint DXXi:2335510015, 2023b.
Kangyu Zhu, Peng Xia, Yun Li, Hongtu Zhu, Sheng Wang, and Huaxiu Yao. Mmedpo: Aligning medical vision-language models with clinical-aware multimodal preference optimization. arXiv preprint arXiv:2412.06141, 2024.
A EXPERIMENT
A.I EXPERIMENTAL SETUP
A.1.1 DATA STATISTICS
The data quantities used in this study are presented Table Table 7 and Table We clarify that for training the retriever, the data refers to the number of image-text text pairs, while for fine-tuning, refers to the number QA items. The "All" category represents the total amount of data used construct the preference dataset for RAG-PT. The training of RAG-PT includes three types of samples: (a) clean samples with originally correct answers that remain correct even after adding noise the images, (b) clean image samples with originally incorrect answers that become correct, and (c) clean image samples with originally correct answers that become incorrect.
Table Data statistics for medical VQA task. "Train (DR)" refers to the number of image-text pairs for retriever training, "All (RAG-PT)" refers to the total data for RAG-PT, and "Train (RAG-PT)- a/b/c" refer to the respective subsets for RAG-PT training.
Dataset
Train (DR)
All (RAG-PT)
Train (RAG-PT)-a
Train (RAG-PT)-b
Train (RAG-PT)-c
Ophthalomology
7000
3247
1082
1030
1135
Radiology
4034
4836
1612
1989
1235
Pathology 5000
1990
663
523
804
Table Data statistics for report generation. "Train (DR)" refers the number of image-text pairs for retriever training, "All (RAG-PT)" refers to the total data for RAG-PT, and "Train (RAG-PT)- a/b/c" refer to the respective sample categories for RAG-PT training.
Dataset
Train (R)
All (RAG-PT)
Train (
Train (RAG-PT)-b
Train (RAG-PT)-c
Ophthalmology
7000
3247
142
78
207
Radiology
4034
4836
126
342
16

---

Published as a conference paper at ICLR 2025
Table Data statistics for various datasets. The rows represent the number of images and QA pairs for each dataset.
Harvard- Farradd-FarrIMMMeee
IU-Xray
MIMIC-CXR
PMC-OA
Quilt-I
Images
713
589
700
530
559
# QA Items
4285
2573
3470
3124
1994
A.1.2 HYPERPARAMETER SETTINGS
Following the settings of CLIP (Radford et al. 2021 we adopt the same architecture and hyperpa- rameters for the vision and encoders. The vision encoder is ResNet-50 (He al. 2016). and the text encoder is a bio-bert-baad bert-basedddd model (Alsentzer et al. 2019). We use the Adam W optimizer with a learning rate of 10-4 and a batch size of 512. The model is trained for 360 epochs. For the first phase, we trained for 3 epochs, and for the second phase, the training was conducted for epoch. Training for 20 hours on one 100 80G GPU. For the RAG-PT phase, we adjust the diffusion noise level, symbolized by through specific formula: & Sigmoid (0.5 10 10- + 10-5, where is drawn from normal distribution. The reports available for retrieval are from the training set of the corresponding dataset. In experiments, parameters with grid search. All the experiments are implemented on PyTorch 2.1.2 using four NVIDIA RTX A6000 GPUs. It takes roughly and hours for fine-tuning CL.IP and LL-VA-Med- 1.5 7B, respectively.
apply cross-vass volsdvanio to tune all hyper-
A.2 EVALUATED DATASETS
We utilize five open-source medical vision-language datasets, i.e., MIMIC-CXR (Johnson et al. 2019). IU-Xray (Demner- et al.] (0166). Farvard- (Luo et al. 2024), PMC- OA (Lin et al. 2023a) and Quilt-IM (Ikezogwo et al. 2024).
2019 is a large publicly available dataset of chest X-ray images in DICOM format with associated radiology reports.
MIMIC-CXR (Johnson et al.
IU-Xray (Demner-Fushmal et al. 2016) is a dataset that includes chest X-ray images and corre-
sponding diagnostic reports.
Harvard-Fair VLMedd (Luo et al. 2024) focuses faimess in multimodal fundus images, taining image and text data from various sources. It aims to evaluate bias Al models on this multimodal data comprising different demographics.
PMC-OA (Lin al. 2023a) is large-scale dataset comprising figure-caption pairs extracted from PubMed Central. It covers 2,478,267 papers and includes a total of 12,211,907 11,907 figure-caption pairs. We only use the pathology subset filtered by GPT-4 based on captions.
Quilt-IM [Ieeoowoo et al. 2024) is the largest vision-language dataset histopathology, contain- ing 1 million image-text pairs sourced from platforms such YouTube, Twitter, research papers, and other parts the internet.
A.3 EVALUATED MODELS
We evaluate five open- source Med-LVLMs. i.e., LLaVA-Med (ii et al. 2023a). Med- Flamingo (Moor 2023). MedVInT (Zhang al.) (0236), RadFM Wu al. 2023b), miniGPT-
Med (Alkhaldi et al. 2024). The selected models are all at the 7B level.
LLaVA-Med (Li et al. 2023a) is a vision- conversational assistant, adapting the general- domain LLa VA (Liu (004b)) model for the biomedical field. The model fine-tuned using a novel curriculum learning method, which includes two stages: aligning biomedical vocabulary with figure-caption pairs and mastering open-ended conversational semantics. demonstrates excellent multimodal conversational capabilities.
Med-Flaming MMor al. 2023) is multimodal few-shot leamer designed for the medical domain. It builds upon the OpenFlaminge continuing pre-trate rre-training with medical image-text data from publications and textbooks. This model aims to facilitate few-shot generative medical visual
17

---

Published as a conference paper at ICLR 2025
question answering, enhancing clinical applications by generating relevant responses and ratio- nales from minimal data inputs.
RadFM (Wu et al. 2023b) serve versatile generalist model in radiology, distinguished by its capability to adeptly process both 2D and 3D medical scans for a wide array of clinical tasks.
It integrates ViT as visual encoder and perceiver module, alongside the MedLLaMA language model, to generate sophisticated medical insights for variety of tasks. This design allows RadFM to not just recognize images but also to understand and generate human-like explanations.
MedVInT (Zhang et al. 2023b), which stands for Medical Visual Instruction Tuning, is designed to interpret medical images by answering clinically relevant questions. This model features two variants align visual and language understanding: MedVInT-TTE and MedVIn'-TDD. Both Med- VInT variants connect a pre-trained vision encoder ResNet-50 adopted from PMC-CLIP (Lin) ct al. 2023a which processes visual information from images. It an advanced model that leverages a novel approach to align visual and language understanding.
miniGPT-Med (Alkhaldi et al. 2024) is a vision-language model derived from large-scale lan- guage models and tailored for radiology diagnosis applications. It handles various medical vision- language task using distinct task identifiers, demonstrating advanced performance in disease grounding, medical report generation, and medical VQA.
A.4 OVERVIEW OF THE BASELINES
We compare MMed-RAG with two types of LVLM hallucination mitigation methods that show promising results in natural image understanding. 1) Decoding-based methods, including Greedy Decoding, Beam Search (Sutskever et al. 2014). DoLa (Chuang et al. 2023). OPERA (Huang et al.) 2023), VCD (Leng al.] 2024). These methods manipulate the logits of the model's output tokens to enhance factual accuracy. 2) Multimodal RAG RAG-based methods, including MedDr (He et al., 2024), FactMM-RAG (Sun et al. 2024b), RULE (Xia al., 2024c).
Greedy decoding involves selecting the most probable next token at each step of generation. While it is efficient and straightforward, it can lead to suboptimal outcomes by getting stuck in repetitive less creative patterns.
Beam search (Sutskever et al. (2144) expands on greedy decoding by maintaining multiple candi- date sequences (or "beams") ") at each step, allowing for broader exploration of possible outputs. This approach balances quality and diversity by selecting the top-k sequences based on their prob- abilities, resulting in more coherent and creative text generation compared to greedy decoding.
DoLa (Chuang et al. 2023) derives the next-token distribution by contrasting the logits projected from later layers against those from earlier layers, leveraging the fact that factual knowledge in LLMs is typically localized within specific transformer layers.
OPERA (Huang et al. (223)) is LVLMs decoding method based on an Over-trust Penalty and Retrospection-Allocaie strategy The key insight that hallucinations are closely tied knowl- edge aggregation patterns in the self-attention matrix, where MILMS tend to focus on summary tokens, neglecting image tokens and resulting in content hallucination.
VCD [Leng al. 2024 is decoding method that tackles the object hallucination issue in LVLMs. contrasts output distributions derived from original and distorted visual inputs to cal- ibrate the model's output without the usage of external tools, reducing the the over-reliance on statistical bias and unimodal priors.
MedDr (He et al. 2024) is healthcare foundation model built upon generated diagnosis-based datasets, demonstrating advanced capabilities in various data modalities. Meddr also integrates retrieval-augmentted medical diagnosis strategy during inferencing to enhance factual accuracy.
FactMM-RAG (Sun al. 2024b) is fact-aware multimodal atrreval-augmentee pipeline for radiology report generation. It utilize RadGraph to annotate chest radiograph reports and mine clinically relevant pairs to train a universal multimodal retriever.
RULE (Xia et al. 2024c) is an advanced medical retrieval-augmenter generation strategy de- signed to enhance the factuality of Med-LVLMs. First, it introduces a robust strategy for control- ling factuality risk through the calibrated selection of retrieved contexts. Second, RULE develops a preference optimization strategy to balance Med-LVLMs intrinsic knowledge and the retrieved information.
18

---

Published as a conference paper at ICLR 2025
Instruction [Round]]
You are professional medical expert. I will provide you with some medical reports. Please gen- erate some questions with answers (the answer should be yes no) based the provided report. The subject of the questions should be the medical image patient, not the report.
Below are given report:
[REPORT]
Instruction [Round2]
Please double-check the questions and answers, including how the questions asked and whether the answers are correct. You should only generate the questions with answers no other unnec- essary information.
Below are the given report and QA pairs in roundl:
[REPORT] IQA PAIRS RIJ
Table The instruction GPT-4 for generating QA pairs.
A.5 PROMPTS
We convert the medical reports into series of closed-ended questions with yes no answers. To ensure the quality of the VQA data, we perform a round of self-checks using GPT-4 (OpenAl 2023). Finally, we conduct round of manual filtering to remove questions with obvious issues those related to multiple images or patient histories. The prompt templates used are shown in Table g
A.6 ADDITIONAL RESULTS
A.6.1 COMPATIBILITY ANALYSIS
To demonstrate the compatibility of our approach across different backbone models, apply it to LLa LLLVA-MM Med--..... As shown in Table our method delivers an average improvement of 40.3% over the original LLaVA-Med-LL 1.0, further highlighting its effectiveness in enhancing RAG performance and its adaptability to various backbones. MMed-RAG can be transferred to different Med-LVLMs, yielding consistent improvements across various domains, demonstrating the compatibility of our method.
Table 10: Performance on different backbones.
Model
IU-Xray VQA
Fair
RG
VQA
RG
LLaVA-Med-L.0
61.73
8.74
59.54
10.59
+MMed-RAG
80.32
22.63
78.49
15.88
A.6.2 DETAILED RESULTS OF OTHER LVLMS
As shown in Table 11 we conduct a comparison of several general LVLMS and other Med-LVLMSs. including GPT-4o OPenAA,, 2023), Gemini-1.5 (Reid et al., 2024). QwenVL. (Bai al., 2023). LLaVA-1.6 (Liu et al.) 2024b), and Intern VL-2 (Chen et al. 2024d). Our findings show that MMed- RAG consistently outperforms these models, further demonstrating its effectiveness.
A.6.3 COMPARISON WITH DOMAIN-SPECIFES MED-LVLMS AND THEM WITH RAG-PT
We conduct experiments to compare our method with domain-specinee Med-LVLMs as follows: Radiology: RadFM (Wu et al. 2023b), Pathology: Quilt-LLaaVA (Seyfioglu et al.) 2024 Ophthal- mology: Retina VLM (Holland et al.) 2024). For radiology, we use the IU-Xray dataset to evaluate VQA. For pathology, we use the PMC-OA pathology subset to evaluate VQA. For ophthalmology, since the domain-specific Med-LVLM, i.e.. RetinaVLM. is only trained report-generation tasks, we use the Sarvard-FairVLLne dataset to evaluate report generation. As shown Table 12 our method significantly outperforms each domain-specific Med-LVLM. Additionally, apply RAG- PT to each domain-specae Med-LVLM.. As shown in Table [2] after incorporating RAG-PT, the performance of these models improve significantly, demonstrating the compatibility of our method.
19

---

Published as a conference paper at ICLR 2025
Table 11: Accuracy (%) different Med-LVLMs based on LLaVA-Med-1.5 on medical VQA task.
Radiology
Ophthalmology Harvand-FairVLMar
Pathology
Models
IU-Xray
MIMIC-CX
Quilt-IM
PMC-OA (Pathology)
LLaVA-Med-Ia,
75,47
75.79
63.03
62,80
59,28
MMed-RAG
89.54
83,57
87.94
72.95
64.54
Med-Flamingo
26.74
61.27
42.06
27.11
32.62
MedVInT
73.34
66.06
35.92
26.81
27.77
RadPM
26.67
69.30
52.47
27.02
25.12
miniGPT-Med
54.87
$3.92
66.73
26.82
27.03
GPT-4o
63.25
60.61
61.50
53.56
49.70
Gemini- 1.5
59.73
61.02
58.53
56.88
52.17
VLv
58.05
63.70
48.52
35.73
38.54
Qwen VL-Chat
59.43
60.43
38.06
28.74
29.53
VL.-2
54.06
59.47
44.38
37.82
34.40
Furthermore, domain-specific Med-LVLMs could outperform generalist Med-LVLMs in their spe- cialized domains, as they are fine-tuned using specialized medical domain data. While this signifi- cantly enhances their medical understanding in specific domains, it may reduce their generalization ability, such as their capacity to comprehend retrieved information. Consequently, even after incor- porating RAG-PT, the performance of several domain-specifie Med-LVLMs (e.g.. RetinaVLM and RadFM) is inferior to MMed-RAG.
Table 12: Model performance comparison with domain -speifcc Med-LVLMs.
Radiology
Pathology
Ophthalmology
Model
Acc
F1
AUC
Acc
Fl
AUC
BLEU
ROUGE-L
METEOR
RadFM
26.67
30.36
55.31
RAG-PT
48.39
39.40
59.70
Quilt-LLaVA
62.59
72.30
56.96
RAG-PT
64.72
73.36
61.39
RetinaVLM
19.96
12.73
13.52
RAG-PT
22.26
14.64
16.87
LLaVA-Med-Il
75.47
64.04
59.28
71.98
$4.19
10.75
MMed-RAG
84.10
71.92
86.40
64.54
73.09
61.42
24.82
16.59
19.85
A.6.4 RESULTS ON OTHER DOMAIN
We apply RAG-PT to one additional domain (i.c., environmental ecosystems modeling) to further validate the effectiveness RAG-PT. We conduct experiments on two environmental system model- ing datasets (Li et al. 2024a). The CRW-Temp dataset is river water temperature prediction dataset aimed at forecasting the daily average water temperature of a specific day based on observed phys- ical variables. The CRW-Flow dataset focuses on predicting river segment flow based observed physical variables. The model used LITE (Li et al.) 2024a), an environmental system large model based on LLaMA2 (Touvron et al. 2023). We train semantic time-series encoder using time-series information- pairs, which works in conjunction with a text encoder as the retriever. Then we re- trieve the most similar environmental descriptions based the current environmental descriptions. As shown Table [3] our approach demonstrates significant performance improvements on tasks this domain.
A.6.5 STATISTICS OF COPY-REFERENT RATE AND LVLMs.
RATE FOR MORE
we conduct two alignment analysis tests multiple open-source Med-LVLMs and commercial LVLMs using the Harvard-Fair VLMed dataset with the incorporation of retrieved information. These tests respec- tively evaluate (1) cross- modality alignment and (2) overall alignment with the ground truth. As
Following the alignment analysis method we apply to LLaVA-Med-LE in Section 3.3
20

---

Published as a conference paper at ICLR 2025
Table 13: Performance comparison of different models on CRW-Temp and CRW-Flow datasets.
CRW-Temp
CRW-Flow
Model
RMSE
MAE
RMSE
MAE
LITE (Li et al. 2024a)
2.02
1.70
2.39
1.02
+RAG
1.93
1.62
2.27
0.96
+RAG-PT
1.74
1.46
2.11
0.90
shown in Table 14 the results indicate that both existing open-source Med-LVI.Ms and commer- cial LVLMs exhibit misalignment issues with retrieved information. In addition, it is worthwhile mention that GPT-4o demonstrates the best alignment performance compared with other models when incorporating RAG, especially in cross-modal alignment. This is likely because GPT-4o has been well-trained in visual perception and may also have utilized some post-training methods (like preference optimization) to optimize modal alignment.
Table 14: Comparison of Copy- Sepy-Reference Rate and Over-Reliance Rate across different models.
Model
Copy-Reference Rate
Over-Reliance Rate
LLa VA-dM--
55.08
43.31
Med-Flaminn Med-Flamingo
60.17
33.74
miniGPT-Med
56.75
46.06
GPT-4o
12.54
24.80
A.6.6 DETAILED ABLATION ANALYSIS
Preference data designed for different alignment objectives can indeed produce varying effects. Therefore, conducting ablation experiments combinations of different types of preference data necessary. We perform comprehensive ablation experiments on RAG-PT 1/2/3 as well as their combinations (RAG-PT 1+2, 2+3, 1+3) analyze the effectiveness of each type of data and their combinations. We find that the combination of 1+3 produced the most significant results, indicat- ing that the two misalignment issues (i.e., cross- cross-modality and over-reliance issues) are the most prominent. Targeted mitigation of these two issues yielded the greatest improvement. However, incorporating data for all three alignment objectives yields the best performance, demonstrating the importance of each alignment component.
Table 15: Ablation results using RAG-PT based on subsets of preference data.
Model
IU-Xray VQA
Harvard-FairVI.Mer
RG
VQA
RG
LLLVA-Med-1.5
68.99
10.04
66.63
13.41
+RAG-PT I
80.19
19.38
79.42
18.37
+RAG-PT
80.27
20.16
79.35
18,66
+RAG-PT
81.30
19.43
80.07
18.92
+RAG-PT 142
82.58
22.74
82.08
18,97
+RAG-PT 1+3
82,96
24.50
82.87
19.22
+RAG-PT 243
83.61
25.77
83.89
19.30
+RAG-PT 1+243
85.58
29.69
87.02
20.31
A.6.7 EXTERNAL VALIDATION
Considering the risk of overfitting, we use external validation datasets from the same domain to evaluate the generalizability of MMed-RAG, We select two domain-spectin subsets from PubMed- Vision (Chen et al. 2024b), i.e., fundus digital photography and microscopy image, for ophthalmol- ogy and pathology, respectively. The results show that MMed-RAG still significantly outperforms other Med-LVLMS on the external validation datasets, indicating MMed-Ra MAG- performs well when generalized external datasets, demonstrating its strong generalization capability.
21

---

Published as a conference paper at ICLR 2025
Table 16: Performance comparison of models on external validation datasets.
Ophthalmology
Pathology
Model
BLEU
ROUGE-L
METEOR
Acc
F1
AUC
LLAVA-Med-1.5
17.11
20.05 14.98
17.09
59.65
71.90
54.87
MMed-RAG
22.64
17.85
62.88
72.24
59.69
A.6.8 DETAILED BLEU SCORE
We report the average BLEU score above. Detailed results are provided in Table 17
Table 17: BLEU Score (%) of different methods based on LLa VA-M-dee- on report generation
task.
Radiology
Ophinalmology
Models
TU-Xra
MIMIC-CXR
Hurvard-Fairr1
BLBU-1
BLEU-3
BLBBU2
BLBU-3
BLEU-2
BLEU33
LLaVA-Med-13
17.09
10.55
6.47
3,83
21.82
13.35
6.11
3.64
32.57
19.36
9.11
5.38
Greedy
21.04
12.57
5.75
3.35
29.91
18.25
8.27
5.05
32.40
19.82
9.04
5:37
Beum Sourch
21.78
12.71
6.05
3.63
30.55
17.79
8.49
5.09
33.07
19.14
9.14
5.48
TolLa
21.22
3.54
30.80
5.15
19.02
OPPEXA
3,13
27.27
15.76
4.51
8.39
MedDe
22.37
12.99
6.19
3.71
33.43
19.33
9.22
5.53
35.64
20
9.82
$.89
FactMM RAG
26.45
15:25
7,26
4.36
33.64
19.84
9.27
5.56
37.47
21.64
10.30
6.18
RULE
69.56
28.61
13,62
8.17
33.47
19.36
9.23
5.54
23.26
11.08
6.66
11.52
A.6.9 DEEPER ANALYSIS OF RETRIEVER
We have tried training general retriever by mixing images from all modalities together, instead of using a domain- speciifc retriever. We conduct experiments based on BiomedCLIP and MedCLIP, but the results are unsatisfactory. Then we adopt MoF (Mixture of Experts) architecture (Shazeer al. 2017 Riquelme et al., (02T). Based on CLIP-MoF, we fine-tune CLIP-MoF: (Zhang et al. 2024 with mixing images from all medical imaging modalities, but the performance is still subop- timal. This might be because CLIP-MoE is not pretrained on large-scale biomedical data. All the ultimately adopt domain-
results reported in Table 18 Considering model performance, specific retriever architecture. In fact, this approach is both flexible and scalable. Similar to a general retriever, encountering completely new modality may still require retraining the retriever to achieve good retrieval performance, which incurs additional costs. For mixed datasets, as the number of modalities increases, training general retriever becomes increasingly challenging, mak- ing it difficult to achieve reliable retrieval within single domain. We address this by using domain identification module to classify the input image by modality and select the corresponding retriever. In the future, potential solution could involve pretraining general retriever on large-scale biomed- ical data using Mixture of Experts (MoE) architecture explore whether it possible develop general retriever.
Table 18: Performance comparison based on different retrievers.
IU-Xray
Model
Acc
F1
AUC
LLaVA-Med
75.47
64.04
67.46
RAG BBiomedCTIP-PP
79.09
65.87
69.52
RAG (MedCLIP-FT)
75.13
63.88
67.16
RAG (CLIP-MoE-FT)
72.13
62.72
65.11
RAG (Ours)
84.82
68.85
77.54
22

---

Published as a conference paper at ICLR 2025
A.6.10 COMPARISON UNDER FEW-SHOT SETTING
All our experiments are conducted under zero-shot setting. We conduct experiments on LLaVA- Med-1.5 using the same few-shot strategy as in Med-Flamingo. The results show that compared to the zero-shot setting. the model's performance significantly decreases, even with RAG applied. Our analysis of this phenomenon reveals that, unlike Med-Flamingo, I.LVA-Med does not use interleaved multimodal data for pretraining. As result, it lacks the capability for few-shot learn- ing. This point has been mentioned in some discussion forums and GitHub issues. addition, LLaVA-1.5's unsatisfactory performance on multi-image understanding benchmarks also supports this observation (Wang et al. 2024 Meng et al. 2024).
Table 19: Performance comparison under zero-shot and few-shot settings.
IU-Xray
Model
Acc
F1
AUC
LLaVA-Med (zero-shot)
75.47
64.04
67.46
+.MMed-RAG
89.54
80.72
87.13
LLAVA-Med (few-shot)
66.77
51.56
66.60
+ +MMed-RAG
84.10
71.92
86.40
A.6.11 PERFORMANCE COMPARISON THE RETRIEVER
Regarding the retriever's performance, shown in Table 20, we compared the performance of our retriever with several CL.IP-based models on radiology datasets for image-to-text retrieval. The results demonstrate that our retriever significantly outperforms the other models in retrieval perfor- mance.
Table 20: Performance comparison of different retrievers Recall (R@1) and Recall @5 (R@5) metrics.
Model
R@S
CLIP
3.91
7.88
PubMedCLIP
1.47
1.64
MedCLIP
6.74
12.69
BiomedCLIIP
15.7
23.8
PMC-CLIP
12.3
21.2
Ours
45.6
71.8
A.6.12 RATIONALE- RAG
For retrieved information, minimize noise by optimizing the number of retrieved contexts (e.g., Adaptive Retrieved Context Selection in Section 3.2). Following this, we introduce RAG-PT specifically address the misalignment issues that arise after incorporating RAG, thereby strength- ening Med-LVLM to balance its internal knowledge and external retrieval information. We employ rationale-guider approach (Wei al. 2024) that uses LLM to explicitly learn denoising re- trieved content through self-synthesized rationales. First, given a question, the retrieved documents, and the ground truth from the training set, we prompt a powerful Med-L.MM (i.e., L.LaMA3-Med42- 70B (Christophe et al.)) to generate rationale. This rationale explains how to derive the answer from potentially noisy imputs. Next, we use the synthesized rationale from the previous step to guide another smaller Med-LLM (i.e., LLaMA3-Med4A ing of the retrieved documents through in in-context learning and supervised learning. By employing this rationale-guided Med-LLM to filter noisy retrieval information, the reliability of our retrieved data improves. Experimental results show that after rationale-guii guided RAG, the model's performance further improved.
(Christophe et al.) to explicitly learn denois-
23

---

Published as a conference paper at ICLR 2025
Table 21: Performance comparison on IU-Xray dataset, including RAG and Rationale-Guided RAG variants.
IU-Xray
Model
Ace
F1
AUC
LLa
75.47
64.04
67.46
RAG
84.82
68.85
77.54
RAG-PT
89.54
80.72
87.13
+ Rationale-Guided RAG
85.38
69.23
77.90
RAG-PT
89.91
80.86
87.32
A.7 THE CONTRIBUTION OF DOMAIN-SPECIHI RETRIEVERS
We design a domain-specific retriever leveraging a generalist Med-LVLM to retrieve information from dedicated database based on the identified modality of the input medical image. Here, the domain identification models used are capable of reliably recognizing modalities with high accuracy (99.83% accuracy in our experiments). For radiology VQA tasks, imput radiology images are classi- fied as "radiology" by the model, enabling the retrieval of knowledge exclusively from the radiology database enhance generation. All retrieved documents are specific radiology and exclude other modalities.
A.8 EXPLANATION OF CROSS-MODALITY ALIGNMENT
To construct preference pairs for mross-mo alignment, we first select preferred response by having the model generate an answer using the correct medical image, clinical query, and retrieved knowledge, ensuring the response matches ground-truth answer. Then, we select a dispreferred response by introducing an unrelated input image. This unrelated image is selected by finding the one with the lowest similarity to the target image and adding noise to distort it further. The dispreferred response is generated when the model uses this noisy, unrelated image along with the query and retrieved knowledge to still produce the correct answer. By comparing these pairs during training, the model learns prioritize relevant and accurate inputs (e.g., the correct medical image) over noisy or irrelevant ones, improving cross-modality alignment.
A.9 ANALYSIS NOISY IMAGE IN
ALIGNMENT
In medical imaging, noise refers random variations in image signals caused by hardware limita- tions or environmental factors (Gravel et al., 2004 Sanchez et al. 2012). However, the noise we refer here pertains images unrelated the original image, generated through two-step pro- cess: 1. We use retriever to select images with the lowest similarity to the target image. 2. We introduce strong diffusion noise to these images. As result, the noisy images in our case are almost entirely random noise and are not merely examples of domain shifts, such as changes in lighting conditions. Refer to the third section of Figure for examples, and additional examples are included in the Figure for for reference.
The motivation behind our design that replacing the original image with a highly noisy image while adding retrieved information corresponding to the original image reveals a significant issue of cross-modal misalignment in the Med-LVLM- namelyy,, it ignores the image information and directly copies the retrieved contexts. To mitigate this issue, construct such preference pairs to specifically strengthen the model's cross-modal alignment capability.
A.10 EXPLANATION OF OVER RELIANCE RATE
The overall alignment issue arises from the conflict between retrieved information and the model's internal knowledge. For retrieved information, we cannot guarantee 100% accuracy, so some noise is inevitable. The Over-Reliance (OR) rate shown in Figure 3 refers to the proportion of initially correct responses that become incorrect after adding the retrieved context, calculated relative to the total number of incorrect samples, not the total number of all samples. This rate represents the
24

---

Published as a conference paper at ICLR 2025
Figure 5: Illustration Examples for noisy images in cross- mros-mmoaality alignment.
proportion of errors caused by over-reliance, rather than indicating poor performance of the retriever.
Through RAG-PT, we can effectively mitigate this issue, significantly reducing the OR rate.
25

---

Published as a conference paper at ICLR 2025
B PROOFS FOR THEORETICAL RESULTS IN SECTION 4
Here we provide proofs for the results in Section
B.1 NOTATIONS
Let Ze. V, Ft, Ty. be input medical image, ground-truth answer, question, and retrieved information, respectively. Denote (Iw: Vw.o) ~ (p(Tw: Vu.oo ,,, and (zt, 31.o) ~ (r(2), Vt, (2,, l Fy) as distri- butions of the preferred responses and dispreferred responses. Let denote (Te Tr, v... We aim
a fine-tune generative model Ta (ylo, xx) through DPO loss (Rafailov et al. 2023):
(a log (B0,
To(yu.olz) log (o(
arg min E (a o.
(14)
To (31,
where (tt) log(1 1 + exp( -)). Define the weight of ze with respect to log (o(y)z) as
2
wt( To) == By)xo((12))
log To())>))
(15)
Or,
B.2 ASSUMPTIONS
Assumption B.1 (Large parameter space) Assume that π (Zee y( lies in the optimization space {Te, 0 } such that (Iv, fl xr) x [T y/a,, xr))
Tw (a (2 (2,00))
Assumption B.1 requires that the parameter space sufficiently large ensure that xo achieve its global optimum, allowing us to represent the optimizer with a closed form.
Assumption B.2 Let h(x, y), abbreviate as h, be
h
Σ To(ll)))
(nr(y)ar) (3)(x))
á
Ime (vlaz) ()((x)))
(16)
Assume that wI( no) < c², where
/
2
To(ylæ)
0 Fee
(o(laa)
h
dy
VoTlllz))
(17)
112
Exv
Assumption B.3 Let by (Ios He, Zr, 4) abbreviate as h1, be
fo Σ (vl==) qu Vz, 2,) + g (y (y) (y)
gld Lul 2e, L: ve xx))
2s xr,, 2..)
(yler, + of (v) 22, II,
(18)
Assume that wr( To) ef and wt (r. To) > 2z where
Ohi
/
Ole
No Rody
Vioo
Ohu
dz,
h1
Dx.
1
(19)
Vioo
Oh, 2
/
Uhh
no
TTo
h1 dy
VTo
Th1 05,
0z,
05,-
h1
05,-
no
12
B.3 PROOFS
Lemma B.1 Suppose that Assumption B.l hold, optimizing equation 74 gives
1
no To(y( x (o(l))))
(20)
(v(v)a))
26

---

Published as a conference paper at ICLR 2025
Lemma B.1 indicates that the model tends to increase (o(())) if (u > q(y/2), which is more likely to occur when (Zp. 3) represents a preferred sample given 2, and 2r- Below, we provide an
application of Lemma B.1 using linear regression example. Lemma B.1 is proved with Lemma
B.2 and Lemma B.3
Lemma B.2 (Lemma C.I in Chen et al. (2024e)) For a, 6 0, the following inequality holds a a.U(I) + ((-1) 2 a alog(1 + b/a) + b blog(1 + a/b)
and equality holds if and only if log(a/b)
Lemma B.3 Denote
Pi (Iwr or Tt, Me 22, qw (zw yw, ow Rt) (1 yr Fe Ør) (2 ow Tt, Mi M2, = (T Vuo Fe, 2,) (u Mt.0 Ie. Ø+)
and abbreviated pı and p2 for notational convenience. Then,
2ED IU (f(Zws
30 L2, Kr) f(21, M1 Te, Ir,],
p1ll P1 1o1+70
P1 + P2 DKL p2
(21)
2o 2 DKL
2
Equality holds if and only if
lw (Nv F vT) vr)
f(x, v) g(x) + log
(22)
qi (I 2 Irr
where g(z) is any function that possibly dependent on zy Ie and Zy.
Proof B.
21D IU (f Nu,o Lt, IT, (21, M1 Le, xr))]
= / ((v1,tr)) (r) Pi Nno - / (K1, Mn It, x. dady
/ (((e))te)) p2 U (S(111 (1,1; W1, - (Jims (Tiryy Yuu We, x.r) drdy
2 ((x1) pr pl.oog log
1+ P1
+ +22 log
1 P2
dxdy
(23)
-2 + a((e:
p1 log
P1 + P2
+
log
P1 + +
)
drdy
2p1
2p2
22 2 KL pill
PI + P
KL
P2
+
2
2
where the first inequality follows from Lemma
B.2
For equivalence,
Gwr (E Be ol,, xr) (1), Vt,o 21, 2r) q (Iess Bo Dr) ) qu (1 Vt (2,
fx, ow It, xx)) {81, o1 Fz, xr.) = log
(24)
Thus, for any Two ow 31, o0 Lt,
qu (Iws Bwe 2T) f (Ii, Ni (t, log
qu (1t, 2e, Ør) (25)
f( Yw,or Lt, xr) log
gi(Tan Muroo De, Xr)
qi (2i, Yt De, xx))
Therefore, equation 25 holds if and only if there exists some g(Iu, De, xr) such that
[I De, y) = g( x,,) + log que U/ 2.,) qi (Tr 2e F.,)
(26)
Lemma B.3 provides a closed-form solution to equation 14 if the parameter space is sufficiently large. This lemma is crucial for the proof Lemma B.1 which follows below
27

---

Published as a conference paper at ICLR 2025
Proof B.2 According the Assumption B.1 we have
1
qu (Ter 31 2a xrr) 9 (To y re 2xr)
(( = (x x. ) (e vloe. x.))
(27)
After reparameterization.
a (Tvs (1,, (Zv yl 2x)
- a loglū(1s,, x,)] + log gu (ovs y late, Fir) (xv, (Tt, 2xy)
a log
(28)
which is the global minimum of arg min ED (U (f(zw Yw 21, - f (01, 31,os Tt, x+))]
(29)
by Lemma B.3 Since (F w/ xT) E tTo, 0 E } lies in the optimization space, we have
Yw, min EdU (f (w Tz It) - f(11, Ml Lt, 2(,,))
Fol Lt. Wrl log To (M o 24 Fe xc,)
To (V1. 11, (er
(30)
min EDU a To Twas U.,
and no (Ne, ((r) is the optimizer of equation 30 which gives
To Ta( U1 z.p)
qu (Zos y yl Trp) g xr) + log
a alog
Te 8 2x,),
qi (Zer (ee xr)
(31)
Ywr (2 (Iivs (re, Tir) (
L (20(e))er)
mo 2(r) To( (lo,,,),,,,
exp
(e y (re, Ft
Then
To (e g1 2x)
exp (2 (g(xx,,z,))
To(y|>) =
To( ylr,,
(2 dl me
xo(
Ey (l 2r)
To (Iu My q (2..))
exp (a (g(21,
(32)
Gu (22 ((= = (o(l)
Toyloo))
Ey Tolyl
9m (ylE n (M
Corollary B.1 Suppose that preferred responses (Iw, and dispreferred responses (s), y)) sat- isfy you = BIw + 61 and = 3x1 + 62 respectively. DPO for y = Ox, + c3 is based on reference model y = f,. + ca. where 's are independent and follow standard normal distribution. Then,
0 = 0, + 13- B)
(33)
Corollary B.I is direct application of Lemma B.1 indicating that the model updates coefficient A. towards the direction of 3 for preferred responses and away from 3 for dispreferred responses.
Proof B.3 Let <b-)) denote the probability density function of standard normal, by Lemma B.
Φ (xx)
Φly - -Ox) x oly - 0zz)
Φly 33x))
1 ----01100 - )
exp y/ Oory eexp (- -3)ay
exp /12²²
(34)
1(3 B)ay
= (0 exp (0,ry) exp
- 0, + (8 3)
28

---

Published as a conference paper at ICLR 2025
Lemma B.4 For linear model y 812. 02 such that (0, 1), wr(a TT) 0f
Proof B.4 Let (6-) denote the probability density function of standard normal,
wi To) = /
2 (y - 0100 -O22ze) 6 - 01 02are)dy
2 0r
0i /
(y 0100 022re)² 6(11 0100 02224) )dy
(35)
0f / (010,, 022z -3 do(y - f1x - 0g dy
dy
of /
dly 0100 02 dy = 09
Theorem B.2 Suppose that Assumption B.2 holds, then cross-modality increase the weight of I..
wI(Ev, TT) > (t)
(36)
Proof B.5 By Lemma B.
we have
To(u|z) = mo(v)z) h(x,)), y). / Tolulz) Mxx vy
(37)
Abbreviate h(x, y) and To( xz) as and To respectively, we have
8 h Ba.
Oz. To
wi(wo, Ta) wI(Kuv, To)
>
Toh dy wi((oo, To)
To
2
| 1|
2
No dy
2 201-22//eee [w((Io,;TT) To)
-wi(Fes To)
Or
h
Ox. 1
(38)
the second inequality follows from Cauchy-Schward Schwarz inequality
f
Ox,
② To
dx.
dy /
dz,
To
Too
vio dr,
dy S vwr(au. To)
|
Dx,
θ h
12
(39)
Denote as
VT Ox
/
θ h
h
h
dy
VTToo
(40)
dxv
h
Ox
the last term in equation 38 is equivalent to
(c- WI((Eon
(Ver(soo) To) + +c+2
Vio
(41)
Ox
Thus, wr(2,, π&) > wr(zo. no) if V <
Theorem B.3 Suppose that Assumption B.3 holds, the overall loss increase the weight of x, and decrease the weight of ā,.
wi(2,, To) > wi(a,, To), wt(ar, me) < wt(ār, To)
(42)
Proof B.6 The distribution of preferred responses can be considered mixture distribution: dl (Ivr ol,, 2r) + (I ol Similarly, for dispreferred responses, the distribution is rep- resented as qf (Ie, Vt,o (ra)) + (x Vt It, xr). By Lemma B.l
To(y x)) To(())) (1(x, v), / To(U(x) te (I, v)dy
(43)
29

---

Published as a conference paper at ICLR 2025
Abbreviate h1 3) as h1. Follow the same procedure in the proof of Theorem wi(Er, me) - wi To) 2
B.2
/1 dr.
11
To
dy
2 /w(((zr To)
To bx,
h1
wi( To)
h1
2
(a
vw((ar; vo))
(Vii(ar, To) ta + VTo
h1
Oxr
(44)
where we apply Cauchy-Schwarz inequality in equation
44
- VilVaolelt
dxr hi + /
8
T ()(())) dy -
a Oxr
||
1 2
dc, 111
(45)
hn
Thus, wi(ar, TT) > wi(ar; To) if V /wi(Zr, To)
< c1. Again, by Cauchy-Schwand inequality
wr(2rr, TT)
wI((ēr, To)
Ohi /
To
OTo 0ī,
2 h1 dy + 2 WI(ZEn: To)
0h1 Oz,
h1
+
To
-WI(ār: To)
(46)
2
(Vinl /Wl((Trr To) 02) (Vi((Tr,) To)
62 +2
2
no
h1
2
where
V|| VTo
a D5,
2
/
0
h1 dy +
a hi
he
hi
+
VTT
(47)
Die
hn
ai,
0ī,
Thus, wi(Ker Te) < wi(Kr, To) if v /w((Err Ta) > C2-
30


Reviews:
[{"review_id": "wBlLwlVrGy", "reviewer": "Reviewer_3Jou", "summary": "This work proposes MMed-RAG, a RAG system that corrects for misalignment issues for context introduced by RAG for multimodal large vision models (LVLMs). MMed-RAG creates preference pairs for cross-modality alignment and finetunes their LVLM (LLaVA-Med-1.5 7B) via DPO. This is described as RAG-based preference fine-tuning (RAG-PT). Other components in MMed-RAG include heuristics for domain-aware retrieval across different sources. MMed-RAG is evaluated across several medical imaging domains (radiology, ophthalmology, pathology), and compared with recent LVLMs (Med-Flamingo, RadFM, miniGPT-Med) and other RAG systems (MedDr, FactMM-RAG, RULE).", "questions": "None", "limitations": "", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "- RAG-PT (RAG-based preference fine-tuning) is a nice idea in improving the alignment performance when introducing RAG context to current LVLMs.\n- Comprehensive evaluation against various medical LVLMs and their strategies for multimodal RAG and mitigating hallucination.", "weaknesses": "- The contribution of domain-specific retrievers for RAG context in MMed-RAG is not clear to me. Practically, it is simpler to build domain-specific LVLMs that solves a targeted clinical problem well than generalist biomedical LVLMs. If we are evaluating an off-the-shelf for VQA in radiology, why would we also include pathology and ophthalmology as possible documents for RAG? We To evaluate this contribution, we would need to compare performance of MMed-RAG against domain-specific LVLMs (radiology-, pathology-, ophthalmology). All comparisons in this experiment here should ideally also adopt RAG-PT.\n- Though the idea of RAG-PT is novel in the application to medical imaging, there exist several related works in exploring DPO for RAG finetuning in LLMs. The authors should discuss these works in the related work section and how MMed-RAG adds to the literature [1,2].\n- It is unclear if RAG-PT is specific to only the medical imaging domains, or if it can be applied broadly to other domains. I think it would be valuable to assess the contribution in RAG-PT in problems outside of medical imaging, especially where image-text prompts are likely to come from heterogenous domains.\n- Is the problem of misalignment in introducing RAG context only in LLAVA-Med 1.5 7B, or does this exist in other open-source (etc., Med-Flamingo, miniGPT-Med) and commercial LVLMs (like GPT-4O)?\n\n\nRefs\n1. Dong, G., Zhu, Y., Zhang, C., Wang, Z., Dou, Z. and Wen, J.R., 2024. Understand what llm needs: Dual preference alignment for retrieval-augmented generation. arXiv preprint arXiv:2406.18676.\n2. Li, X., Mei, S., Liu, Z., Yan, Y., Wang, S., Yu, S., Zeng, Z., Chen, H., Yu, G., Liu, Z. and Sun, M., 2024. RAG-DDR: Optimizing Retrieval-Augmented Generation Using Differentiable Data Rewards. arXiv preprint arXiv:2410.13509.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "bs2OGhd4Gi", "reviewer": "Reviewer_tmxy", "summary": "The authors proposed MMed-RAG, a Multimodal Medical Retrieval-Augmented Generation framework. The authors introduce a domain-aware retrieval mechanism, using a domain identification module to adaptively select the appropriate retrieval model for medical images. MMed-RAG employs adaptive calibration to determine the number of retrieved contexts. RAG-based preference fine-tuning is used to enhance cross-modality and overall alignment, ensuring the model effectively utilizes input medical images and relevant retrieved information.", "questions": "The same as weakness.", "limitations": "", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "The authors address the limitations of the original retrieval-augmented generation (RAG) method by tackling two key challenges: the reliance on retrieved contextual knowledge without consideration of the input image, and interference from incorrect retrievals, which results in misalignment with the ground truth. To solve these challenges, the authors proposed MMed-RAG. First, it aims to enhance cross-modality alignment by ensuring the model utilizes input medical images before generating responses. Second, it aims to improve overall alignment by prompting the model to rely on retrieved contexts when uncertain, while minimizing interference from irrelevant information.\nMMRAG, as an innovative multimodal retrieval-augmented generation framework, significantly enhances the performance of medical image-text tasks. The manuscript is well-written and easy to follow.", "weaknesses": "(1) The manuscript lacks a comparison with other general methods, such as GPT-4, Gemini, and QwenVLM. An intuitive comparison to these general-purpose models would provide valuable context for understanding the advantages and limitations of MMed-RAG.\n(2) A major concern is the use of preference datasets for fine-tuning, which may cause the model to focus excessively on hard samples, potentially leading to challenges such as catastrophic forgetting and overfitting. It would be beneficial to use an external validation dataset from the same domain to assess the model's generalizability.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "Sgi8mGObYa", "reviewer": "Reviewer_WFqW", "summary": "This paper presents MMed-RAG, a retrieval-augmented generation (RAG) framework to improve factual accuracy in medical vision-language models (Med-LVLMs), addressing issues of factual inaccuracies in current models. MMed-RAG incorporates a domain-aware retrieval mechanism, adaptive context selection, and preference-based fine-tuning to enhance alignment with medical knowledge. Experimental results across five datasets show improved accuracy compared to existing methods. While primarily refining existing RAG techniques for medical applications, the approach is well-executed and clearly presented. MMed-RAG offers a focused improvement for Med-LVLMs, potentially increasing their reliability in clinical use.", "questions": "please refer to Weaknesses", "limitations": "", "rating": 8, "confidence": 3, "soundness": 3, "presentation": 4, "contribution": 3, "strengths": "1. MMed-RAG introduces a domain-specific retrieval mechanism, adapting RAG techniques to address the nuances of different medical fields. By implementing adaptive context selection and preference-based fine-tuning, the paper offers a creative and impactful advancement over traditional RAG methods, effectively enhancing cross-modal and ground-truth alignment.\n2. The paper is methodologically sound, thoroughly evaluating MMed-RAG on five diverse medical datasets. Empirical results show significant improvements in factual accuracy, with gains up to 43.8%, underscoring the effectiveness and robustness of the approach.\n3. The paper is well-structured, making complex ideas accessible. Each component of the framework—domain-aware retrieval, adaptive context selection, and fine-tuning—is clearly explained, with helpful visual aids to support understanding.\n4. Addressing factual reliability in medical diagnostics, this work has substantial real-world implications. By reducing hallucinations in Med-LVLMs, MMed-RAG contributes to safer, more reliable AI diagnostics and could be adapted to other high-stakes domains where factual accuracy is critica", "weaknesses": "1. In the analysis of RAG-PT, the authors evaluate the effectiveness of each individual RAG-PT component (1, 2, and 3). To better understand how RAG-PT mitigates misalignment and improves performance, it would be helpful to include experiments with different combinations, such as RAG-PT 1+2, RAG-PT 1+3, and RAG-PT 2+3.\n2. In Table 2, it is unclear which specific BLEU score is being reported—is it BLEU-1, BLEU-2, BLEU-3, BLEU-4, or an average of these?\n3. In the \"Comparison with Other Med-LVLMs\" section, it’s not clear which metrics are averaged for the medical VQA and report generation tasks. Additional clarification on this point would help interpret the results.\n4. In the section on Cross-Modality Alignment, more intuitive explanations are needed to clarify how constructing preference pairs (by introducing noisy images) addresses the issue of cross-modality alignment.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "47p4xSlVPV", "reviewer": "Reviewer_mHiX", "summary": "The authors propose a method, MMed-RAG, to address alignment issues in medical applications, which is essential for enhancing factual accuracy. MMed-RAG is a generalizable approach for medical RAG applications, with key contributions as follows: (1) a domain-aware retriever, (2) adaptive context selection, and (3) a direct preference-optimized algorithm based on a newly developed preference dataset curation process. Additionally, a theoretical justification is provided.\n\nMMed-RAG is evaluated on five datasets across three different medical image types—Radiology, Ophthalmology, and Pathology—on tasks such as VQA and report generation. The authors also conducted a detailed ablation study and analysis of the misalignment issues addressed by MMed-RAG. Overall, MMed-RAG achieves better performance across all tasks and demonstrates the ability to reduce misalignment.", "questions": "My main questions are\n(1) Do authors have any rational judgement on not adding more basic baseline so that we can have a better reference method in the paper? \n(2) I am willing to hear any thought or clarification about my concern of alignment analysis.", "limitations": "", "rating": 8, "confidence": 4, "soundness": 4, "presentation": 4, "contribution": 3, "strengths": "- Originality:  The perspective of decomposing RAG error into cross-modal and overall misalignment is novel, which bring about a new algorithm to collect preference data\n- Quality: The quality is solid with a theoretical proof\n- Clarity: The authors illustrate their motivation and proposed method well with good example and figures, which is easy to follow\n- Significance: The authors are working on improving the safety of VLLM in high-stake field (i..e, medicine). Therefore, I appreciate the the idea and improvement from MMed-RAG.", "weaknesses": "I think the method is overall well-motivated and well-written. However, I have some concerns which make me doubt the evidences demonstrated in the experiments and would like hear author's thought \n\n- A bit over-claimed novelty: The author claimed MMed-RAG is a general approach to address images from different sources. While it is technically true, the way author implement is merely put 3 independent piece together without certain level creativity and claimed it is general (i.e. still relying on training \"individual\" retriever model on each image source). It is more like engineering trick than methodological novelty.  \n- Lack of potential baseline and need more clarification in the evaluation :\n     - When comparing to Med-Flamingo, which setting did you use? (6-shots or zero-shot?)\n     -  I would suggest including LLaVA-Med 1.5 few shot as baseline. Based on the github record [1], LLaVA-Med 1,5 was published on 05/13/2024. while Med-Flamingo, RadFM,MedVInT and miniGPT-Med were published on 2023 based on the reference. So, I suppose LLaVA-Med 1.5 is a more powerful VLM than those. I think it is important to see how far this new more powerful model can go with simple trick (i.e., Few shot in context learning) like what Med-Flamingo author did (they have 0-shot and few-shot results)\n- Concerns about the alignment analysis:\n    - First of all, what data did you use to evaluate in Figure 3?\n    - Copy reference rate: If I understand correctly, this refer to line 8 y_{l,o1} in Algorithm 1 , which model overly rely on retrieved text. I find the construction of this idea confusing in practice. Since X_{r} is based on context retrieved by original image then you pair this context with the noise image. However, in practice, you would never get this clean X_{r} from ground truth. What you will have is noisy image + noisy context, then the model take this as input. I am bit unsure what is the point of this evaluation\n    - Over reliance rate:  If I understand correctly, this refer to line 16 y_{l,03} in Algorithm 1m which model is interfered by noisy retrieved context. My question is? Does it just mean the retriever is bad or not well-trained so that the retrieved context is nosy and  hurt the model alignment? Referring to Figure 3 OR, it shows over 0.4. With such number, I wonder how well the retriever is trained in the paper, do authors has some results or any observation?\n   -  Following the logic above, I feel like a better evaluation scheme is to (1) Use original image (2) Feed it to retriever (3) Selecting data with bad retrieved context (i.e., retriever fail). Then see how MMMed-RAG can rescue these examples.  Any comment for this?\n\n\n\n\n[1] https://github.com/microsoft/LLaVA-Med", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)