MMedPO: Aligning Medical Vision-Language Models with Clinical-Aware Multimodal Preference Optimization
Authors: Kangyu Zhu
Peng Xia
Yun Li
Hongtu Zhu
Sheng Wang
Huaxiu Yao
Venue: ICML 2025 poster (2025)
Keywords: medical vision-language models
preference optimization
Primary area: applications->health_medicine
TL;DR: A clinical-aware multimodal preference optimization method for vision-language model

Abstract: The advancement of Large Vision-Language Models (LVLMs) has propelled their application in the medical field. However, Medical LVLMs (Med-LVLMs) encounter factuality challenges due to modality misalignment, where the models prioritize textual knowledge over visual input, leading to hallucinations that contradict information in medical images. Previous attempts to enhance modality alignment in Med-LVLMs through preference optimization have inadequately addressed clinical relevance in preference data, making these samples easily distinguishable and reducing alignment effectiveness. In response, we propose MMedPO, a novel multimodal medical preference optimization approach that considers the clinical relevance of preference samples to enhance Med-LVLM alignment. MMedPO curates multimodal preference data by introducing two types of dispreference: (1) plausible hallucinations injected through target Med-LVLMs or GPT-4o to produce medically inaccurate responses, and (2) lesion region neglect achieved through local lesion-noising, disrupting visual understanding of critical areas. We then calculate clinical relevance for each sample based on scores from multiple Med-LLMs and visual tools, enabling effective alignment. Our experiments demonstrate that MMedPO significantly enhances factual accuracy in Med-LVLMs, achieving substantial improvements over existing preference optimization methods by 14.2% and 51.7% on the Med-VQA and report generation tasks, respectively. Our code are available in https://github.com/aiming-lab/MMedPO}{https://github.com/aiming-lab/MMedPO.

MMedPO: Aligning Medical Vision-Language Models with Clinical-Awand Multimodal Preference Optimization
Kangyu Zhu 12
Peng Xia"
Yun Li
Hongtu Zhu'
Sheng Wang²
Huaxiu Yao
Abstract
Please generate report based on this inage.
The advancement of Large Vision-Language Mod- els (LVLMs) has propelled their application in medicine. However, Medical LVLMs (Med- LVLMs) encounter factuality issues due to modal- ity misalignment, where the models prioritize tex- tual knowledge over visual input, causing hallu- cinations that conflict with medical images. Pre- vious attempts on preference optimization have inadequately mitigated clinical relevance in pref- erence data, making these samples easily distin- guishable and reducing alignment effectiveness. To address this challenge, we propose MMedPO, novel multimodal medical preference optimiza- tion approach that considers the clinical relevance of preference samples to enhance LLL-LVLM alignment. MMedPO curates multimodal prefer- ence data by introducing two types of disprefer- ence: (1) plausible hallucinations injected through target Med-LVLMs or GPT-4o to produce medi- cally inaccurate responses, and (2) lesion region neglect achieved through local lesion-noising, dis- rupting visual understanding of critical areas. We then calculate clinical relevance for each sample based on scores from Med-L.LMs and visual tools, and integrate these scores into the preference op- timization process as weights, enabling effective alignment. Our experiments demonstrate that MMedPO significantly enhances factual accuracy, achieving improvements over existing baseline methods by averaging 14.2% and 31.7% across the Med-VQ2 -QA and report generation tasks. Our code are available in https://gihhub..ommiiiinng lab/MM
sildly entarged. Putronary vascularity increased. there
mild elevation the right hemidiaphrage. Air space disease and/or stelectasis is noted the right lung base.
This image is chest X-ray (CXB), captured Trontal projection. The X-ray shows the presence of gallstone. The pallstone i5 located in the right labe of the lung.
Figure I. An illustration of preference data pair. The dispreferred response contains nonfactual and clinically meaningless content.
1. Introduction
Artificial intelligence is increasingly being applied in the medical field (Tăuçan al., 2021; Wang et al., 2019; Ye et al., 2021; Tu et al., 2024; Xia et al., 2024c; Wang et al., 2025; Hu et al., 2024; 2023; Li et al., 2024), including areas such as disease diagnosis and treatment planning. With the recent surge in popularity of Large Vision-Language Mod- cls (LVLMs) (Liu et al., 2024b;a; Zhu et al., 2023), Medical LVLMs (Med-LVLMs)) have begun to develop rapidly, draw- ing significant attention (Li et al., 2023a; Moor et al., 2023; Zhang al., 2023; Wu et al., 2023c; Xia et al., 2024f;e). However, these models still face the challenge of factuality, which is largely due to modality misalignment issues (Cui et al., 2023; Zhou et al., 2024a; Sun et al., 2024). Models with poor modality alignment tends to prioritize the textual knowledge learned during training over the actual visual input. As result, Med-LVLLMs often produce hallucina- tions, generating text that appears coherent but contradicts the information in the corresponding medical image (Xia et al., 2024a; Royer et al., 2024).
To tackle this issue, several studies have employed pref- erence optimization on Med-LVLMs, aiming to improve alignment between medical image and text modalities with factuality improvement (Hein et al., 2024; Sun et al., 2024; Banerjee et al., 2024). However, these methods simply leverage the preference data generation process used for aligning general LVLMs on natural images, overlooking the clinical relevance of the generated preference samples. Consequently, these preference samples become relatively easily distinguishable, reducing their effectiveness align-
'Equal Contribution. Work was done during Kangyu Zhu's internship at UNC. 'UNC Chapel-Hill "Brown University University of Washington. Correspondence Kangyu Zhu <kangyu@une.cdue Peng Xia <pxia@cs..ue.edus: Huaxiu Yao uun....dus...........
Proceedings of the 42nd International Conference Machine Learning, Vancouver, Canada. PMLR 267, 2025. Copyright 2025 by the author(s).

---

MMcdPO: Aligning Medical Vision-Language Models with Clinical-Aware Multimodal Preference Optimization
ing Med.LVLMMs. Clinical relevance can be considered from
2. Preliminaries
two perspectives. First, in these preference samples, it is essential that both preferred and dispreferred responses are clinically meaningful; if dispreferred responses lack clin- ical relevance, Med-IVL.MMs can easily distinguish them, diminishing the sample's s effectiveness. For instance, a dis- preferred response such as "a gallstone in the right lobe of the lung.." reflects a clear factual error with limited clin- ical relevance (Tu et al., 2024). Second, when improving alignment between the generated medical response and the input medical image, focused attention on local lesion areas is essential for accurate medical image understanding. Cor- recting dispreferred responses that arise from overlooking these lesion regions crucial for achieving more precise medical alignment.
2.1. Medical Large Vision Language Models
Medical Large Vision-Language Models (Med-LVLMs) are advanced architectures primarily comprising a Large Lan- guage Model (LLM) integrated with a specialized visual module. The visual module analyzes medical images to extract relevant information, transforming it into represen- tation compatible with the LLM's processing capabilities. Given a medical image Vi and a clinical query Tr, the com- bined input is represented as x = (vz, The model then autoregressively predicts the probability distribution the next token in the sequence, leveraging the multimodal input. The text output generated by the model is denoted y.
To address this challenge, we introduce MMedPO, novel Multimodal Medical Preference Optimization approach designed to quantify preference sample importance based on clinical relevance, enabling more effective preference optimization in Med-LVLMs. In MMedPO, we first cu- rate multimodal medical preference data using two strate- gies: (1) introducing dispreference by leveraging target Med- LVLMs (Li al., 2023a) or GPT-4o (OpenAI, 2023) to in- ject plausible hallucinations into responses, ensuring dispre- ferred outputs contain evident medical inaccuracies, such as incorrect imaging interpretations, misleading descriptions, or inaccurate diagnoses; and (2) provoking dispreference by neglecting lesion regions through a visual tool-guided local lesion- noising process, which disrupts the model's under- standing of these areas, leading responses that overlook critical regions, thus being marked dispreferred. We then quantify each preference sample's 's clinical significance by formulating sample importance scores, which integrate (1)
2.2. Preference Optimization
Preference optimization has proven highly effective in fine- tuning L.LMs (Rafailov et al., 2023; Bai et al., 2022), leading to significant alignment between model behavior and target objectives. In preference optimization, given an input the language model policy тe generates a conditional distribu- tion To x)), where y represents the output text response. One of the notable methods, Direct Preference Optimization (DPO) (Rafailov al.. 2023), leverages preference data to facilitate alignment within LLMs. The preference dataset is defined as D = ((=t)) (
.ut ) where y/u? denotes the preferred response and y/ the dispreferred response for given input x. The probability of preferring the over in is modeled as p(B > yt) = or((z, Vuu) r(I, vt), with (-)) representing the sigmoid function. In DPO, the optimization process is expressed as a following loss computed over the preference data:
clinical significance scores of dispreferred responses, evalu- ated by a multiple Med-LLMs collaboration process, and (2) confidence scores from visual tools assess lesion region detection accuracy. These sample importance scores are then feed into a preference optimization process, enabling more effective alignment based on the clinical relevance of each preference sample.
Coro Teet) Er WD) [log o (a co(umln) - a log Iolw x)
(1)
Tar (x)
where ma represents the reference policy, which is the LLM fine-tuned through supervised fine-tuning.
3. Multimodal Medical Preference Optimization (MMedPO)
The primary contribution of this paper is the introduction of MMedPO, aiming to quantify the clinical significance of curated preference samples to achieve more effective alignment and enhance factual accuracy in Med-LVLMs, Empirical results on two Medical Visual Question Answer- ing (QAd (Lau et al., 2018: Liu et al., 2021) and two report generation datasets (Johnson et al., 2020; Demner- Fushman et al., 2016) demonstrate that MMedPO substan- tially improves the factual accuracy of Med-LVLLLE LMMMs, achiev- ing significant gains over the best previous preference opti- mization methods, with improvements of 14.2% and 51.7% on the Med- VQA and report generation tasks, respectively.
In this section, we propose MMedPO, a clinical-aware mul- timodal preference optimization method to address modality misalignment challenges in Med-LVLMs, which consists of three steps and the entire framework is illustrated in Figure 2. Firstly, we use the target Med-LVLM or GPT along with medical visual tools to jointly construct medical multimodal preference data. Second, we evaluate the clinical relevance of each preference sample using collaborative process with multiple Med-LLMs and confidence scores from medical vi- sual tools for lesion region detection. Lastly, the normalized clinical relevance scores are integrated into the preference
2

---

MMcdPO: Aligning Medical Vision-Language Models with Clinical-Aware Multimodal Preference Optimization
1. Preference Data Curation
Il. Quantifying Clinical Relevance Score
Med-LLLM
6/10 Score
Image
Question
Med-LVLM
Dispreferrence
Multi-Agent Collaboration
40
Image
Tools
Confidence
8/10 Score
III. Clinical-Aware Preference Fine-tuning,
Image
Tools
Weight
Constructed Preference Pairs
Preference Fine-Tuning
Stronger Med-LVLM
Noise
Figure The overview of MMedPO outlines comprehensive framework consisting multimodal preference data curation, quantified preference scoring module, and clinical- -ware preference optimization. For data curation, the hallucinated text response and localized noisy images are joint constructed as preference data. Then the clinical relevance score obtained through multi-agent collaboration system and visual tools. Finally, these scores, serve weights for the clinical-aware preference optimization.
optimization process to achieve clinical-aware preference optimization. We detail these steps as follows:
alignment between generated medical responses and input medical images, concentrated attention on localized lesion areas is vital for accurate interpretation. Thus, we construct dispreferred response that stem from neglecting these lesion regions. Specifically, we leverage a medical visual tool (e.g., MedKLIP (Wu et al., 2023b)) T(-), to predict disease- related local regions h TTxx.) for each medical image z We then introduce noise into these detected localized lesion regions within the original image. The noise step is defined as k; and the noised image at step k can be expressed as follows:
3.1. Preference Data Curation
In the first step, our goal is to construct high-quality, medical-specifie multimodal preference dataset using two strategies: (1) introducing dispreference by using target Med-LVLMs or GPT-4o (OpenAI, 2023) to inject halluci- nations into medical responses, ensuring that dispreferred responses include significant medical inaccuracies; (2) pro- voking dispreference by neglecting lesion regions through medical visual tool-augmented local lesion-noising process, resulting in dispreferred responses that overlook critical regions. We detail both strategies as follows:
x -VE (2. EE
(eth))
(2)
where & = II; & and & (0, 1) are hyperparameters. In this approach, the original image Ty paired with the ground truth y is considered preferred, while the image with localized noise I& paired with the same ground truth is regarded as dispreferred. The preference data constructed using this strategy is denoted as D...
Generating Hallucinated Medical Responses. In the first strategy, we aim generate hallucinated medical response, designated as the dispreferred response, while the ground truth serves the preferred response. To achieve this, we first perform multiple rounds of sampling on the target Med- LVLMs M(-) to collect set of potential hallucinated re- sponses. We then use GPT-4o to evaluate all candidate responses and select the response with the highest level of hallucination, displaying clear conflicts with the ground truth. none of the candidates exhibit significant hallu- hallucinated
Finally, we merge the two preference sets generated by the above two strategies and denote the preference dataset as
DA, D. UD (,)()=²*()) yl " where z(1) and x:" (+) denote the normal and noisy input, (s) s/" represent preferred and dispreferred responses, respectively.
GPT-4o to generate response based ground truth to ensure that disprefer- ence contain factual inaccuracies, such as incorrect imaging interpretations, misleading condition descriptions, or erro- neous diagnoses. The preference pairs constructed using this strategy are denoted as D,,
cinations,
3.2. Quantified Clinical Relevance Score
After obtaining multimodal medical preference data, we will quantify the clinical relevance of each preference sam- ple to drive effective optimization. Our hypothesis that responses with higher clinical relevance are more valuable for preference optimization, while low-quality responses,
Adding Noise Localized Lesion Region. To improve the
3

---

MMcdPO: Aligning Medical Vision-Language Models with Clinical-Aware Multimodal Preference Optimization
in turn, reduce the effectiveness of optimization. We will explain in detail how clinical relevance is calculated below.
ease through noise can further enhance the model's focus these critical areas. However, if noisy regions are inac- curately defined, the reliability of these samples decreases, potentially impacting the model performance. Therefore, quantifying the accuracy of critical lesion detection to repre- sent sample importance during optimization is importance. To achieve this, we use the confidence scores s from visual tools that generate heatmaps of local regions as an indicator of clinical relevance. We assign different clinical relevance scores to preference pairs based on the confidence scores provided by visual tools for lesion detection.
3.2.1. CLINICAL RELEVANCE SCORES FOR DISPREFERRED MEDICAL RESPONSES
For samples generated by the target Med-LVLM and GPT- 4o (i.e., samples in Dt), we evaluate the clinical relevance of the dispreferred response based solely on the model's internal medical knowledge, without the need for visual in- put (Tian et al., 2024; Thirunavukarasu et al., 2023). Includ- ing medical images for this evaluation is unnecessary and may even hinder the process. Therefore, we rely on Med- LLMs with high levels of medical expertise to assess the clinical relevance of these text responses. Moreover, relying on a single Med-L.MM for evaluating clinical relevance may introduce bias and result in unreliable assessments (Chan et al.). To address this, we implement multi-agent col- laboration system comprising multiple Med-LLMs, each with varying levels of medical expertise. These Med-LLMs collaborate through a structured debating process to reach a consensus on clinical relevance scores, thereby improving the reliability of clinical relevance evaluations.
3.3. Clinical-Aware Preference Fine-tuning
Following the previous steps, we construct multimodal med- ical preference data and assign a quantified clinical rele- vance score to each preference sample. During preference optimization, we treat this score as the sample weight rep- resenting the contribution of each preference data pair to the overall objective. To prevent underfitting caused by an excessively small overall loss, we apply normalization strategy, mapping the scores a fixed range while maintain- ing their mean and variance. Specifically, for each clinical relevance score s, the normalized score s' is calculated as: s [سسم], then we clip s' to values of [a, 3]. Here a and 3 denote the predefined upper and lower bounds for the normalized score, and μ and a represent the mean and vari- ance of the original scores, respectively. After obtaining the normalized clinical relevance score, we fine-tune the Med-LVVLM using a weighted DPO. Following Eqn. 3, the adjusted loss with clinical relevance as sample weights is calculated as follows:
Specifically, for each Med-LLM G. where 0 < <isg g and g represents the total number of Med-LLMs, the objec- tive of the multi-agent collaborative system is to establish consensus on the clinical relevance score across all agents (i.e., Med-LLMs). This process comprises r rounds. In each round, each Med-LLM evaluates the clinical relevance score passed from the previous Med-I.LM. The process begins with the first Med-LLM, G1, which evaluates a dis- preferred response yI, generating clinical relevance score $1 G1 (31) and recording it in the score history S. Sub- sequently, each following Med-LLM G retrieves the prior scores s;-1 and determines whether to agree. If a Med- LLM concurs, it adopts s;-1 as its clinical relevance score 8;; otherwise, it generates a new score as 8g- This process continues until all Med-LLMs reach consensus and produce final score. To prevent excessive evaluations, threshold limits the number of evaluation rounds. If this threshold is reached before consensus, the final score is defined as the average of the scores in the history: s Σاقل & ensuring efficient consensus that reflects clinical relevance, where |S| represents the total number of scores.
Lmmeddo -B Mu. D'
[e(Nulz) a log =o(ur|z)) b'ooga lal 7o(Mula)
(3)
F.(01(z))
4. Experiment
In this section, we evaluate the effectiveness of MMedPO to answer the following questions: (1) Can MMedPO en- hance the factual accuracy of Med-LVLMs compared to other alignment baselines? (2) How does each individual component of the framework contribute to overall perfor- mance? (3) Can MMedPO be compatible with different Med-LVI.M architectures?
[
(4) Does MMedPO improve Med-LVLMs responses in terms of clinical relevance?
3.2.2. CONFIDENCE SCORES FOR LOCALIZED LESION REGIONS FROM VISUAL TOOLS
For preference data in D. distinct noisy regions corre- spond to disease-related lesion areas. Introducing noise into images to generate dispreferred responses for prefer- ence comparison can improve the visual understanding of Med-LVLMs (Zhou et al., 2024a; Zhao al., 2023; Wang et al., 2024a). Emphasizing lesions associated with the dis-
4.1. Experimental Setups
Evaluation Datasets. To verify the effectiveness of MMedPO in improving factuality, we utilize four medical datasets: two medical VQA datasets, i.e, VQA-RAD (Lau et al., 2018) and SLAKE (Liu et al., 2021), and two report generation datasets, i.e., MIMIC-CXR (Johnson et al., 2020)

---

MMcdPO: Aligning Medical Vision-Language Models with Clinical-Aware Multimodal Preference Optimization
Algorithm 1: Multimodal Medical Preference Op-
Evaluation Metries. For Med-VQA task, we ac- curacy and recall for both closed-ended and open-ended questions. For the report generation task, we use BLEU Score (Papineni et al., 2002), ROUGE-L (Lin, 2004) and METEOR (Banerjee & Lavie, 2005) as the metrics.
timization (MMedPO)
Input: D {2 2!") /f
Mi=1: Dataset, -M:-));
Mcd-LVLM; T()): Visual Tool; (()): Med-LLM;
:): Localized Nosiy Process; (()): Normalization.
Output: Ty: Parameters of the Med-LVLM.
4.2. Main Results
Initialize D. with an empty set Ic, y) E do 3 I >-Preference Data Curation
2 foreach (I,
In this section, we present a comprehensive comparison of MMedPO with baseline methods.
Generate responses of the Med-LVLM < MMzx,
Comparison with Baseline Methods. As shown in Table 1, we evaluate our model's s performance against the original LLaVA-Med-1.5 and several preference optimization base- lines. MMedPO demonstrates superior performance across both Medical VQA and report generation tasks. Specifically, for VQA task, MMedPO significantly outperforms the best baseline (i.e., original DPO) by an average of 1 15.8% and 10.3% across the open-ended and closed-ended ques- tions, respectively. We also observe that the overall perfor- mance improvement on open-ended questions is greater than that on closed-ended questions, indicating that MMedPO is particularly effective in guiding accurate open-ended genera- tion. Additionally, MMedPO exhibits superior performance on the report generation task, surpassing the best baseline by 61.9% and 26.0% on IU-Xray and MIMIC-CRR, re- spectively. This demonstrates that, by constructing a multi- modal preference dataset and assigning quantified clinical relevance scores to measure sample importance, MMedPO ensures that clinical relevance is fully considered during the preference optimization process, resulting in more accurate and clinically meaningful responses.
Select the dispreferred response y G)
b Quantify the Clinical Relevance
Quatify the clinical relevance using Mcd-LLMs
St < ((x)
Put {ar. y. H.M., ytt,ss St3 into D.:
Obtain the heatmap of lesion region 4- ((x)) 10 Save the confidence score from visual tool
Su < p(h(a.)
Add noise to the localized region xt (2)
12
Put ( ] into Do;
13 & Clinical Preference Optimization
14 foreach (x,xx, ; Vee y1:
D. do Normalize the score s' t ((s)
15
16
Update ny through Eq. (3)
and IU-Xray (Demner-Fushmant et al., 2016).
Implementation Details. We utilize ILLVA--Me-1.5 7B (Li et al., 2023a) as the base model. During the prefer- ence optimization stage, we apply LoRA fine-tuning (Hu et al., 2021), with a batch size of 4, a learning rate of lc-7, and train for 3 epochs. For curating preference data, we use GPT-4o evaluate and generate dispreferred responses. In
Comparison with Baseline Methods Enhanced by SFT. To demonstrate the compatibility of our approach with other training methods, we conduct experiments by applying MMedPO and other baseline methods SFT. As shown in Table 1, MMedPO consistently outperforms the SFT base- line across all four datasets, with an average improvement of 14.2%. When compared to other baselines applied to SFT, MMedPO achieves significantly better performance, with an average improvement of 10.5%. These results fur- ther corroborate the effectiveness and compatibility of our approach, demonstrating its ability to integrate seamlessly with other training techniques to enhance model alignment.
the multi-agent collaboration system, multiple Med-LLMs, including JLaMA3-Med42- (Christophe et al., 2024), MedMM33-ed42---- BioMistral-7B (Labrak et al., 2024), are used to evaluate the relevance scores for the preference data. See Appendix B for more details.
Baselines. We compare MMedPO with Direct Preference Optimization (DPO) (Rafailov al., 2023) and its variants, including the self-rewarding method (Yuan et al., 2024) and STLLaVA-Med (Sun et al., 2024). In the self-rewarding method, the model generates its own responses to form preference pairs, while STLLaVA-Med further refines the preference selection process using GPT-4o and apply it in Med-LVLMs. We further compare three VLM prefer- ence fine-tuning methods originally designed for natural images: POVID (Zhou et al., 2024a), FiSAO (Cui et al., 2024), and SIMA (Wang et al., 2024b). Additionally, we evaluate MMedPO and all baselines on models that have un- dergone supervised fine-tuning (SFT) with the correspond- ing datasets and compare their performance. Please see more details in Appendix C.
4.3. Quantitative Analysis
In this section, we first conduct ablation study analyze the effectiveness of each strategy and component in MMedPO for enhancing factual accuracy. Then, we evaluate the model's compatibility with different backbones. We fur- ther explore how our approach improves Med-LVLMS' re- sponses in terms of clinical significance and visual under- standing.
5

---

MMcdPO: Aligning Medical Vision-Language Models with Clinical-Aware Multimodal Preference Optimization
Table Performance comparison on medical VQA and report generation tasks covering SIAKKE VQA-RAD, and IU-Xray datasets. For
open-ended questions, we report recall (Open); for closed-ended questions, accuracy (Closed). The BLEU score denotes the average of BLEU-1/2/3/4, +SFT indicates the model
fine-tur ane-tuned with SFT before applying the corresponding baselines. best results
and second best results highlighted in red and blue respectively.
SLAKE
VQA-RAD
IU-Xray ROUGE-L
MIMIC-CXI -XR
Models
Open
Closed
Open
Closed
BLEU
METEOR
BLEU
ROUGE-L
METEOR
Aldee- v1.5
44.26
61.30
29.24
63.97
14.56
10.31
10.95
10.25
9.38
7.71
+ Self-Rewarding
42.63
61.30
33.29
64.17
14.20
10.38
10.52
10.78
9.27
7.73
DPO
49.30
62.02
29.76
64.70
16.08
12.95
17.13
11.19
9.45
7.80
+ POVID
52.43
70.35
31.77
65.07
20.80
24.33
30.05
11.21
9.66
7.84
SIMA
51.77
69.10
31.23
64.80
17.11
22.87
29.10
11.16
9.58
7,49
FISAO
52.69
70.46
32.70
64.11
21.06
25.72
30.82
11.32
9.68
7.62
VALLaVA-Med
48.65
61.75
30.17
64.38
16.11
10.58
10.51
11.11
9.29
7.72
MMedPO(Ours)
53.99
73.08
36.36
66.54
23.49
29.52
34.16
12.85
11.13
10.03
SFT
50,45
65.62
31.38
64.26
22.75
28.86
33.66
12.39
10.21
8.75
Self-Rewarding
50.62
65.89
32.69
65.89
22.89
28.97
33.93
12.15
10.05
8.77
DPO
53,50
69.47
32.88
64.33
23.07
29.97
34.89
12.37
10.38
9.10
POVID
52.18
70.67
32.95
64.97
23.95
29.75
34.63
11.85
10.45
9,05
SIMA
51.75
69.28
32.50
64.08
23,90
29.41
34.45
12.44
10.25
9.02
FISAO
52.80
70.82
32,94
65.77
23.57
29.88
35.01
12.97
10.69
9.39
STLLaVA-Mes
52.72
66.69
33.72
64.70
22.79
28.98
34.05
12.21
10.12
MMedPO(OUrs)
55.23
75.24
34.03
67.64
24.00
30.13
35.17
13.28
13.22
10.20
4.3.1. ABLATION STUDY
Table 2. Comparison performance across different datasets with and without clinical relevance score (CRS) for diffcrent preference curation strategies. Here, stage and stage denote generating hallucinated medical responses and adding noise to localized lesion regions, respectively. We report the average score on each dataset.
Different Preference Curation Strategies. To as-
sess the impact of different preference curation strategies in MMedPO, namely generating hallucinated medical re- sponses and adding noise to localized lesion regions, we evaluated their performance on these two components. The results, presented in Figure 3, reveal that adding noise to localized lesion regions has more pronounced effect on open-ended generation tasks (e.g., report generation) com- pared to generating hallucinated medical responses. For medical VQA tasks, the performance improvements from both preference curation processes are comparable. By inte- grating both strategies, MMedPO achieves the best overall performance across four datasets, effectively combining their strengths to maximize performance gains.
SLAKE
VQA-RAD
IU-Xray
MIMIC-CXR
Stage CRS
55.65
47.23
10.95
Stage
$7.62
48.67
15.66
6.58
Stage w/o CRS
60.59
45.94
19.30
7.17
Stage CRS
60.88
46.97
25.00
7.24
The results indicate that incorporating clinical relevance scores weights in preference optimization improves the ef- fectiveness of fine-tuning, Specifically, as shown in Table 2, for VQA task, models utilizing clinical relevance scores as weights consistently outperform those without them, with an average improvement of 2.3%. Also, significant per- formance gains are observed the report generation task, where clinical relevance scores contributed positively across different preference curation strategies, achieving clear average margin of 18.5%. The clinical relevance scores assigned to each preference pair provide positive benefits to preference optimization, helping the Med-LVLMs generate responses that are more clinically meaningful and accurate.
Report Generation
Rage
Elage
Performenca
Performanca
Closed
ROUGE-
METEOR
Figure 3. Comparison of the effectiveness of different preference curation strategics. "stage 1": generating hallucinated medical responses; "stage 2": adding noise to localized lesion regions; "stage 1+2": merged preference data. We report the average score each dataset.
4.3.2. MULTIPLE vS. SINGLE MED-LLM
To explore the impact of the multi-agent collaboration mech- anism in generating clinical relevance scores, we conduct analytical experiments, comparing the performance using clinical relevance scores from single Med-LLM and mul- tiple Med-LLMs. As shown in Table 3, we find that the consensus scores reached by multiple Med-LLMs positively
Clinical Relevance Score. To investigate the role of clinical relevance score as weight in the preference optimization pro- cess, we compare the results of applying this weight versus not applying it under different preference curation strategies.
6

---

MMcdPO: Aligning Medical Vision-Languaat Models with Clinical-Aware Multimodal Preference Optimization
proach when integrated with other powerful Med-LVLMs. MMedPO can be transferred to wider range of base mod- els, demonstrating strong generalizability for applications in clinical scenarios.
Table Comparison of model performance using clinical rele- vance scores from single Med-LLM and multiple Med-LLM, for MMedPO. We report the average score on each dataset.
Models
SLAKE
VQA-RAD
IU-Xray
MIMIC-CXR
Single-LILM
56.09
48.67
15.67
6.58
Mult--LLMs
57.53
51.14
15.86
6.86
wio MMNePDO
wro MMMddFO MikedPO
NWMddPO
Performanc
Table 4. Performance comparison between introducing local noise and global noise on the stage of constructing preference data by adding noise to medical images.
Sake
VOA-RAD
1U-Waa
MIMIC-EXR
Noise Location
SLAKE
VQA-RAD
IU-Xray
MIMIC-CXR
Global
58.88
46.91
24.88
6.80
Figure 4. Analysis compatibility using LLLVA-Med++ as the backbone model. Averaged metrics across datasets are presented.
Local
59.88
46.98
25.00
7.24
contribute to performance improvement by an average of 3.6% over four datasets. This aligns with our expectations, as relying on single Med-LLM will introduce biases. The observed improvement is driven by reduced bias through the collaborative efforts of multiple Med-LLMs,, resulting in more accurate and clinically meaningful relevance evalu- ations. In addition, the performance gains on the Med-V -QA task using multiple Med-L..Ms are notably larger compared to the report generation task. This may be attributed to greater disagreement among Med-LLMs on rejected VQA answers, allowing them to benefit more from achieving con- sensus.
4.4. Qualitative Analysis and Case Study
In this section, we further conduct qualitative experiments and case analyses.
4.4.1. QUALITATIVE ANALYSIS
How does MMedPO in Improving Visual Understand- ing? To better understand the model's visual comprehension capability, we visualize its attention map on image tokens. As shown in Figure 5. compared to the attention map of the original model, the utilization of MMedPO significantly en- hances the model's focus on visual information, particularly on critical lesion areas. This allows the model to extract sufficient information from visual inputs and improve con- sistency between text and images. Thus the model can reduce hallucinations and provide more accurate answers.
4.3.3. IMPACT OF LOCALIZED LESION NOISE
To evaluate the impact of localized lesion noise during the preference optimization process, we compare the perfor- mance of preference data composed of images with local- ized noise versus those with global noise. Global noise refers to adding noise uniformly across the entire image. As shown in Table 4, introducing localized noise consistently outperforms global noise across the four datasets. This indi- cates that lesion regions detected by visual tools are more prominent than the entire image. Introducing localized noise based on these regions helps the model better understand critical lesions, leading more factually accurate responses.
the TAN0
Chest! Twayo
No
TALL The CRaST
Mod-1.
Figure 5. Visualization of attention map of image tokens. The red box region is labeled with the attentions that are enhanced.
4.3.4. COMPATIBILITY ANALYSIS
To evaluate the compatibility of our approach with different base models, particularly more powerful backbone archi- tectures, we replace the backbone of LLaVA-Med-I.. 1.5 and conduct a series of experiments based on this configuration. Specifically, we apply our method to LLaVA-Med++ (Xie al., 2024), which uses LLaMA-3 (Dubey et al., 2024) as language backbone and enhances its performance using large-scale medical multimodal dataset MedTrinity-255M, As illustrated in Table 4, similar to the results obtain with LLaVA-Med-I 1.5, applying MMedPO leads to performance improvements across all four datasets. These findings high- light the strong compatibility and effectiveness of our ap-
Analysis Clinical Significance of Model's Response. Through the analysis of previous results, Med-LVLMs en- hanced by MMedPO demonstrate a significant improvement in factuality accuracy. Additionally, from the clinical per- spective, we aim evaluate the clinical significance of the responses to verify the effectiveness of MMedPO in en- hancing the clinical relevance of the model's outputs. As demonstrated in Figure 6, Med-LVLMS with MMedPO out- performs both the original model and the one applied with DPO. The response with MMedPO accurately capture the condition of the cardiac silhouette and rib fracture in the image, aligning with the ground truth. This also improves
7

---

MMcdPO: Aligning Medical Vision-Language Models with Clinical-Aware Multimodal Preference Optimization
Please generate nedical report based the inage.
Ques tiontion
cardiovediastinal sithouette nornal consolidation, pneumothorax large pleural effusion. Negative for acute displaced rib fracture.
contour. No focal
Clinicat
GT
Relerance
The
chest X-ray (CK) taken the arteropesterior (AP) view. rays, where the xoray beon passes frox the frunt (anterior) of the patient's is chest the back (posterior). This view allous for clear visumlization of the heart, lungs, and other structures within the chest cevity.
view
used chest X-
0.70
L
-Hed
The image is a chest X-ray, which is a disgrostic leaging technique used to visualize the structures within the chest, including the lunas, heart, and bones of the chest and spine, The X-ray is taken free an anterior- pastr-rsttrrrr (AP) view, which asans X-ray bese passes through the patiient's is body the frunt (antersur) heart, lungs, other structures within the
back (posterior) This viev
0.75
conmonly ly used in chest X-rays to.prsvide
smage
DPO
chest x-ray shows a normal cardiar silhouetse, which neans rediastinus which is the central coepartrent the thoracit position, and the costoshrenic angles, which are the angles formed the diaphrage and the chest wall, are share, suppesting so pteural effusion or other Fluid accurutation. The ribs and spine are also romall.
size shape or other ditomelition. The normal
heart appear
withan
slear,
signs also normal. dimphrags
0.90
Figure 6. Examples demonstrating the clinical relevance of responses generated by MMedPO. Our approach not only enhances the factual accuracy but also significantly improves the clinical relevance, including various meaningful medical-level explanations.
clinical significance judged by Med-LLMs, whereas the original model and other baselines produced duplicate and clinically irrelevant content. The evaluation of response us- ing clinical relevance from Med-LLM, quantitatively shows that MMedPO consistently achieves significantly higher clinical relevance scores.
ing results in the medical field (Li et al., 2023a; Moor et al., 2023; Thawkar et al., 2023; Wu et al., 2023c). However, the current Med-LVLMs still exhibit significant factual er- rors (Wu et al., 2023a; Li et al., 2023b; Xia et al., 2024a; Chen et al., 2024; Jiang et al., 2024; Su et al., 2024a). For example, they often lack sufficient judgment ability for com- plex content, and frequently generate responses with hallu- cinations that contradicts the visual information provided. This issue is particularly pronounced in medical domain, as it can potentially lead to misdiagnoses or missed diagnoses. Recently, there are several benchmarks (Xia et al., 2024a; Royer et al., 2024) that highlight the factuality issues of Med-LVLMs multiple tasks such as the visual question answering and report generation.
vhst pattolleyy 8355 the 15age represeect
Kedrey syst.
Stage rrprusenta scate
ceccrtlltti which sakee
Preference Optimization in Med-LVLMs. Aligning with human preferences for large models is an effective way to address hallucination issues (Lee et al., 2024; Zhou et al., 2024a;b; Deng et al., 2024). Preference fine-tuning in LVLMs generally involves two approaches: one aligns mod- els based on human feedback (Bai et al., 2022; Rafailov et al., 2023), while the other uses feedback generated by AI (Lee et al., 2024; Zhou et al., 2024a;b; Wang et al., 2024a; Zhou et al., 2025; Tong et al., 2025; Su et al., 2024b). Re- cently, the preference fine- mun-ttunng technique has also been adapted for medical imaging (Banerjee et al., 2024; Sun et al., 2024; Hein et al., 2024) by generating dispreferred responses using GPT-4 the target Med-LVLM. Although these methods have shown promise, they neglect the clini- cal relevance of generated samples. In Med-LVLMs,, local visual information is crucial for accurate responses, yet cur- rent approaches rarely guide the model's focus to specific lesion areas during preference fine-tuning. To tackle these issues, we incorporate quantified clinical relevance scores as weights enhance modality alignment and introduce lo- calized noise in medical images to construct dispreference, improving its understanding of key lesions.
Figure 7. Illustration of factuality enhancement by MMcdPO.
4.4.2. CASE STUDY
We amalyze two examples from Medical VQA task to illus- trate how the model fine-tuned with MMedPO reduces factu- ality errors. As illustrated in Figure 7, MMcdPO shows im- proved performance in factual accuracy. In this case, when asked about pathology, MMedPO provides a more detailed response, focusing on the problem of cyst, which is similar to the ground truth, outperforming both LLaVA-Med and LLe with DPO. This demonstrates that MMedPO effectively reduces hallucinations in Med-LVI.Mss minimiz- ing factual errors in multimodal understanding tasks.
5. Related Work
Factuality Issues in Med-LVLMs. The development of Large Vision-Language Models (LVLMs)) is progressing rapidly (Liu et al., b024a;b; Zhu et al., 2023; Bai et al., 2023; Xia et al., 2025; 2024d; Han et al., 2025; Xia et al., 2024b), which has, in turn, driven advancements in Medical Vision-Language Models (Med-LVV.Ms), achieving promis-
8

---

MMcdPO: Aligning Medical Vision-Language Models with Clinical-Aware Multimodal Preference Optimization
6. Conclusion
suppressing hallucinated prior exams in radiology report generation. arXiv preprint (XXix:4406006990 2024.
In this work, we propose novel clinical-aware multimodal preference optimization approach named MMedPO which considers the clinical relevance of each preference sample in preference optimization process. This method enhances Med-LVLM alignment while effectively reducing factual hallucinations. Specifically, to construct multimodal prefer- ence data, we introduce plausible hallucinations and apply local noise to critical lesion regions. Furthermore, we assign clinical relevance for data samples through Med-L.LMs and visual tools, and then incorporate these scores as weights in the preference fine-tuning process. We evaluate the effective- ness of MMedPO on the Med-VQA and report generation tasks, demonstrating superior performance.
Banerjee, and Lavie, A. Meteor: An automatic metric for mt evaluation with improved correlation with human judgments. In Proceedings of the acl workshop on in- trinsic and extrinsic evaluation measures for machine translation and/or summarization, pp. 65-72, 2005.
Chan, C.-M., Chen, W., Su, Y., Yu, Xue, W., Zhang, S., Fu, J., and Liu, Z. Chateval: Towards better Ilm-based evaluators through multi-agent debate. In The Twelfth International Conference on Learning Representations.
Chen, J., Yang, D., Wu, T., Jiang, Y., Hou, X., Li, M., Wang. S., Xiao, D., Li, K., and Zhang, L. Detecting and evaluating medical hallucinations in large vision language models. arXiv preprint TrXiv:2406.10180 2024.
Acknowledgement
This work is partially supported by Cisco Faculty Re- search Award and NIH ROIAG085581, R01 AG079291 and P50HD103573. The Authors acknowledge the National Artificial Intelligence Research Resource (NAIRR) Pilot, NCSA DeltaAl and OpenAl API for contributing to this research result.
Christophe, C., Kanithi, P. K., Raha, T., Khan, S., and Pimentel, M. A. Med42-v2: A suite of clinical 1lms. arXiv preprint arXiv:2408. 06142, 2024.
Cui, C., Zhou, Y., Yang, X., Wu, S., Zhang, I.., Zou, J., and Yao, H. Holistic analysis of hallucination in gpt-4v (ision): Bias and interference challenges. arXiv preprint arXiv:2311.0322 2023.
Impact Statement
Cui, C., Zhang, A., Zhou, Y. Chen, Z., Deng, G., Yao, H., and Chua, T.SS. Fine-grained verifiers: Preference modeling as next-token prediction in vision-language alignment. arXiv preprint arXiv:2410. 14148, 2024.
The broader impact of this work lies in its potential to hance the reliability and accuracy of Al-driven medical di- agnostics by reducing hallucinations and improving visual- textual alignment in Med-LVLMs. This advancement could lead to more trustworthy AI tools in healthcare, benefiting patient outcomes. However, ethical considerations are cru- cial to ensure responsible deployment, prevent misuse, and avoid over-reliance on Al-generated medical advice. Future societal benefits may include reduced diagnostic errors and improved healthcare efficiency, but ongoing research and ethical oversight are essential to align these advancements with the best interests patients and providers.
Funner-Fushman, D., Kohli, M. D., Rosenman, M. B., Shooshan, S. E., Rodriguez, I.., Antani, S., Thoma, G. R., and McDonald, C. Preparing a collection of radiology examinations for distribution and retrieval. Journal of the American Medical Informatics Association, 23(2): 304-310, 2016.
Deng. Y., Lu, P., Yin, F., Hu, Z., Shen, S., Zou, J., Chang, K.-W., and Wang, W. Enhancing large vision language models with self self-training on image comprehension. arXiv preprint arXiv:2405.I 19761 2024.
References
Bai, J., Bai, S., Yang, S., Wang, S., Tan, S., Wang, P., Lin, J., Zhou, C., and Zhou, J. Qwen-vl: A versatile vision-language model for understanding. localization, text reading, and beyond. 2023.
Dubey, A., Jauhri, A., Pandey, A., Kadian, A., Al-Dahle, A., Letman, A., Mathur, A., Schelten, A., Yang, A., Fan, A., et al. The llama 3 herd of models. arXiv preprint 2rXXv22407.2/11 2024.
Bai, Y., Jones, A., Ndousse, K., Askell, A., Chen, A., Das- Sarma, N., Drain, D., Fort, S., Ganguli, D., and Henighan, T. Training a helpful and harmless assistant with rein- forcement learning from human feedback. arXiv preprint wrXiv:2204.05822 2022.
Han, S., Xia, P., Zhang, R., Sun, T., Li, Y., Zhu, H., and Yao, H. Mdocagent: multi-modal multi-agent framework for document understanding. arXiv preprint arXiv:2503.1 25t::2503..394.. 2025.
Hein, D., Chen, Z., Ostmeier, S., Xu, J., Varma, M., Reis, E. P., Michalson, A. E., Bluethgen, C., Shin, H. J., Lan- glotz, C., al. Preference fine-tuning for factuality in
Banerjee, O., Zhou, H.-Y., Adithan, S., Kwak, S., Wu, K., and Rajpurkar, P. Direct preference optimization for

---

MMcdPO: Aligning Medical Vision-Language Models with Clinical-Aware Multimodal Preference Optimization
chest x-ray interpretation models without human feed- back. arXiv preprint arXiv:2410.0702.. 2024.
Li, W., Xiong, X., Xia, P., Ju, L.., and Ge, Z. Tp- drseg: Improving diabetic retinopathy lesion segmen- tation with explicit text-prompts assisted sam. arXiv preprint arXiv:2406. 15764, 2024.
Hu, E. J.. Shen, Y., Wallis, P., Allen-Zhu, Z., Li, Y., Wang, S., Wang, L., and Chen, W. Lora: Low-rank adaptation of large language models. arXiv preprint arXiv:2 00660, 2021.
Li, Y., Liu, Y., Wang. Z., Liang, X., Liu, L., Wang, L., Cui, L., Tu, Z., Wang, L., and Zhou, L. A comprehensive study of gpt-4v's multimodal capabilities in medical imaging. arXiv preprint arXiv:2310.20381, 2023b.
Hu, M., Wang, I.., Yan, S., Ma, D., Ren, Q.. Xia, P., Feng, W., Duan, P., Ju, L., and Ge, Z. Nurvid: A large expert- level video database for nursing procedure activity un- derstanding. Advances in Neural Information Processing Systems, 36:18 (6:18146-18864 2023.
Lin, C.-Y. Rouge: package for automatic evaluation of summaries. Text summarization branches out, pp. 74-81, 2004.
Liu, B., Zhan, L.-M., Xu, L.,, Ma, L.., Yang, Y., and Wu, X.- M. Slake: A semantically- labeled knowledge-enhanced dataset for medical visual question answering. In 2021 IEEE 18th International Symposium on Biomedical Imag- ing (ISBI), pp. 1650-16555 1654. IEEE, 2021.
Hu, M., Xia, P., Wang, L., Yan, S., Tang, F., Xu, Z., Luo, Y., Song, K., Leitner, J., Cheng, X., et al. Ophnet: A large- scale video benchmark for ophthalmic surgical workflow understanding. arXiv preprint arXiv:2406.0747,, 2024.
Jiang, Y., Chen, J., Yang. D., Li, M., Wang. S., Wu, T., Li, K., and Zhang, L. Medthink: Inducing medical large- scale visual language models to hallucinate less by think- ing more. arXiv preprint ArXiv:2406. 1451, 2024.
Liu, H., Li, C.,, Li, Y., and Lee, Y. J. Improved baselines with visual instruction tuning. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 26306, 2024a.
Johnson, A., Bulgarelli, L., Pollard, T., Horng, S., Celi, L. A.. and Mark, R. sioNet. Available online at: https://physionet. org/contentmimiciill.. http://mmiiiiii/I..aeeeeee Maccessed August 23, 2021), 2020.
Liu, H., Li, C., Wu, Q., and Lee, Y. Visual instruction tun- ing. Advances in neural information processing systems, 36, 2024b.
Mimic-iv. Phy-
Moor, M., Huang, Q., Wu, S., Yasunaga, M., Dalmia, Y., Leskovec, J., Zakka, C., Reis, E. P., and Rajpurkar, P. Med-flamingo: a multimodal medical few-shot learner. In Machine Learning for Health (ML4H), pp. 357-367. PMLR, 2023.
Labrak, Y., Bazoge, A., Morin, E., Gourraud, P.-.,, Rou- vier, M., and Dufour, R. Biomistral: A collection of open-source pretrained large language models for medi- cal domains. arXiv preprint arXiv:2402. 10373, 2024.
OpenAl. Gpt-4 technical report, 2023. sttps://an /tttps://arxiv.. org/ abs / 22333 08774.
Langley, P. Crafting papers on machine learning. Langley, P. (ed.), Proceedings of the 17th International Conference on Machine Learning (ICML 2000), pp. 1207-1216, Stan- ford, CA, 2000. Morgan Kaufmann.
Papineni, K., Roukos, S., Ward, T., and Zhu, W.-J Bleu: a method for automatic evaluation of machine transla- tion. In Proceedings of the 40th annual meeting of the Association for Computational Linguistics, pp. 311-318, 2002.
Lau, J. J., Gayen, S., Ben Abacha, A., and Demner- Fushman, D. dataset of clinically generated visual questions and answers about radiology images. Scientific data, 5(1):1-10, 2018.
Rafailov, R., Sharma, A., Mitchell, E., Manning, C. D., Ermon, S., and Finn, C. Direct preference optimization: Your language model is secretly areward model. In Thirty- seventh Conference Neural Information Processing Systems, 2023.
Lee, H., Phatale, S., Mansoor, H., Mesnard, T., Ferret, J., Lu, K. R., Bishop, C., Hall, E., Carbune, V., Rastogi, A., et al. Rlaif vs. rlhf: Scaling reinforcement learning from human feedback with ai feedback. In Forty-first International Conference on Machine Learning, 2024.
Royer, C., Menze, B., and Sekuboyina, A. Multimedeval: A benchmark and toolkit for evaluating medical vision- language models. arXiv preprint arXiv:2402.09262. 2024.
Li, C., Wong, C., Zhang, S., Usuyama, N., Liu, H., Yang, J., Naumann, T., Poon, H., and Gao, J. Llava-med: Training a large language-and-vin one day. In Thirty-sevents Conference on Neural Informa- tion Processing Systems Datasets and Benchmarks Track, 2023a.
assistant for biomedicine in
Su, Z., Zhang, J., Qu X., Zhu, T., Li, Y., Sun, J., Li, J., Zhang, M., and Cheng, Y. Conflictbank: A benchmark for evaluating the influence of knowledge conflicts in Ilm. arXiv preprint arXiv:2408. 2006, 2024a.
10

---

MMcdPO: Aligning Medical Vision-Language Models with Clinical-Aware Multimodal Preference Optimization
Su, Z., Zhang, J., Zhu, T., Qu, X., Li, J., Zhang, M., and Cheng, Y. Timo: Towards better temporal reasoning for language models. arXiv preprint 1rXx::2406.141922 2024b.
Wang, Z., Li, M., Xia, P., Jiang, C., Shen, T., Ma, J., Bai, Y., Zhang, S., Lai, Y., Li, S., et al. Screening cognitive im- pairment in patients with atrial fibrillation: a deep learn- ing model based on retinal fundus photographs. Heart Rhythm 02, 2025.
Sun, G., Qin, C., Fu, H., Wang, L.,, and Tao, Z. Stilava- med: Self-training large language and vision assistant for medical. arXiv preprint arXiv:2406. 19973, 2024.
Wu, C., Lei, J., Zheng, Q., Zhao, W., Lin, W., Zhang, X., Zhou, X., Zhao, Z.,, Zhang, Y., Wang, Y., et al. Can gpt- 4v (ision) serve medical applications? case studies on gpt-4v for multimodal medical diagnosis. arXiv preprint arXiv:2310.0093 2023a.
Täujan, A.-M., Jonescu, B., and Santareechh,, E. Artificial intelligence in neurodegenerative diseases: A review of available tools with focus on machine learning tech- niques. Artificial Intelligence in Medicine, 117:102081, 2021.
Wu, C., Zhang. X., Zhang, Y., Wang, Y., and Xie, W. Medklip: Medical knowledge enhanced language-image pre-training for x-ray diagnosis. Proceedings of the JEEE/CVF International Conference on Computer Vision, pp. 21372-21383, 2023b.
Thawkar, O., Shaker, A., Mullappilly, S. S., Cholakkal, H., Anwer, R. M., Khan, S., Laaksonen, J., and Khan, F. S. Xraygpt: Chest radiographs summarization us- ing medical iisionllnguage models. arXiv preprint arXiv:2306.0799 077971,10.777,, 2023.
Wu, C., Zhang, X., Zhang, Y., Wang, Y., and Xie, W. To- wards generalist foundation model for radiology. arXiv preprint arXiv:2308. 02463, 2023c.
Thironavukarasu, A. J.. Ting, D. S.J., Elangovan, K., Gutier- rez, L., Tan, T. F., and Ting, S. W. Large language models in medicine, Nature medicine, 1930)199,- 2023.
Xia, P., Chen, Z., Tian, J., Gong, Y., Hou, R., Xu, Y., Wu, Z., Fan, Z., Zhou, Y., Zhu, K., et al. Cares: A comprehensive benchmark of trustworthiness in medical vision language models. arXiv preprint arXiv:2406.0600 200000 2024a.
Tian, S. Jin, Q., Yeganova, L., Lai, P.-T., Zhu, Q. Chen, X., Yang, Y., Chen, Q., Kim, W., Comeau, D. C., et al. Opportunities and challenges for chatgpt and large lan- guage models in biomedicine and health. Briefings in Bioinformatics, 25()):bbad493, 2024.
Xia, P., Han, S., Qiu, S., Zhou, Y., Wang, Z., Zheng, W., Chen, Z., Cui, C., Ding, M., Li, L.,, al. Mmie: Massive multimodal interleaved comprehension bench- mark for large vision -anguuge models. arXiv preprint arXiv: 2410. 2410.1019, 2024b.
Tong, H., Wang, Z., Chen, Z., Ji, H., Qiu, S., Han, S., Geng, K., Xue, Z.. Zhou, Y., Xia, P., et al. Mj-video: Fine- grained benchmarking and rewarding video preferences in video generation. arXiv preprint arXiv:2502.01719, 2025.
Xia, P., Hu, M., Tang, F., Li. W., Zheng, W., Ju, L., Duan, P., Yao, H., and Ge, Z. Generalizing unseen domains in diabetic retinopathy with disentangled representations, In arXiv preprint
2024c.
Tu, T., Azizi, S., Driess, D., Schaekermann, M., Amin, M., Chang. P.-C., Carroll, A., Lau, C., Tanno, R., Ktena, I,, et al. Towards generalist biomedical ai. NEJM AI, 1(3): Aloa2300138, 2024.
Xia, P., Xu, D., Hu, M., Ju, L., and Ge, Z. Lmpt: Prompt tuning with class spasi-i embedding loss for long-tailed multi-label visual recognition. In Proceedings of the 3rd Workshop on Advances in Language and Vision Research (ALVR), pp. 26-36, 2024d.
Wang, C., Zhu, X., Hong, J. C., and Zheng, D. Artificial intelligence in radiotherapy treatment planning: present and future. Technology in cancer research & treatment, 0811333338118883922.. 2019.
Xia, P., Zhu, K., Li, H., Wang, T., Shi, W., Wang. S., Zhang, L., Zou, J., and Yao, H. Mmed-rag: Versatile multimodal rag system for medical vision language models. arXiv preprint arXiv:2410.1 3085, 2024e.
Wang, F., Zhou, W. Huang, J. Y., Xu, N., Zhang, S., Poon, H., and Chen, M. mdpo: Conditional preference opti- mization for multimodal large language models. arXiv preprint arXiv:2406. 1839, 2024a.
Xia, P., Zhu, K., Li, H., Zhu, H., Li, Y., Li, G., Zhang, L., and Yao, H. Rule: Reliable multimodal rag for factuality in medical vision language models. In Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing, pp. 1081-1093, 2024f.
Wang. X., Chen, J., Wang. 7., Zhou, Y., Zhou, Y., Yao, H., Zhou, T.. Goldstein, T.. Bhatia, P., Huang, F., et al. Enhancing vasual-language modality alignment large vision language models via self-improvement. arXiv preprint arXiv:2405. 15973, 2024b.
Xia, P., Yu, X., Hu, M., Ju, L., Wang, Z., Duan, P., and Ge. Z. Hgelip: Exploring languala models with graph representations for hierarchical understanding. In

---

MMcdPO: Aligning Medical Vision-Language Models with Clinical-Aware Multimodal Preference Optimization
Proceedings of the 31 International Conferencee on Com- putational Linguisties, pp. 269-280, 2025.
Xie, Y., Zhou, C., Gao, L., Wu, J., Li, X., Zhou, H.-Y., Liu, S., Xing, L.., Zou, J., Xie, C., et al. Medtrinity-25mm A large-scale multimodal dataset with multigranular anno- tations for medicine. arXiv preprint arXiv:2408.0220 020000 2024.
Ye, Q.. Hsieh, C.-Y., Yang, Z., Kang, Y., Chen, J., Cao, D., He, S., and Hou, T. unified drug-target interaction prediction framework based on knowledge graph and recommendation system. Nature communications, 12(1): 6775, 2021.
Yuan, W., Pang, R. Y., Cho, K., Li, X., Sukhbaatar, S., Xu, and Weston, J. E. Self-rewarding language mod- els. In Forty-first International Conference on Machine Learning, 2024.
Zhang, X., Wu, C.. Zhao. Z.. Lin, W., Zhang, Y., Wang, Y., and Xie, W. Pmc-vqa: Visual instruction tuning for medical visual question answering. arXiv preprint arXiv:2305. 10415, 2023.
Zhao, Z., Wang, B., Ouyang, L., Dong, X., Wang, J., and He, Beyond ballucinations: Enhancing Ivlms through hallucination-aware direct preference optimization, arXiv preprint arXiv:231 16839, 2023.
Zhou, Cui, C., Rafailov, R., Finn, C., and Yao, H. Align- ing modalities in vision large language models via pref- erence fine-tuning. arXiv preprint 1 1411, 2024a.
Zhou, Y., Fan, Z., Cheng, D., Yang, S., Chen, Z., Cui, C., Wang, X., Li, Y., Zhang, L., and Yao, H. Calibrated self-rewarding vision language models. arXiv preprint arXiv:2405.14622 14622, 2024b.
Zhou, Y., Wang, 7.,, Wang, T., Xing, S., Xia, P., Li, B., Zheng, K., Zhang, Z., Chen, Z., Zheng, W., et Anypre- fer: An agentic framework for preference data synthesis. arXiv preprint arXiv:2504.199 192766 2025.
Zhu, D., Chen, J., Shen, X., Li, X., and Elhoseiny, M. Minigpt-4: Enhancing vision-language understanding with advanced large language models. arXiv preprint 20X1v:2304.10592 2023.
12

---

MMcdPO: Aligning Medical Vision-Language Models with Clinical-Aware Multimodal Preference Optimization
A. Data
A.1. Data Statistics
The data statistics are shown in Table 5 and Table 6, In the training datasets, the reported quantities for the two datasets in report generation represent image-report pairs, while the quantities for the two datasets in the medical VQA task represent question-answer pairs.
Table Data statistics for the training set of four datasets under two different task settings. "Train (visual)" refers to the number of visual-only preference data, while "Train (text)" indicates the number of text-only preference data.
Dataset
Train (visual)
Train (text)
Train (all)
IU-Xray
2069
2069
4138
MIMIC-CXR
800
800
1600
SLAKE
4919
4919
9838
VQA-RAD
1797
1797
3594
Table 6. Data statistics of test set. #lmages, #QA items and #Reports mean the number of images, QA pairs and reports, respectively.
Dataset
#Images
#QA items
#Reports
Xr-Xray
590
590
MIMIC-CXR
200
200
SLAKE
641
1061
VQA-RAD
315
451
A.2. Involved Datasets
We leverage four open-source medical vision-language datasets: MIMIC-CXR (Johnson al., 2020), IU-Xray (Demner- Fushman al., 2016), SLAKE (Liu et al., 2021), and VQA-RAD (Lau al., 2018). These datasets are designed for different tasks: the first two focus medical report generation, while the latter are tailored for medical visual question answering.
IU-Xray a dataset that includes chest X-ray images and corresponding diagnostic reports.
MIMIC-CXR is a widely accessible dataset containing chest X-ray images in DICOM format along with corresponding
radiology reports.
SLAKE an English-Ching Chinese bilingual dataset comprising 642 images and 14,028 question- -answer pairs designed for training and evaluating Med-VQA systems.
VQA-RAD is the first dataset manually curated by clinicians, featuring naturally occurring questions about radiology images along with corresponding reference answers.
B. Hyperparameter Settings
For the usage visual tools, we employ "disease" as the text description to guide MedKLIP (Wu et al., 2023b) in generating heatmaps. For multi-agent collaboration, the process is conducted over rounds. During score normalization, the parameters are set as: 0.75, 1.25. f 1, and o² 0.1. All hyperparameters are kept consistent across the experiments to eliminate any potential bias introduced by hyperparameter tuning. All experiments are implemented using PyTorch 2. on four NVIDIA RTX A6000 GPUs, with training requiring approximately 2 to 3 hours.
C. Involved Baselines
DPO (Rafailov et al., 2023) is a fine-tuning approach designed to align large language models (LLMs) with human preferences in stable, efficient, and computationally lightweight manner. Unlike traditional Reinforcement Learning
13

---

MMcdPO: Aligning Medical Vision-Language Models with Clinical-Aware Multimodal Preference Optimization
Table Detailed performance comparison on report generation tasks covering IU-Xray and MIMIC-CXR datasets. denotes BLFU.
IU-Xray BL-4
MIMIC-CXR
Models
BL-1
BL-2
BL-3
ROUGE-L
METEOR
BL-1
BL-2
BL-3
BL-4
ROUGE-L
METEOR
LLaVA-Med v1.5
38.42
13:40
4.74
1.67
10.31
10.95
29.41
10.19
3.58
1.26
9.38
7.71
Sela-Rewarding
38.25
13.17
3.61
1.08
10.38
10.52
29.29
10.32
3.67
1.30
9.27
7.73
DPO
41.63
15.13
5.56
2.03
12.95
17.13
29.61
10.29
3.61
1.27
9.45
7.81
POVID
50.84
20.65
8.38
24.33
30.05
29.68
10.29
7.84
SIMA
42.67
16.82
5.98
2.95
22.87
29.10
29.58
10.23
3.59
1.24
9.58
7.49
FISAO
51.10
20.92
8.64
3.59
25.72
30.82
29.76
10.37
3.74
1.39
9.68
7.62
STLLaVA-Med
42.38
15.27
5.59
1.20
10.58
10,51
29.33
10.27
3.58
1.27
9.29
7.72
MMedPO (Ours)
55.58
23.93
10.36
4.40
29.52
34.16
33.67
11.91
4.28
1.54
11.13
10.03
Table Detailed component ablation study on report generation covering IU-X IU-Xray MIMIC-CX. datasets. denotes BLEU, Here, stage I and stage 2 denotes generating hallucinated medical responses and adding noise to localized lesion regions, respectively.
IU-Xray BL.-4
MIMIC-CXR
Models
BL.11
B1-2
Bl.3
ROUGE-L
METEOR
BL.11
B1-2
BL-3
BL.4
ROUGE-L
METEOR
Stage (Single-LLM)
43.45
16.05
5.99
2.21
19.66
22.65
29.41
10.19
3.58
1.26
9.33
7.77
Stage (MuliiLLLMs)
43.95
16.44
6.21
2.31
19.57
22.92
29.85
10.38
3.65
1.28
9.62
8.18
Stage
55.15
23.59
10.13
4.23
29.02
34.26
30.96
10.89
3.87
1.38
9.85
8.81
SStage 1+2 (Single-LLM)
55.36
23.85
10.34
4.39
29.30
34.22
32.96
11.63
4.14
1.46
10.99
10.03
.Stge 142 (Multi-LLMs)
55.58
23.93
10.36
4.40
29.52
34.16
33.67
11.91
4.28
1.54
11.13
10.05
from Human Feedback (RLHF), which involves training reward model and using reinforcement learning to maximize the reward, DPO simplifies the process by reframing the problem. It parameterizes the reward model in way that allows the optimal policy to be derived directly through a classification loss, eliminating the need for complex sampling or extensive hyperparameter tuning during fine-tuning.
Self-Rewarding (Yuan et al., 2024) a novel approach where the language model itself acts as judge, generating rewards via LLM-gs-a-Judge prompting during training. Unlike traditional methods that rely on reward models trained from human preferences, which are limited by human performance and static design, this method enables the model to iteratively improve both its instruction-folloiing abilities and its reward-generating quality during iterative DPO training-
STLLaVA-Med (Sun al., 2024) refines the preference selection process using GPT-4o and applies it in medical vision- language tasks. STLLaVA-Med extends the DPO approach by incorporating self-training mechanism specifically tailored for the medical domain.
POVID (Zhou et al., 2024a) addresses the hallucination problem in vision-language models by generating feedback data using Al models. It uses ground-truth instructions as preferred responses and creates dispreferred data by injecting plausible hallucinations and distorting images, integrating these strategies into an RLHF pipeline via DPO.
FiSAO (Cui al., 2024) introduces fine-grained self-alignment optimization method that leverages the model's visual encoder to improve vision-language alignment. By utilizing token-level feedback from the vision encoder, it enhances alignment without the need for additional external data, outperforming traditional preference tuning methods.
SIMA (Wang al., 2024b) is framework that enhances visual and language modality alignment through self- improvement, eliminating the need for external models or data. It uses prompts from existing datasets to self self-generate responses and employs an in-context self-critic mechanism with vision metrics to select optimal response pairs for preference tuning.
D. A Additoonal Results
In this section, we present a detailed benchmark analysis for the report generation task. Table compares our method with other baseline approaches. Additionally, Tables and 9 provide comprehensive component ablation results for both the Medical VQA and report generation tasks.
14

---

MMcdPO: Aligning Medical Vision-Language Models with Clinical-Aware Multimodal Preference Optimization
Table Detailed component ablation study on SLAKE and VQA-RAD datasets for hoth open and elosed settings. Here, stage and stage 2 denotes generating ballucinated medical responses and adding noise to localized lesion regions, respectively.
Method
SLAKE
VQA-RAD
Open
Close
Open
Close
Stage (Singlc-LLM)
47.99
64.18
32.27
65.07
Stage (Multi-LLMs
49.39
65.87
32.42
69.85
Stage
51.25
68.51
31.09
62.87
E. Prompts
We utilize GPT-4o to generate hallucinated responses for constructing preference data, illustrated by the prompts in Figure 8. Subsequently, multi-agent system comprising Med-LLMs is employed to evaluate the clinical relevance scores of these rejected responses, with the evaluation prompts shown Figure 9.
Prompt for Generation of Hallucinated Response Using GPT-4o
You are provided with ground truth report: (gt report). Please focus on the critical lesion descriptions within the report and inject hallucinations into these key descriptions to construct dispreferred response. Ensure that the dispreferred responses contain significant medical inaccuracies. You may refer to the following examples for guidance: <example 1>, <example 2>, Your response should consist only of the hallucination- injected response.
Figure The instruction GPT-4o the rejected hallucinated answer.
Prompt Med-LLMs eva uation the collaboration process
You are given a set consisting of question, answer, and score. The question is: (prompt), and the answer is: (rejected value). The provided score(s) this answer (weighted_scee _ccrre. These scores were assigned by other medical large language models based on evaluation criteria that assess the clinical value of the answer, emphasizing its relevance and specificity. The evaluation also considers whether the without relying solely on general or common knowledge. Based on the question and answer, evaluate the yourself. If you disagree with the provided score(s), assign new score between 0.0 and 1.0 your response. If you agree with the provided score(s), simply copy the score or respond with "agree." Response:
directly addresses the question
Figure 9. The instruction Med-LLMs for evaluating and generating clinical relevance
F. More Cases
present additional examples in Figure 10, illustrating how our method effectively reduces hallucinated emrors.
15

---

MMcdPO: Aligning Medical Vision-Languee Models with Clinical-Aware Multimodal Preference Optimization
Wht disoases are included in the picture?
MedP Poce
mht are the bright white structures?
Locludee with eyclosd Leubosia (ANL) and acute
Likely to represent calcifications iume deptn celcien tissses and they can appcar as bright white arous
LLas
Hed
Liver cirrhosts, Iepatocelluter and portal, hypertension.
Lmage are Likely to be areas of celecfication.
are Likely the vestricles braifu.
Where you see acute Infarcts?
naty in
ADrte infercts are vincble in the PRT touge, siocifically parictal pccipitel
-Fed
see acate Infarcte in the right parintal occipital lobes
120
the right Frontal labe
What largest organ the picture?
Mhat color coes the Left lung STow 1n the
The Left lung in the chest K-ray appcars To
teart.
Liver.
White
lung
Largest
MedPix
taken
Mhee in/ere abnoraslity located?
Right Lurg,
axint
wyeloud Leukemia (ANE)
affecting the atcordna wall essoulature.
Tyeptol lastic teuie (ALL),
The irage is taker in the acial slane.
The Jonormolity is located in the rigit larg.
Figure 10. More cases that reduce hallucinated enrors.
16


Reviews:
[{"review_id": "iaG1Mitn4w", "reviewer": "Reviewer_Tkt9", "summary": "This paper focuses on aligning Medical Vision-Language Models (Med-LVLMs) with clinical-aware multimodal preference optimization to improve factual accuracy and reduce hallucinations. The authors identify modality misalignment as a major issue, where models prioritize textual knowledge over visual input, leading to clinically incorrect responses. To address this, they propose MMedPO, a novel framework that curates multimodal preference data with two types of dispreference: plausible hallucinations generated by Med-LVLMs or GPT-4o and lesion region neglect introduced via local lesion-noising. Clinical relevance scores, derived from medical large language models (Med-LLMs) and visual tools, are integrated into the preference optimization process to weigh preference samples effectively. Experimental results on medical VQA and report generation tasks demonstrate that MMedPO outperforms existing baselines.", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for your valuable feedback to help us improve our paper. We detail our response below and please kindly let us know if our response addresses your concerns. **All tables referenced in this rebuttal can be found in** https://anonymous.4open.science/r/ICML_rebuttal-0304/README.md \n****\n>**Q1**: The definition of hallucination in Med-LVLMs is unclear in this paper, and the reasons behind hallucination causes remain ambiguous.\n\n**A1**: We define hallucination as a response that contradicts the image content given the question. As noted in the introduction, a key issue is modality misalignment [R1-R2], where the model overly relies on text rather than grounding responses in visual content.\n\n[R1] Zhou Y, et al. Analyzing ...Models. ICLR 2024.\n\n[R2] Chen J, et al. Detecting ...models. arXiv 2024.\n\n****\n>**Q2**: Some empirical results appear inconsistent… a) The reported performance of the base model...is much worse than in its original paper [1]... b) Table 7 shows extremely low BLEU-2/3/4 scores on MIMIC-CXR ... fine-tuning?\n\n[1] LLaVA-Med: Training … in One Day. NeurIPS 2023.\n\n**A2**: a) The performance drop of the base model (LLaVA-Med v1.5) on SLAKE and VQA-RAD compared to the original paper [1] is due to differences in experimental setup. While prior works used fully fine-tuned LLaVA-Med v1.0 checkpoints, such checkpoints are not available for v1.5.  Therefore, in our experiments, we used the LLaVA-Med v1.5 pre-trained checkpoint and performed LoRA fine-tuning, which naturally leads to some performance gap compared to full fine-tuning. b) The low BLEU-2/3/4 scores in Table 7 were due to the specific BLEU settings used in our initial evaluation. We appreciate your attention to this and have since updated our BLEU configuration. The revised results (**Table R1**) are more consistent with expectations. Importantly, this correction does not affect the overall trends or conclusions.\n\n****\n\n>**Q3**: Multi-agent collaboration for clinical relevance scoring...questioning the necessity of this step.\n\n**A3**: We discuss the benefits of using multiple Med-LLMs for clinical relevance scoring in Section 4.3.2. As shown in Table 3, multi-agent discussion yields a 3.6% improvement in clinical relevance scores over single-LLM scoring, suggesting that incorporating diverse perspectives can enhance the robustness, even if it is heuristic in nature. It is worth noting Table 8 reports results for the report generation task, not clinical relevance scoring. While generation performance appears similar across single- and multi-LLM setups, Table 3 shows that the multi-agent approach yields meaningful gains in clinical relevance evaluation.\n****\n>**Q4**: Why is an additional clinical relevance weight necessary?...weighting may be redundant...better to conduct a baseline without this step.\n\n**A4**:  While the Preference Data Curation step ensures dispreferred responses include hallucinations or region-level neglect, not all errors are equally harmful—some are clearly incorrect and easily avoidable, while others are subtle and clinically significant. We introduce clinical relevance weighting to capture this distinction and better guide optimization.\nTo validate its effectiveness, we added experiments in **Table R2** and **Table R3** comparing models trained with and without relevance-based weighting. The results show consistent performance drops without weighting, confirming its benefit.\n****\n>**Q5**: The math formula of weighted DPO needs consistency checking...Algorithm 1 is not described in the main text.\n\n**A5**: Thanks for pointing this out. We will make sure to correct the inconsistencies in the mathematical notations across the text, equations, and Algorithm 1 in the final PDF version. Additionally, we will add a proper reference and description of Algorithm 1 in the main text to ensure clarity and completeness.\n****\n>**Q6**: Lack of clinically meaningful evaluation metrics...using NLG metrics (BLEU, ROUGE-L, METEOR), but these do not reflect medical accuracy. More relevant clinical efficacy (CE) metrics...should be reported, following prior work like METransformer [2]. This is especially crucial since ...without CE metrics, it is unclear whether improvements are meaningful.\n\n[1] LLaVA-Med, NeurIPS 2023. [2] METransformer, CVPR 2023.\n\n**A6**: We have included clinical efficacy metrics—precision, recall, and F1-score—in **Table R4**, comparing our method against baselines. Our approach consistently improves both NLG and clinical efficacy metrics, indicating better diagnostic relevance and clinical accuracy of the generated reports.\n****\n>**Q7**: Would other types of perturbations...improve lesion-based preference optimization?\n\n**A7**: We explored various noise types (**Table R5**) and found that while Diffusion and Gaussian noise performed similarly, random noise offered a slight improvement. Many thanks for your valuable advice.", "claims_and_evidence": "1. The definition of hallucination in Med-LVLMs is unclear in this paper, and the reasons behind hallucination causes remain ambiguous. A clearer justification is needed.\n\n2. Some empirical results appear inconsistent or unexpectedly low, raising concerns about experimental validity.", "methods_and_evaluation_criteria": "1. Multi-agent collaboration for clinical relevance scoring: While using multiple Med-LLMs for preference scoring is interesting, the scoring process is heuristic, lacks interpretability, and does not provide a reasoning mechanism. Besides, as shown in Table 8, the results from single-LLM and multi-LLMs are comparable, questioning the necessity of this step.\n\n2. Clinical relevance weighting in DPO: The Preference Data Curation step already introduces \"clinical-aware preference\" by distinguishing plausible hallucinations and lesion-region neglect. Why is an additional clinical relevance weight necessary? If the preference data is already \"clinically aware,\" weighting may be redundant. It would be better to conduct a baseline without this step (Part II in Figure 2) for comparison.", "theoretical_claims": "The paper does not introduce new theoretical claims or proofs, so this section is not applicable. However, the mathematical formulation of weighted DPO needs consistency checking, as notations in text, equations, and Algorithm 1 do not always align. BTW, Algorithm 1 is not described in the main text.", "experimental_designs_or_analyses": "1. Unexpectedly low performance in some datasets:\n\na) The reported performance of the base model (LLaVA-Med v1.5) on SLAKE and VQA-RAD is much worse than in its original paper [1] (e.g., open setting on SLAKE: 44.26 vs. 87.11). Why is there such a drastic drop?\n\nb) Table 7 shows extremely low BLEU-2, BLEU-3, and BLEU-4 scores on MIMIC-CXR, which seems unrealistically poor for LLaVA-Med v1.5. Could this be a reporting error, or does it indicate some issue in fine-tuning?\n\n2. Lack of clinically meaningful evaluation metrics:\n\nThe paper evaluates report generation using NLG metrics (BLEU, ROUGE-L, METEOR), but these do not reflect medical accuracy. More relevant clinical efficacy (CE) metrics such as macro-precision, recall, or F1-score should be reported, following prior work like METransformer [2]. This is especially crucial since the base model’s performance on SLAKE and VQA-RAD is already low. Without CE metrics, it is unclear whether improvements are meaningful for medical diagnosis.\n\n[1] LLaVA-Med: Training a Large Language-and-Vision Assistant for Biomedicine in One Day. NeurIPS 2023.\n\n[2] METransformer: Radiology Report Generation by Transformer with Multiple Learnable Expert Tokens. CVPR 2023.", "relation_to_broader_scientific_literature": "The paper contributes to preference optimization for Med-LVLMs, building on techniques like DPO, self-rewarding methods, and multimodal alignment.", "essential_references_not_discussed": "None", "other_strengths_and_weaknesses": "*Strengths\n\nThe combination of hallucination-based and lesion-region dispreference is an interesting strategy.\n\n*Weaknesses\n\n1. Empirical results are inconsistent across datasets, and some results need explanation.\n\n2. Mathematical formulation has notation inconsistencies, making it difficult to follow the optimization steps.\n\n3. Weighting preference samples based on clinical relevance is heuristically defined.", "other_comments_or_suggestions": "None.", "questions_for_authors": "1. Why does the base model (LLaVA-Med v1.5) perform significantly worse than reported in its original paper on SLAKE and VQA-RAD?\n\n2. Have you considered a baseline where all preference samples are weighted equally?\n\n3. Would other types of perturbations (e.g., adversarial noise, saliency masking) improve lesion-based preference optimization?", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "CZvymDdg3O", "reviewer": "Reviewer_MzeL", "summary": "This work proposes MMedPO, a DPO-based Preference Alignment paradigm that aims to let LVLMs provide more accurate and expertise textual responses to X-ray/medical images. The authors design a way to curate preferred-and-dispreferred responses using hallucination-inducing noisy images, and proposes a quantified metric called Clinical Relevance Score to give weights to the curated preference data. Applied to LLaVA-Med 1.5, MMedPO helps achieve SOTA performances on multiple medical VQA benchmarks.", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for reviewing our paper and for your valuable feedback. Below, we address your concerns point by point. We would appreciate it if you could let us know whether your concerns are addressed by our response. **All tables referenced in this rebuttal can be found in** https://anonymous.4open.science/r/ICML_rebuttal-0304/README.md\n*****\n\n>**Q1**: Given the sensitive nature of the task and the data involved, is there any human involved when quantifying the relevances during the Multi-Agent step? Even though Medical AI agents are highly capable these days, one can't be 100% sure if only relying on LLMs can guarantee the most expertise evaluations. If there is no human-in-the-loop, what is the reason behind such a decision in design?\n\n**A1**: To ensure reliability, we additionally involved human experts in the evaluation process. Specifically, we selected 100 samples and invited three experienced clinical experts to assign clinical relevance scores to the preference data. We then compared these human scores with those generated by Med-LLMs using multiple correlation metrics.\nAs shown in **Table R6**, the Med-LLM scores exhibit strong correlation with human judgments, validating their effectiveness. Moreover, the multi-LLM setting outperforms the single-LLM approach, showing better alignment with expert assessments.\nThis human-in-the-loop validation supports the credibility of our automated scoring mechanism while demonstrating that well-designed multi-agent LLM setups can approximate expert-level evaluations with reasonable accuracy.\n\n****\n\n>**Q2**: When assigning CRS to the curated preference data, is the preferred response also given weights by the agents, presumably always larger than its dispreferred counterpart? Please clarify if CRS's scale is 'relevant' (where the preferred is always assumed to be 1.0), or 'absolute'; if CRS is a 'relevant' scale, it might not best reflect how a response is objectively good in terms of clinical response quality.\n\n**A2**: Thank you for the question. To clarify, we assign the Clinical Relevance Score (CRS) only to the dispreferred response in each DPO preference pair. The preferred response is not explicitly scored, as its superiority is already established through the preference annotation.\nTherefore, the CRS is defined in an absolute manner rather than a relative scale. That is, it directly reflects the clinical quality of the dispreferred response, independent of its counterpart. This allows us to distinguish between cases where the dispreferred response is still reasonably acceptable (e.g., CRS ≈ 0.8) versus cases where it is clearly irrelevant or incorrect (e.g., CRS ≈ 0.2).\n****\n>**Q3**: L171, the reference to Chan et al is apparently missing the year.\n\n**A3**: Thank you for pointing this out. We will correct the citation and include the missing year for the reference to Chan et al.", "claims_and_evidence": "No claim in this work is overtly outlandish that needs special attention.", "methods_and_evaluation_criteria": "My main concerns are regarding the scheme of CRS in Step II of Figure 2, or Section 3.2 .\n\n* Given the sensitive nature of the task and the data involved, **is there any human involved** when quantifying the relevances during the Multi-Agent step? Even though Medical AI agents are highly capable these days, one can't be 100% sure if only relying on LLMs can guarantee the most expertise evaluations. If there is no human-in-the-loop, what is the reason behind such a decision in design?\n\n* When assigning CRS to the curated preference data, **is the preferred response also given weights by the agents** , presumably always larger than its dispreferred counterpart? Please clarify if CRS's scale is 'relevant' (where the preferred is always assumed to be 1.0), or 'absolute'; if CRS is a 'relevant' scale, it might not best reflect how a response is *objectively good* in terms of clinical response quality.", "theoretical_claims": "All underlying theories are in line with previous works that involve DPO.", "experimental_designs_or_analyses": "I am assuming evaluating only one backbone model that is LLaVA-Med-1.5 is sufficient, given the general difficulty to ethically obtain medical data in the first place.", "relation_to_broader_scientific_literature": "The proposed approach can be extended beyond medicine-related human preference alignments. As long as we need a more reliable LVLM with a certain highly sophisticated field (Security, Education, to name a few), we may apply the same expertise-aligning curation strategy proposed in MMedPO in a similar way.", "essential_references_not_discussed": "None.", "other_strengths_and_weaknesses": "Please find my main concerns regarding CRS in the Methods And Evaluation section.", "other_comments_or_suggestions": "L171, the reference to Chan et al is apparently missing the year.", "questions_for_authors": "None.", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "STh3lbhO50", "reviewer": "Reviewer_MFrc", "summary": "The paper proposed a novel medical large visual language model (LVLM) preference optimization alignment method called MMedPO. To solve the hallucination issue in the regular preference data curation process, MMedPO uses existing VLM and LLM to create depreferred data with better clinical correspondence. It further uses the Med-KLIP model to localize the region of interest within the image and create depreferred data by adding noise to these regions. To further improve the alignment process, the author further proposes to use 2 significant scores to weight the preference optimization loss. The proposed method is proven to outperform baselines in 4 different medical VQA and report generation dataset.", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for your constructive comments and suggestions. **All tables and images referenced in this rebuttal can be found in** https://anonymous.4open.science/r/ICML_rebuttal-0304/README.md\n****\n>**Q1**: The performance improvement with SFT is much smaller.\n\n**A1**: As clarified in Section 4.2, we report MMedPO’s performance in both with- and without-SFT settings. Even when combined with SFT, MMedPO yields consistent gains across all four datasets, with an average improvement of 10.5%. While gains may be smaller in absolute terms, these results demonstrate the robustness and modularity of our method, highlighting its compatibility with other training strategies.\n****\n>**Q2**: The quality of the generated dispreferred data pair [a].\n\n[a] Yan, Qianqi, et al. Worse than random? arXiv 2024.\n\n**A2**:  Dispreferred responses are generated via GPT-4o with guided hallucinations to ensure clinical plausibility, while preferred ones are real reports. Expert review gave dispreferred samples an average score of 6.4/10, confirming suitability for preference optimization. We’ll include example pairs in the final version for transparency.\n****\n>**Q3**: The relevance score of these generated reports is also evaluated by text-only LLM..\n\n**A3**: In our setting, the evaluated reports are intentionally perturbed via GPT-4o to introduce plausible errors. Therefore, the role of the text-only LLM is not to verify image-text alignment, but to assess the clinical severity and plausibility of these injected faults. In other words, the LLM is used to judge whether a faulty report still retains clinical value (e.g., contains common or benign errors) or if it includes clear factual mistakes that are unlikely to be made by competent practitioners. This scoring helps us assign lower training weights to less harmful dispreferred responses, and higher weights to truly misleading ones — ultimately improving the effectiveness and safety of preference optimization.\n****\n>**Q4**: The evidence provided in Figure 5 …the image.\n\n**A4**: In Figure 5, our method enhances the model's attention to the image, suggesting improved focus on relevant regions. To make this more intuitive and convincing, we have added visualizations of the attention weights as overlays on the image (**Image R1-R5**). These visualizations clearly illustrate where the model is attending within the image. We will include several examples of these attention visualizations in the revised version of the PDF.\n****\n>**Q5**: One additional scaling experiment on VLMs of different sizes ....\n\n**A5**: As shown in **Table R7**, our method has been successfully extended to Med-VLMs based on both VILA-M3-8B and VILA-M3-13B [R1]. In both cases, we observe consistent and significant performance improvements, demonstrating the generalizability and scalability of our approach across models of different sizes. \n\n[R1] Nath V, et al. VILA-M3. CVPR 2025.\n****\n>**Q6**: Some of the designs for the method… expert evaluations..\n\n**A6**: We conducted expert evaluation on 50 samples, where three clinicians rated reports using a 5-point scale (Excellent, Good, Fair, Poor, Very Poor). As shown in **Table R8**, our method consistently received higher scores than baselines, supporting the effectiveness of our design choices.\n****\n>**Q7**: The evaluation in Table 4 compared global noise...random noise masking.\n\n**A7**: We incorporated random noise masking and observed performance gains. As shown in **Table R5**, while different noise types yield comparable results, the improvement largely stems from lesion-aware preference optimization, underscoring the value of localized, clinically meaningful perturbations. Thanks!\n****\n>**Q8**: Providing more examples of the generated depreferred data.\n\n**A8**: We will provide more examples of dispreferred data and more cases in the future version. We will show more examples like Figure 6 in the future version.\n****\n>**Q9**: There is no discussion on the reliability of these composing VLM/LLMs used in the method.\n\n**A9**: We agree reliability is crucial. While we use LLaVA-Med v1.5 for its strong benchmark performance, we recognize its limitations in complex cases. To enhance robustness, we adopt a multi-agent setup with cross-checking. We’ll revise the manuscript to discuss these concerns and cite [a–c], while highlighting future directions like human-in-the-loop and confidence calibration.\n****\n>**Q10**: How does the generated dispreferred data look like? Is it possible to evaluate the quality of these data quantitatively?\n\n**A10**: Dispreferred responses are generated by injecting plausible hallucinations into preferred ones using GPT-4o. We will include more examples in the final version. For evaluation, we use a clinical relevance score to assess each dispreferred response, which also serves as a weight in preference learning. We agree that human expert validation is important and plan to include it in future work to further assess clinical soundness.", "claims_and_evidence": "While the proposed method seems to have improved the performance of baseline Med-LVLM in the evaluation, the reviewer still has the following concerns about the claims made in the paper.\n\n1. The performance improvement is not as obvious as mentioned in the abstract. While the relative performance is indeed improved in the setting without SFT, the performance improvement with SFT is much smaller in all 4 datasets, mostly only around 1%.\n2. Additionally, the reviewer is also concerned about the quality of the generated depreferred data pair. On the one hand, existing papers [a] have illustrated that the LLMs like LLaVA-1.5 and GPT-4o are strongly biased and perform poorly on medical VQA tasks. Generating and evaluating depreferred reports with these models is not very convincing. On the other hand, there are no examples provided in the paper for the generated preferred data. Making it harder to evaluate the quality of this claim.\n3. Moreover, the relevance score of these generated reports is also evaluated by text-only LLM. The author claims it is better since the model should solely focus on the internal medical knowledge. However, it is possible that the generated data is internally correct but does not align with the corresponding image; the relevance score generated in this case may be incorrect as well.\n4. Additionally, the visual noise masking is designed to make the model focus more on the region of interest during evaluation. However, the evidence provided in Figure 5 is not very intuitive and convincing to the reviewer. An enhanced vision attention does not mean it corresponds to the correct region. It would be more effective to visualize the attention weight as an overlay on the image.\n5. Lastly, it seems that the paper only evaluated the proposed method on two 7B level VLMs. This is insufficient to validate the effectiveness of the proposed method. It is expected to have at least one additional scaling experiment on VLMs of different sizes to prove its effectiveness.\n\n\n[a] Yan, Qianqi, et al. \"Worse than random? An embarrassingly simple probing evaluation of large multimodal models in medical VQA.\" arXiv preprint arXiv:2405.20421 (2024).", "methods_and_evaluation_criteria": "As mentioned above, it is hard to tell if the model is generally significant to the domain for the following reasons:\n\n1. The performance improvement is limited, especially in the case of using STF. The lack of scaling experiments further makes it harder to provide meaningful insight for the domain. Considering the fact that the method itself is relatively simple, it is also expected to evaluate it on more baselines.\n2.  While the ablation experiments are helpful in terms of understanding the behavior of each component and proved they are effective. The reviewer is still not convinced by some of the designs for the method. It would be helpful if more examples or expert evaluations were provided.", "theoretical_claims": "N/A. There is no novel theoretical claim proposed in the paper.", "experimental_designs_or_analyses": "This paper has provided a complete major evaluation on 4 different datasets and have compared with multiple baselines. The ablation experiment is also included. However, there are a few concerns in terms of the soundness of the experimental design. \n\n1. As mentioned above, the visualization in Figure 5 is not that straightforward and convincing in terms of proving its claim.\n2. The evaluation in Table 4 compared global noise against the proposed local noise masking, but it might be helpful to also compare it with random noise masking.\n3. Providing more examples of the generated depreferred data can also help validate the effectiveness of the proposed method.", "relation_to_broader_scientific_literature": "The proposed method is a medical variation of the Direct preference optimization method. It is developed via using multiple existing VLM/LLMs. However, there is no discussion on the reliability of these composing VLM/LLMs used in the method.", "essential_references_not_discussed": "Including more discussion about the reliability of the VLM/LLMs used here, like [a], will be helpful.\n\n[a] Yan, Qianqi, et al. \"Worse than random? An embarrassingly simple probing evaluation of large multimodal models in medical VQA.\" arXiv preprint arXiv:2405.20421 (2024).\n[b] Shi, Congzhen, et al. \"A survey on trustworthiness in foundation models for medical image analysis.\" arXiv preprint arXiv:2407.15851 (2024).\n[c] Nakamura, Yuta, et al. \"It is not time to kick out radiologists.\" Asian Bioethics Review 17.1 (2025): 9-15.", "other_strengths_and_weaknesses": "As discussed above, there are a few fundamental concerns about the proposed method. Although the experimental results demonstrate it is better than the baselines in the given evaluation settings. The reviewer believes that the significance and soundness of the paper may still need to be improved.", "other_comments_or_suggestions": "N/A", "questions_for_authors": "1. How does the generated depreferred data look like? Is it possible to evaluate the quality of these data quantitatively?\n2. It would be great if the authors could provide more examples like in Figure 6, but not healthy data. The unhealthy examples are often of more interest in medical VQA or report generation.", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "dw90iD3KBR", "reviewer": "Reviewer_HmNL", "summary": "This paper introduces MMedPO, a clinical-aware multimodal preference optimization approach for aligning Medical Large Vision-Language Models (Med-LVLMs). The authors leverage sentence corruption techniques from GPT-4o and a noise interaction process to generate rejected samples. They then use specialized tools to create positive answer samples from original images and employ multi-agent collaboration to evaluate the clinical relevance of the generated rejection samples. Experimental results demonstrate the effectiveness of the proposed methodology.\n\n## update after rebuttal\nThe rebuttal dose help clarify my concerns and I have increased the score", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for your valuable feedback to help us improve our paper. **All tables and images referenced in this rebuttal can be found in** https://anonymous.4open.science/r/ICML_rebuttal-0304/README.md\n\n>**Q1**: More diverse clinical datasets could strengthen the evaluation.\n\n**A1**:  We conducted experiments on a diverse dataset PMC-VQA [R1], which includes various medical imaging modalities. As shown in **Table R9**, our method demonstrates significant improvements over several baseline methods, further validating the generalizability of our approach.\n\n[R1] Zhang X et al. PMC-VQA. Nature Communications Medicine 2025.\n****\n>**Q2**: ... doesn't present significant theoretical proofs…\n\n**A2**: Due to the rebuttal length constraints, we were unable to include the theoretical proofs in the rebuttal. We plan to add theoretical justifications in the revised PDF to demonstrate that our method can theoretically enhance model performance based on DPO. We aim to formally analyze the interplay between noise, relevance, and preference optimization — potentially building connections to existing theories.\n****\n>**Q3**: Lacks comprehensive implementation specifications, including baseline method details, hyperparameter selection, and ablation studies. \n\n**A3**: We will include detailed implementation specs in the revised manuscript. Baselines like DPO, SFT, STLLaVA-Med, FiSAO, and others follow official or original pipelines. Our method uses LoRA (rank 128, alpha 256), a 2e-5 learning rate for the multimodal projector, cosine schedule with 3% warmup, and DeepSpeed ZeRO-3. Hyperparameters align with prior work (e.g., LLaVA-Med) for fair comparison. Our key distinction is incorporating clinical relevance into preference tuning, guiding the model toward medically meaningful outputs. Detailed ablations will also be added.\n****\n>**Q4**: The authors should discuss recent papers.\n\n**A4**:  We agree and will revise the related work section to include recent Med-LVLM alignment efforts, domain-specific preference learning methods, and clinical validation metrics. This will better contextualize our contributions and highlight key distinctions.\n****\n>**Q5**: The novelty … feature-level [1], image-level [2], and text-level noise injection [3]...\n\n[1] Strengthening… Optimization [2] Self-Supervised … Alignment [3] Aligning … Fine-Tuning [4] SILKIE\n\n**A5**: MMedPO introduces clinically meaningful corruptions targeting disease-relevant regions, enabling fine-grained, domain-aware supervision. Unlike prior methods using global noise (e.g., BPO, POVID), MMedPO integrates clinical context directly into both preference construction and optimization, achieving more precise and effective alignment for medical tasks.\n****\n>**Q6**: give a short explanation of what the 'tools' represent in Figure 2' captions\n\n**A6**: *Tools* refers to external medical visual grounding models used to extract region-level information from the image. Specifically, we utilize MedKLIP as the visual tool $T(x_v)$, which predicts disease-related local regions $h = T(x_v)$ for each medical image $x_v$. This enables the model to better attend to clinically relevant visual features during alignment and scoring. We will revise the figure caption to clarify this point.\n****\n>**Q7**: How does MMedPO specifically address medical domain challenges that general MLLM alignment methods cannot?\n\n**A7**: Unlike general MLLM alignment methods, MMedPO is tailored for medical tasks through two key components: (1) visual grounding via tools like MedKLIP to focus on disease-relevant regions, and (2) a clinical relevance scorer to prioritize responses that are not just fluent but medically meaningful. This ensures alignment better suited to the precision demands of the medical domain.\n****\n>**Q8**: What safeguards ensure the clinical relevance scoring accurately reflects medical expertise?\n\n**A8**: We validated the scoring by comparing Med-LLM outputs with ratings from three clinical experts on 100 samples. The strong correlation (**Table R6**) confirms alignment with expert judgment. Notably, the multi-LLM setup outperformed single-LLM scoring, demonstrating that our approach approximates expert-level evaluation with credible accuracy.\n****\n>**Q9**: How was the performance validated with actual healthcare professionals?\n\n**A9**:  We conducted an expert evaluation on 50 samples, where three medical professionals rated reports from our method and baselines using a 5-point scale (Excellent, Good, Fair, Poor, Very Poor). As shown in **Table R8**, our method consistently received higher scores, confirming its clinical effectiveness.\n****\n>**Q10**: What specific metrics were used to evaluate clinical relevance beyond standard VQA metrics?\n\n**A10**: We use a Med-LLM-based clinical relevance score that assesses the appropriateness and usefulness of answers in context. Unlike standard VQA metrics (e.g., accuracy), this score reflects real-world clinical value and aligns with expert judgment.", "claims_and_evidence": "The paper's methods align with the medical visual language model alignment domain. The approach of using clinical awareness for multimodal preference optimization addresses the specific challenges of medical image interpretation. The evaluation on standard medical datasets (SLAKE, VQA-RAD) is appropriate for benchmarking Med-LVLM performance, though more diverse clinical datasets could strengthen the evaluation.", "methods_and_evaluation_criteria": "The paper's methods align with the medical visual language model alignment domain. The approach of using clinical awareness for multimodal preference optimization addresses the specific challenges of medical image interpretation. The evaluation on standard medical datasets (SLAKE, VQA-RAD) is appropriate for benchmarking Med-LVLM performance, though more diverse clinical datasets could strengthen the evaluation.", "theoretical_claims": "The paper doesn't present significant theoretical proofs, focusing instead on empirical validation. The conceptual framework for noise interaction and clinical relevance scoring is sound but would benefit from more theoretical grounding on why these specific approaches enhance medical domain alignment better than alternatives.", "experimental_designs_or_analyses": "The experimental design demonstrates improvements over baselines, but lacks some critical details:\n1.\tImplementation details for baseline methods (DPO, SFT) are insufficiently described\n2.\tHyperparameter selection process is not transparent\n3.\tThe ablation studies are thorough but could more clearly isolate the impact of clinical awareness versus general preference optimization", "relation_to_broader_scientific_literature": "The paper builds upon preference optimization in multimodal LLMs but applies it to the medical domain. The approach resembles general MLLM preference distillation techniques [4] with domain-specific adaptations. The sentence corruption technique borrows from GPT-4o, while the noise interaction process has similarities to methods in [1] and [2]. The clinical scoring component appears to be the most novel contribution relative to existing literature.", "essential_references_not_discussed": "The authors should discuss:\n1.\tRecent Med-LVLM alignment papers that use similar corruption techniques\n2.\tDomain-specific preference learning approaches in other specialized fields that could provide contextual comparison\n3.\tLiterature on clinical validation metrics for AI-generated medical explanations", "other_strengths_and_weaknesses": "Paper Strengths\n1. The paper presents a clear and well-organized structure, with detailed algorithms and illustrative figures that effectively communicate the core components of the proposed approach.\n2. The experimental results are compelling, with comprehensive evaluations on standard medical image datasets (SLAKE, VQA-RAD, etc.) demonstrating the effectiveness of MMedPO through fair comparisons.\n3. The thorough ablation studies effectively validate each component of MMedPO, particularly Figure 3, which highlights the importance of integrating both textual and visual cues during the alignment process.\n\nMajor Weaknesses\n1. The novelty of MMedPO appears limited when compared to existing approaches for multimodal alignment. Similar techniques have been previously explored, including feature-level noise injection [1], image-level noise injection [2], and text-level noise injection [3]. The authors should clearly articulate the distinctive aspects of their approach compared to these existing methods, particularly [1] and [2], to establish MMedPO's novel contribution.\n2. While the authors claim to construct preference samples based on clinical relevance scores, this methodology closely resembles preference distillation approaches already established for general multimodal language models [4]. The primary distinction appears to be the medical domain focus rather than a fundamental methodological innovation.\n3. Experimental details lack sufficient clarity, particularly regarding the implementation of baseline DPO and SFT methods. The main paper and supplementary materials provide only general descriptions without specific implementation details. More comprehensive documentation would help verify that performance improvements stem from the method itself rather than hyperparameter optimization.\n4. It will be better if the author give a short explanation of what the 'tools' represent in Figure 2' captions\n\nReferences:\n[1] Strengthening Multimodal Large Language Model with Bootstrapped Preference Optimization\n[2] Self-Supervised Visual Preference Alignment\n[3] Aligning Modalities in Vision Large Language Models via Preference Fine-Tuning\n[4] SILKIE: Preference Distillation for Large Visual Language Models", "other_comments_or_suggestions": "The paper would benefit from:\n•\tA more detailed comparative analysis with existing methods [1-4]\n•\tAdditional experiments demonstrating real-world clinical utility\n•\tClearer discussion of limitations and potential negative impacts in healthcare settings", "questions_for_authors": "1.\tHow does MMedPO specifically address medical domain challenges that general MLLM alignment methods cannot?\n2.\tWhat safeguards ensure the clinical relevance scoring accurately reflects medical expertise?\n3.\tHow was the performance validated with actual healthcare professionals?\n4.\tWhat specific metrics were used to evaluate clinical relevance beyond standard VQA metrics?", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Meta-review: The submission presents a novel approach for aligning medical vision-language models via clinical-aware multimodal preference optimization, which reviewers identified as a key strength. The method’s integration of lesion-level perturbations and its demonstrated improvements across multiple medical VQA and radiology report generation benchmarks were especially praised as innovative and impactful contributions. The reviewers did raise several concerns, notably an unclear definition of “hallucination” in the context of the paper, limited theoretical grounding for the proposed methods, and some empirical inconsistencies (particularly regarding the performance of the base model in ablation studies). In their rebuttal, the authors effectively addressed many of these issues with additional experiments, expert clinician evaluations, and detailed clarifications—especially shedding light on how clinical relevance is scored and providing evidence of improved visual attention alignment between the model and important clinical image features. While not every critique was fully resolved, the consensus is that the paper’s strengths far outweigh the remaining concerns. Overall, this work is deemed a valuable and practical contribution to the field of medical AI alignment, backed by sound methodology and promising results, and thus the decision is to accept this submission.
Decision: Accept (poster)