Worse than Random? An Embarrassingly Simple Probing Evaluation of Large Multimodal Models in Medical VQA
Authors: Qianqi Yan, Xuehai He, Xiang Yue, Xin Eric Wang
Accepted at: acl25

Worse than Random? An Embarrassingly Simple Probing Evaluation of Large Multimodal Models in Medical VQA

Qianqi Yan1, Xuehai He1, Xiang Yue2, Xin Eric Wang1

1University of California, Santa Cruz, 2Carnegie Mellon University

![Bar charts showing accuracy of six LMMs on two types of specialized questions in medical diagnoses, with and without adversarial pairs.](page_347_682_1057_312.png)

Figure 1: Accuracy of six LMMs on two types of specialized questions in medical diagnoses, with and without adversarial pairs. The significant drop in accuracy with adversarial pairs highlights the models’ unreliability in handling medical diagnoses with our probing method.

Abstract

Large Multimodal Models (LMMs) have demonstrated impressive performance on existing medical Visual Question Answering (Med-VQA) benchmarks. However, high reported accuracy does not necessarily reflect their true diagnostic reliability in clinical settings. This study reveals that state-of-the-art models perform worse than random guessing on medical diagnosis questions when subjected to simple Probing Evaluation for Medical Diagnosis (ProbMed). ProbMed challenges models through probing evaluation and procedural diagnosis. Particularly, probing evaluation features pairing ground-truth questions with adversarial counterparts that feature negated and hallucinated attributes, while procedural diagnosis requires reasoning across various dimensions for each image, including modality recognition, organ identification, clinical findings, abnormalities, and positional grounding. Our evaluation reveals that even top-performing models like GPT-4o, GPT-4V, and Gemini Pro perform worse than random guessing on specialized diagnostic questions, indicating significant limitations in handling fine-grained medical inquiries. Furthermore, our ablation study on open-source models (e.g., LLaVA, LLaVA-Med, and Med-Flamingo) identifies poor visual understanding as a primary bottleneck—a limitation that can be partially mitigated by incorporating visual descriptions generated by GPT-4o, resulting in an average performance improvement of 9.44%. These findings underscore the urgent need for more robust evaluation methods and domain-specific expertise to ensure the reliability of LMMs in high-stakes medical applications.

1 Introduction

Foundation models, such as large language models (LLMs) (Achiam et al., 2023; Touvron et al., 2023; Jiang et al., 2023; Anil et al., 2023; Chung et al., 2024) and large multimodal models (LMMs) (OpenAI, 2024, 2023; Reid et al., 2024; Li et al., 2023; Liu et al., 2023a; Chen et al., 2023), have demonstrated impressive capabilities in understanding complex visual and text inputs, generating human-like language, and achieving high accuracy on various benchmarks. The integration of these foundation models into real-life medical practice holds immense potential given their advanced computational capabilities (Wu et al., 2023a; Yang et al., 2023) and promising progress on existing medical
Figure 2: An example illustrating the potential for misleading accuracy in existing evaluations. While the model correctly identifies the position of an existing finding in the standard evaluation, it fails to differentiate between actual and hallucinated positions when subjected to an adversarial evaluation.

Visual Question Answering (Med-VQA) benchmarks (Lau et al., 2018; Liu et al., 2021; He et al., 2020; Zhang et al., 2023). As we stand on the precipice of integrating these models into critical decision-making domains, one natural question appears: how much can we trust these models in real-world scenarios, such as medicine and healthcare, where the stakes are high?

Before discussing the reliability of LMMs in critical domains like Med-VQA, we must first address a fundamental question: Are we evaluating LMMs correctly? To address this question, we introduce a simple yet effective probing evaluation method that exposes the weaknesses of LMMs by creating binary questions with hallucination pairs over existing benchmarks. An example is shown in Figure 2. Despite the high accuracy reported on current Med-VQA tasks, our study reveals a significant vulnerability in LMMs when faced with adversarial questioning, as illustrated in Figure 1. The observed performance drops are alarming: even advanced models like GPT-4o, GPT-4V, and Gemini Pro perform worse than random guessing, with an average decrease of 27.78% across the tested models.

Based on this, we further analyze a critical question: How reliable are LMMs in medical diagnosis, ranging from general questions to specialized diagnostic questions? To address this question, we introduce ProbMed, which features procedural diagnosis designed to rigorously evaluate model performance across multiple diagnostic dimensions. We curated ProbMed from 6,303 images sourced from two widely-used biomedical datasets, MedICaT (Subramanian et al., 2020) and ChestX-ray14 (Wang et al., 2017). These images cover various modalities, including X-ray, MRI, and CT scans, and span multiple organs such as the abdomen, brain, chest, and spine. Using GPT-4 and a positional reasoning module, we generated metadata for each image, extracting information about abnormalities, condition names, and their corresponding locations. This metadata facilitated the automatic generation of 57,132 high-quality question-answer pairs, covering dimensions like modality recognition, organ identification, abnormalities, clinical findings, and positional reasoning.

Our systematic evaluation of twelve state-of-the-art LMMs on ProbMed revealed several critical insights. First, even the best-performing models, such as GPT-4o and Gemini Pro, performed close to random guessing on specialized diagnostic categories like Condition/Finding and Position, highlighting their limitations in handling fine-grained medical inquiries. Second, introducing adversarial pairs significantly reduced the accuracy of all models, with LLaVA-Med-v1.5’s performance dropping by up to 28.22% and GPT-4o’s accuracy decreasing by 20.71% in ProbMed. These findings emphasize the importance of adversarial testing in Med-VQA to uncover model weaknesses. Third, by incorporating chain-of-thought reasoning and adding visual descriptions generated by GPT-4o, we observe substantial improvements in model performance, suggesting that poor visual understanding is a critical bottleneck. The results indicate that augmenting these models with more accurate visual information could significantly improve their ability to handle complex medical tasks. Moreover, the CheXAgent model, which was exclusively trained on chest X-rays, demonstrated that specialized domain knowledge is crucial. It showed that expertise gained on one particular organ could be transferable to another modality of the same organ in a zero-shot manner, highlighting the value of domain-specific training for improving model performance.

In summary, our work highlights significant gaps in the reliability of LMMs for medical diagnosis despite their impressive performance on existing general domain benchmarks. The insights from ProbMed underscore the urgent need for robust evaluation methodologies to ensure the accuracy and reliability of LMMs in real-world medical applications. Our findings also suggest that poor visual understanding is a key limitation for open-source models, which can be mitigated by incorporating chain-of-thought reasoning and accurate visual descriptions. This research inspires the devel-
opment of more trustworthy AI systems in healthcare and beyond, ultimately contributing to better diagnostic outcomes and patient care.

2 Related Work

Large Multimodal Models in the Medical Domain The advancements in Large Multimodal Models (LMMs) have significantly enhanced the understanding and generation of medical content that integrates both visual and linguistic elements. Notable models include GPT-4o (OpenAI, 2024), GPT-4V (OpenAI, 2023), Gemini Pro (Reid et al., 2024), LLaVA (Liu et al., 2023a, 2024), and MiniGPT-v2 (Chen et al., 2023). The scalability and exceptional performance of these large foundation models have driven their application in the biomedical field.

Further progress has been made in fine-tuning general-domain LMMs for the biomedical field, resulting in specialized models like BiomedGPT (Zhang et al., 2024), LLaVA-Med (Li et al., 2024), Med-Flamingo (Moor et al., 2023), MedBLIP (Chen and Hong, 2024), RadFM (Wu et al., 2023b) and MedViNT (Zhang et al., 2023). Despite the promising results from these domain-specific LMMs, ongoing exploration exists into training smaller multimodal models to address specific clinical needs. For instance, models like LLaVA-RAD (Chaves et al., 2024) and CheXAgent (Chen et al., 2024) have been developed for chest X-ray interpretation, aiming to bridge competency gaps in radiology tasks.

Comprehensive surveys of LLMs for healthcare highlight the progress, applications, and challenges in deploying LLMs in clinical settings (He et al., 2023; Zhou et al., 2024; Peng et al., 2023). Task-specific evaluations (Yan et al., 2023; Liu et al., 2023b) underline the potential and challenges of LMMs in the medical domain. As we move towards integrating these models into critical decision-making processes, it becomes imperative to assess their reliability in high-stakes environments like healthcare and medicine.

Medical Visual Question Answering Medical Visual Question Answering (Med-VQA) plays a crucial role in assessing the capabilities of models in interpreting and responding to queries about medical images. Some benchmarks, like VQA-RAD (Lau et al., 2018) and SLAKE (Liu et al., 2021), are manually constructed with categorical question types. While this method ensures high-quality question-answer pairs, it is labor-intensive and results in limited dataset scales.

Automated curation methods have been developed to address scalability. PathVQA (He et al., 2020) uses CoreNLP1 tools, and PMCVQA (Zhang et al., 2023) employs generative models to create larger datasets. However, these methods often sacrifice fine-grained question categories, and some require additionally trained models for question filtering.

Different evaluation methods are employed for assessing LMMs, including closed-ended VQA, multiple choice VQA, and open-ended generation tasks such as captioning and report generation. Open-ended VQA and report generation are typically considered more challenging and harder to evaluate, often requiring human or model evaluation alongside automated lexical similarity metrics like ROUGE-L and BLEU-4. Recent works (Wang et al., 2024; Zheng et al., 2024; Zong et al., 2024) argue that multiple-choice questions may not be ideal due to inherent selection bias and permutation sensitivity. In our work, we choose a relatively easy-to-evaluate method: closed-ended VQA augmented with adversarial evaluation methods featuring hallucinated attributes. By requiring the model to accurately distinguish relevant features, we enhance the reliability of the evaluation process. This method allows for clear and definitive assessments, improving the overall robustness of our findings in medical contexts.

3 Probing Evaluation for Medical Diagnosis

In this section, we introduce two complementary evaluation strategies that rigorously assess state-of-the-art LMMs for Med-VQA. Our approach is designed to answer the following research questions:

1. Is the current evaluation of LMMs for Med-VQA reliable?

2. How reliable are LMMs on medical diagnosis, ranging from general questions to specialized diagnostic questions?

Despite current high accuracy, we find that the models struggle with simple probing evaluation on existing benchmarks. Our evaluation framework, ProbMed (Probing Evaluation for Medical

1https://stanfordnlp.github.io/CoreNLP
Figure 3: Flow diagram of the ProbMed data curation process. Two comprehensive biomedical datasets were utilized to collect source data and construct a metadata file, enabling the automatic generation of high-quality question-answer pairs for the ProbMed dataset.

Diagnosis), is built on adversarial testing and multi-faceted diagnostic reasoning to further expose these vulnerabilities and provide a thorough analysis of model performance. We also explore enhancements through chain-of-thought reasoning and the incorporation of external visual descriptions (e.g., from GPT-4o) to address the noted limitations of open-sourced models.

3.1 Probing Evaluation with Adversarial Pairs

A core element of our framework is the use of adversarial pairs to test model robustness. For each image, we generate pairs of: ground-truth questions that query the presence of a specific entity (e.g., a particular clinical finding) with corresponding adversarial questions that introduce a negated or hallucinated attribute (e.g., a non-existent finding or an alternative organ). This pairing challenges the model to discern between actual diagnostic features and spurious details, thereby revealing its ability—or inability—to filter out irrelevant or misleading information. The performance drop observed under adversarial conditions highlights the fragility of current evaluation protocols and motivates the need for more robust assessment methods.

3.2 Procedural Diagnosis

Beyond binary question-answering, ProbMed incorporates procedural diagnosis to evaluate the models’ diagnostic reasoning. Each image is associated with questions spanning multiple diagnostic dimensions, including Modality Recognition: Identifying the imaging technique (e.g., X-ray, MRI, CT). Organ Identification: Determining the anatomical region under investigation. Clinical Findings and Abnormalities: Detecting abnormal conditions. Positional Reasoning: Localizing findings spatially within the image. This multifaceted evaluation framework requires models to integrate diverse pieces of information for each test sample, thereby providing a more comprehensive measure of their diagnostic capabilities.

3.3 Data Filtering and Curation

ProbMed is curated from two widely recognized biomedical datasets: MedICaT and ChestX-ray14. The data curation process, summarized in Figure 3, involves the following steps:

Image Selection: From MedICaT (Subramanian et al., 2020), we extracted 4,543 image-caption pairs that focus on a single organ and modality with clear indications of normal or abnormal conditions. From ChestX-ray14 (Wang et al., 2017), we
selected 1,760 frontal-view X-ray images balanced between healthy and abnormal cases, including those with bounding box annotations for abnormalities.

Metadata Generation: For each image, we generate a unified metadata record:

\[
D_i = \{\text{mod}_i, \text{organ}_i, \{\text{condition}_j, \text{pos}_j\}_{j=1}^{n_i}\}
\]

where \( \text{mod}_i \) and \( \text{organ}_i \) denote the imaging modality and anatomical region, respectively, and each \( \{\text{condition}_j, \text{pos}_j\} \) pair represents a detected clinical finding and its positional description. For MedICaT, GPT-4 is leveraged via few-shot prompting to extract abnormality details and positional cues from captions. For ChestX-ray14, a dedicated positional reasoning module generates descriptive text based on bounding box coordinates.

3.4 Evaluation Protocol
For each diagnostic entity in the metadata, we automatically generate a ground-truth question \( Q_i \), asking the model to confirm the presence of that specific entity. An adversarial question \( Q'_i \), constructed by randomly selecting an alternative or hallucinated entity (e.g., an incorrect organ or false condition) and expecting a “no” response.

Crucially, our accuracy metric is defined in a strict manner: an entity is considered correctly predicted only if the model provides the correct answer for both \( Q_i \) and \( Q'_i \). In other words, if a model answers “yes” to both questions for a given entity, it is deemed incorrect rather than receiving partial credit. For images containing more than one \( \{\text{condition}_j, \text{pos}_j\} \) pair, the accuracy under the Condition/Finding and Position category is computed as the average accuracy over all \( n_i \) pairs—there is no bonus for partial correctness. This evaluation setup ensures that only unambiguous, fully correct responses are counted as hits, highlighting the models’ true diagnostic reliability. (See Appendix B for detailed statistics on the number of questions in each category.)

3.5 Expert Study
To validate the reliability of our metadata and the corresponding question-answer pairs, we conducted an expert verification study. Two medical experts independently reviewed 100 randomly sampled metadata entries out of 6,303 from ProbMed, as well as the 1,090 QA pairs corresponding to those metadata entries. The review process yielded an average accuracy of 94.00% for the metadata and 97.79% for the QA pairs. This rigorous validation underscores the quality and thorough curation of the ProbMed dataset. In addition, we conducted a study using the expert-verified data on GPT-4o to verify the robustness of our probing evaluation framework to dataset errors, detailed in Appendix C.

As of statistics reported in Table 1, our data curation process produced a total of 57,132 question-answer pairs (averaging 9 pairs per image), spanning a comprehensive set of diagnostic dimensions. These high-quality, balanced pairs provide a robust foundation for evaluating model performance.

<table>
  <tr>
    <th>Organ, Modality</th>
    <th>Image</th>
    <th>Question</th>
    <th>Question with Answer "yes"</th>
    <th>Unique Condition</th>
    <th>Unique Positional Description</th>
  </tr>
  <tr>
    <td>Abdomen MRI</td>
    <td>84</td>
    <td>257</td>
    <td>375</td>
    <td>107</td>
    <td>75</td>
  </tr>
  <tr>
    <td>Brain MRI</td>
    <td>566</td>
    <td>5,046</td>
    <td>2,298</td>
    <td>697</td>
    <td>446</td>
  </tr>
  <tr>
    <td>Chest MRI</td>
    <td>40</td>
    <td>382</td>
    <td>189</td>
    <td>52</td>
    <td>38</td>
  </tr>
  <tr>
    <td>Spine MRI</td>
    <td>324</td>
    <td>3,346</td>
    <td>1,664</td>
    <td>461</td>
    <td>336</td>
  </tr>
  <tr>
    <td>Abdomen CT scan</td>
    <td>270</td>
    <td>6,555</td>
    <td>3,098</td>
    <td>909</td>
    <td>552</td>
  </tr>
  <tr>
    <td>Brain CT scan</td>
    <td>270</td>
    <td>2,417</td>
    <td>1,200</td>
    <td>335</td>
    <td>209</td>
  </tr>
  <tr>
    <td>Chest CT scan</td>
    <td>548</td>
    <td>5,161</td>
    <td>2,572</td>
    <td>727</td>
    <td>353</td>
  </tr>
  <tr>
    <td>Spine CT scan</td>
    <td>87</td>
    <td>941</td>
    <td>470</td>
    <td>148</td>
    <td>98</td>
  </tr>
  <tr>
    <td>Abdomen X-ray</td>
    <td>232</td>
    <td>2,046</td>
    <td>1,018</td>
    <td>277</td>
    <td>160</td>
  </tr>
  <tr>
    <td>Brain X-ray</td>
    <td>3,178</td>
    <td>27,530</td>
    <td>13,278</td>
    <td>1,418</td>
    <td>694</td>
  </tr>
  <tr>
    <td>Chest X-ray</td>
    <td>202</td>
    <td>2,052</td>
    <td>1,020</td>
    <td>300</td>
    <td>172</td>
  </tr>
  <tr>
    <th>Total</th>
    <th>6,403</th>
    <th>57,132</th>
    <th>28,003</th>
    <th>/</th>
    <th>/</th>
  </tr>
</table>

Table 1: Dataset Statistics of ProbMed. There are 6.3k images and 57k VQA pairs in total. The dataset is balanced within each question type and image type.

4 Experiments and Analysis

We conducted a systematic evaluation and comprehensive analysis using ProbMed on twelve state-of-the-art LMs to identify their strengths and weaknesses in imaging diagnostics. Apart from proprietary GPT-4o (OpenAI, 2024), GPT-4V (OpenAI, 2023) and Gemini Pro (Reid et al., 2024), we selected nine open-source models spanning across general models including LLaVA-v1 (Liu et al., 2024), LLaVA-v1.6 (Liu et al., 2023a), MiniGPT-v2 (Chen et al., 2023) and specialized models including LLaVA-Med-v1, LLaVA-Med-v1.5 (Li et al., 2024), Med-Flamingo (Moor et al., 2023), BiomedGPT (Zhang et al., 2024), RadFM (Wu et al., 2023b) and CheXagent (Chen et al., 2024). These models were chosen based on their computational cost, efficiency, and inference speed, making them practical for integration into medical practice.

4.1 RQ1: Reliability of Current Med-VQA Evaluation

Adversarial Evaluation in VQA-RAD. To assess whether current Med-VQA evaluations capture model vulnerabilities, we first introduce adversarial
Table 2: Model accuracy on the VQA-RAD test subset and ProbMed with adversarial pairs. Accuracy is reported in two ways: (1) averaged across individual questions in a pair and (2) requiring both the ground truth and adversarial questions for the same image to be answered correctly. The drop in accuracy across models demonstrates their vulnerability to adversarial questions, with percentage decreases shown in parentheses.

<table>
  <tr>
    <th rowspan="2">Models</th>
    <th colspan="2">VQA-RAD</th>
    <th colspan="2">ProbMed</th>
  </tr>
  <tr>
    <th>Averaged Accuracy (%)</th>
    <th>Accuracy (%) with Adversarial Pairs</th>
    <th>Averaged Accuracy (%)</th>
    <th>Accuracy (%) with Adversarial Pairs</th>
  </tr>
  <tr>
    <td>LLaVA-v1</td>
    <td>62.28</td>
    <td>25.42 (-36.84)</td>
    <td>55.82</td>
    <td>19.30 (-36.51)</td>
  </tr>
  <tr>
    <td>LLaVA-v1.6</td>
    <td>44.06</td>
    <td>8.47 (-35.59)</td>
    <td>56.02</td>
    <td>24.96 (-31.06)</td>
  </tr>
  <tr>
    <td>MiniGPT-v2</td>
    <td>66.10</td>
    <td>46.61 (-19.49)</td>
    <td>59.82</td>
    <td>27.67 (-32.14)</td>
  </tr>
  <tr>
    <td>LLaVA-Med-v1</td>
    <td>43.22</td>
    <td>3.38 (-39.83)</td>
    <td>52.26</td>
    <td>17.90 (-34.35)</td>
  </tr>
  <tr>
    <td>LLaVA-Med-v1.5</td>
    <td>48.30</td>
    <td>15.25 (-33.05)</td>
    <td>68.41</td>
    <td>40.19 (-28.22)</td>
  </tr>
  <tr>
    <td>CheXAgent</td>
    <td>55.50</td>
    <td>21.18 (-34.32)</td>
    <td>58.70</td>
    <td>30.61 (-28.08)</td>
  </tr>
  <tr>
    <td>BioMedGPT</td>
    <td>56.35</td>
    <td>17.79 (-38.55)</td>
    <td>60.14</td>
    <td>33.34 (-26.79)</td>
  </tr>
  <tr>
    <td>Med-Flamingo</td>
    <td>61.01</td>
    <td>25.42 (-35.59)</td>
    <td>64.13</td>
    <td>35.66 (-28.47)</td>
  </tr>
  <tr>
    <td>RadFM</td>
    <td>67.79</td>
    <td>38.98 (-28.81)</td>
    <td>67.70</td>
    <td>41.00 (-26.70)</td>
  </tr>
  <tr>
    <td>Gemini Pro</td>
    <td>63.13</td>
    <td>44.91 (-18.22)</td>
    <td>75.08</td>
    <td>55.08 (-20.00)</td>
  </tr>
  <tr>
    <td>GPT-4V</td>
    <td>58.47</td>
    <td>33.89 (-24.57)</td>
    <td>75.70</td>
    <td>55.28 (-20.42)</td>
  </tr>
  <tr>
    <td>GPT-4o</td>
    <td>69.91</td>
    <td>55.08 (-14.83)</td>
    <td>76.31</td>
    <td>55.60 (-20.71)</td>
  </tr>
</table>

pairs on the VQA-RAD test set (Lau et al., 2018). Because VQA-RAD provides only finalized QA pairs without detailed metadata, adversarial pairs were manually constructed by medical experts for 118 “yes” instances (yielding 236 QA pairs total). As shown in Table 2 (left), despite being based on limited information and scale, these adversarial questions lead to drastic accuracy drops. For example, models such as GPT-4o show a reduction from 69.91% to 55.08% (a 14.83% decrease), highlighting the need for robust evaluation protocols.

Adversarial Evaluation in ProbMed. The ProbMed dataset systematically incorporates adversarial pairs on scale for all 57k QA pairs. Here, each diagnostic entity is paired with a ground-truth question and a corresponding adversarial question. Table 2 (right) demonstrates a similar significant impact: even the best-performing models experience a minimum 20.00% drop in accuracy (with an average decrease of 27.78% across models) when evaluated under this rigorous scheme. This result emphasizes that high accuracy on standard benchmarks can be misleading without essential adversarial evaluation for uncovering model weaknesses.

4.2 How Reliable Are LMMs in Medical Diagnosis?

After “correcting” inflated model accuracy by introducing adversarial pairs, we continue to address the second research question. We conducted diagnostic probing ranging from general to specialized diagnostic questions using the ProbMed dataset.

Performance across Diagnostic Questions Table 3 shows the categorical accuracy of different models aggregated among all image types. While GPT-4o, GPT-4V, and Gemini Pro outperform other models and excel in general tasks such as recognizing image modality and organs, their low performance in specialized tasks like determining the existence of abnormalities and answering fine-grained questions about condition/finding and position highlights a significant gap in their ability to aid in real-life diagnosis.

On more specialized diagnostic questions, even top-performing models like GPT-4o, GPT-4V, and Gemini Pro performed close to random guessing. Their accuracy in identifying conditions and positions was alarmingly low, underscoring their limitations in handling fine-grained medical inquiries. RadFM, LLaVA-Med-v1.5, and Med-Flamingo outperform other specialized models in general questions, yet still struggle with specialized questions. LLaVA-Med-v1.5 achieves much higher accuracy among open-sourced models in identifying conditions/finding and their positions, but still performs around 10% worse than the proprietary models.

Among the open-sourced general-purpose models, MiniGPT-v2 performs the best, surpassing domain-specific models such as LLaVA-Med-v1 in determining positions of condition/finding without domain-specific training. A more detailed breakdown of the performance of different models on different image types across each question type
Table 3: Categorical and overall accuracy (%) of different models aggregated among all image types in ProbMed (averaging over three runs). The overall accuracy is weighted by the number of questions in each type. The best result in each question category is in-bold, and the second best is underlined.

<table>
  <tr>
    <th rowspan="2">Models</th>
    <th colspan="2">General Question</th>
    <th colspan="3">Specialized Question</th>
    <th rowspan="2">Overall</th>
  </tr>
  <tr>
    <th>Modality</th>
    <th>Organ</th>
    <th>Abnormality</th>
    <th>Condition/Finding</th>
    <th>Position</th>
  </tr>
  <tr>
    <td>Random Choice</td>
    <td>25.00</td>
    <td>25.00</td>
    <td>50.00</td>
    <td>35.67</td>
    <td>36.48</td>
    <td>32.13</td>
  </tr>
  <tr>
    <td>LLaVA-v1</td>
    <td>25.30±1.18</td>
    <td>41.92±1.21</td>
    <td>50.00±2.01</td>
    <td>0.35±0.03</td>
    <td>0.14±0.06</td>
    <td>19.30±0.18</td>
  </tr>
  <tr>
    <td>LLaVA-v1.6</td>
    <td>6.95±0.24</td>
    <td><b>80.33±0.34</b></td>
    <td>45.89±0.24</td>
    <td>3.67±0.10</td>
    <td>1.37±0.17</td>
    <td>24.96±0.11</td>
  </tr>
  <tr>
    <td>MiniGPT-v2</td>
    <td>3.25±0.13</td>
    <td>76.95±0.59</td>
    <td>50.08±0.84</td>
    <td>15.23±0.76</td>
    <td>7.96±0.79</td>
    <td>27.67±0.25</td>
  </tr>
  <tr>
    <td>LLaVA-Med-v1</td>
    <td>5.72±0.21</td>
    <td>34.36±1.21</td>
    <td>38.30±2.83</td>
    <td>20.79±0.47</td>
    <td>5.22±1.10</td>
    <td>17.90±0.38</td>
  </tr>
  <tr>
    <td>LLaVA-Med-v1.5</td>
    <td>56.14±0.90</td>
    <td>67.96±0.08</td>
    <td>49.12±0.05</td>
    <td>21.91±0.06</td>
    <td>11.65±0.03</td>
    <td>40.19±0.13</td>
  </tr>
  <tr>
    <td>CheXagent</td>
    <td>37.25±0.50</td>
    <td>33.75±0.17</td>
    <td><b>73.31±0.01</b></td>
    <td>28.52±0.08</td>
    <td>7.48±0.06</td>
    <td>30.61±0.02</td>
  </tr>
  <tr>
    <td>BiomedGPT</td>
    <td>60.25±0.27</td>
    <td>46.81±0.62</td>
    <td>50.31±0.24</td>
    <td>14.13±0.90</td>
    <td>6.11±0.23</td>
    <td>33.34±0.17</td>
  </tr>
  <tr>
    <td>Med-Flamingo</td>
    <td>44.38±0.20</td>
    <td>62.02±0.54</td>
    <td>50.00±0.01</td>
    <td>26.17±0.13</td>
    <td>5.72±0.06</td>
    <td>35.66±0.14</td>
  </tr>
  <tr>
    <td>RadFM</td>
    <td>83.72±0.26</td>
    <td>41.04±0.33</td>
    <td>60.83±0.32</td>
    <td>23.05±0.14</td>
    <td>9.10±0.29</td>
    <td>41.00±0.19</td>
  </tr>
  <tr>
    <td>Gemini Pro</td>
    <td>96.47±0.88</td>
    <td>75.69±1.89</td>
    <td>60.29±1.99</td>
    <td>27.93±1.82</td>
    <td>18.44±0.77</td>
    <td>55.08±0.93</td>
  </tr>
  <tr>
    <td>GPT-4V</td>
    <td>92.51±1.10</td>
    <td>71.73±2.45</td>
    <td>53.30±1.90</td>
    <td><b>35.19±1.16</b></td>
    <td>22.40±1.89</td>
    <td><b>55.28±0.98</b></td>
  </tr>
  <tr>
    <td>GPT-4o</td>
    <td><b>97.03±0.34</b></td>
    <td>68.13±1.15</td>
    <td>61.79±2.28</td>
    <td>29.30±2.55</td>
    <td><b>24.06±1.80</b></td>
    <td><b>55.60±1.05</b></td>
  </tr>
</table>

is available in Appendix A. Distribution plots of ground-truth answers and model responses within each question category are available in Appendix F.

Error Analysis in Procedural Diagnosis An error analysis was conducted on two top-performing models (GPT-4V and Gemini Pro) across three specialized diagnostic question types: Abnormality, Condition/Finding, and Position. The procedural diagnosis is conducted by conditioning performance on later and more challenging questions on models’ success on simple prior ones. For example, model accuracy under the Abnormality category is calculated within those test samples where the model correctly identifies both imaging modality and organ. And the accuracy under the Condition/Finding category is further conditioned on those success samples from prior categories, including Abnormality.

As shown in Table 4, both models show vulnerabilities to hallucination errors, particularly in the later stages of the diagnostic procedure. For example, errors under the Abnormality arise either from incorrect responses or a tendency to over-reject challenging questions. In the Condition/Finding and Position categories, errors are largely due to the acceptance of hallucinated entities. Notably, Gemini Pro is more prone to accepting false conditions and positions, which significantly lowers its strict accuracy in these areas.

Table 4: Error Analysis of GPT-4V and Gemini Pro on ProbMed. The table shows the accuracy and types of errors for three specialized question types: Abnormality, Condition/Finding, and Position. Errors are categorized into wrong answers, rejection to answer, denying ground truth, and accepting hallucinations, providing a detailed breakdown of model performance and failure modes.

<table>
  <tr>
    <th rowspan="2">Question Type</th>
    <th rowspan="2">Accuracy and Error Type</th>
    <th colspan="2">Models</th>
  </tr>
  <tr>
    <th>GPT-4V</th>
    <th>Gemini Pro</th>
  </tr>
  <tr>
    <td rowspan="3">Abnormality</td>
    <td>Accuracy</td>
    <td>66.06</td>
    <td>67.05</td>
  </tr>
  <tr>
    <td>E_wrong_answer</td>
    <td>67.47</td>
    <td>100.00</td>
  </tr>
  <tr>
    <td>E_reject_to_answer</td>
    <td>32.52</td>
    <td>0.00</td>
  </tr>
  <tr>
    <td rowspan="3">Condition/Finding</td>
    <td>Accuracy</td>
    <td>36.90</td>
    <td>39.97</td>
  </tr>
  <tr>
    <td>E_deny_ground-truth</td>
    <td>51.69</td>
    <td>39.04</td>
  </tr>
  <tr>
    <td>E_accept_halucination</td>
    <td>42.12</td>
    <td>59.69</td>
  </tr>
  <tr>
    <td>E_reject_to_answer</td>
    <td>6.18</td>
    <td>1.26</td>
  </tr>
  <tr>
    <td rowspan="3">Position</td>
    <td>Accuracy</td>
    <td>39.97</td>
    <td>26.40</td>
  </tr>
  <tr>
    <td>E_deny_ground-truth</td>
    <td>39.04</td>
    <td>23.31</td>
  </tr>
  <tr>
    <td>E_accept_halucination</td>
    <td>59.69</td>
    <td>76.68</td>
  </tr>
  <tr>
    <td>E_reject_to_answer</td>
    <td>1.26</td>
    <td>0.00</td>
  </tr>
</table>

4.3 Exploring Model Limitations and Potential Improvements

Impact of Chain-of-Thought Prompting and Visual Understanding To further investigate the underperformance of open-source models, we conducted an extensive ablation study on LLaVA-v1, LLaVA-v1.6, LLaVA-Med-v1, LLaVA-Med-v1.5, Med-Flamingo, and GPT-4o. In this study, we examined two additional experimental settings: (1) applying a Chain-of-Thought (CoT) approach where models first generate visual descriptions from the image on their own, then utilize the description to augment the question, (2) enhancing the models by providing external visual descriptions generated by GPT-4o in addition to the ques-
Figure 4: Accuracy comparison of LLaVA-v1, LLaVA-v1.6, LLaVA-Med-v1, LLaVA-Med-v1.5, Med-Flamingo, and GPT-4o under three different settings: vanilla (baseline performance), chain-of-thought (CoT) reasoning, and CoT with GPT-4o-generated visual descriptions. All models demonstrate significant performance improvement when visual descriptions from GPT-4o are included, indicating that poor visual understanding is a key factor limiting baseline performance. Chain-of-thought reasoning alone also leads to notable gains in accuracy, particularly in general-purpose models.

As shown in Figure 4, employing the chain-of-thought approach alone—without external visual descriptions—resulted in an average accuracy increase of 6.51%. In particular, LLaVA-Med-v1.5’s accuracy improved from 40.19% to 54.55%, closing the gap to within 1.05% of the vanilla GPT-4o model. Interestingly, GPT-4o’s performance decreased by 3.55% when the CoT mechanism was applied, potentially indicating that the model already internally employs its own chain-of-thought process.

Notably, all open-source models exhibited improved performance when augmented with visual descriptions generated by GPT-4o, suggesting that their baseline limitations stem primarily from poor visual comprehension. On average, these models showed an accuracy improvement of 9.44% across all question categories. This observation suggests that poor visual understanding is a major limitation of existing models, and augmenting them with external visual reasoning can lead to notable gains. Detailed performance changes of each model, organized by question category, can be found in Appendix D.

Transferability of Domain Expertise As shown in Table 14, CheXagent, a model trained exclusively on chest X-ray images, performs best in detecting abnormalities and identifying conditions/findings among all models when tested on chest X-ray images. We conducted a finer-grained analysis to explore whether the model’s expertise in identifying features of a particular organ can be transferred to other imaging modalities.

Figure 5: Accuracy comparison of CheXagent in identifying organs and conditions/findings across different modalities. The model demonstrates significantly higher accuracy in identifying organs on chest images compared to images of other organs for both MRI and CT scans. Additionally, CheXagent shows improved accuracy in identifying conditions/findings on chest images, indicating the transferability of its specialized knowledge from chest X-ray training to other imaging modalities.

As illustrated in Figure 5, CheXagent achieves significantly higher accuracy in identifying chest-related features compared to other organs, confirming our assumption that the model’s pre-training
on chest X-rays enhances its performance on recognizing chest images across different modalities. Interestingly, CheXAgent also demonstrated higher accuracy in identifying conditions and findings in CT scans and MRIs of the chest, achieving a 3% increase in accuracy for MRIs and a 4% increase for CT scans compared with other organs within the same unseen modality. This indicates that specialized knowledge gained on chest X-rays can be transferred to other imaging modalities of the same organ in a zero-shot manner, highlighting the potential for cross-modality expertise transfer in real-life medical imaging diagnostics.

5 Conclusion

Evaluating the reliability of LMMs in the medical domain requires robust methods, and ProbMed, our newly introduced framework, addresses this by incorporating probing evaluation and procedural diagnosis. Our study reveals significant limitations in models like GPT-4o and Gemini Pro, which perform worse than random guessing on specialized diagnostic questions, while CheXAgent’s results highlight the critical importance of domain-specific knowledge. Furthermore, our additional experiments, which introduced Chain-of-Thought reasoning with external visual descriptions generated by GPT-4o, suggested that poor visual understanding is a major limitation of existing models and augmenting them with external visual reasoning can lead to notable gains.

6 Limitations

Despite the contributions, limitations such as the imbalanced image distribution favoring Chest X-rays (see Table 1) and the absence of open-ended evaluations, such as report generation, remain. The broader impact of our work includes the potential for improved diagnostic accuracy and better patient care, but it also highlights the risks of deploying unreliable models in healthcare. We recommend rigorous testing, continuous performance monitoring, and the incorporation of domain-specific expertise to mitigate these risks. Ultimately, our work aims to contribute to the development of trustworthy AI systems in healthcare, advancing diagnostic outcomes and patient safety.

Acknowledgments

This research project has benefited from the Microsoft Accelerate Foundation Models Research (AFMR) grant program.

References

Josh Achiam, Steven Adler, Sandhini Agarwal, Lama Ahmad, Ilge Akkaya, Florencia Leoni Aleman, Diogo Almeida, Janko Altenschmidt, Sam Altman, Shyamal Anadkat, et al. 2023. Gpt-4 technical report. arXiv preprint arXiv:2303.08774.

Rohan Anil, Andrew M Dai, Orhan Firat, Melvin Johnson, Dmitry Lepikhin, Alexandre Passos, Sihamak Shakeri, Emanuel Taropa, Paige Bailey, Zhifeng Chen, et al. 2023. Palm 2 technical report. arXiv preprint arXiv:2305.10403.

Juan Manuel Zambrano Chaves, Shih-Cheng Huang, Yanbo Xu, Hanwen Xu, Naoto Usuyama, Sheng Zhang, Fei Wang, Yujia Xie, Mahmoud Khademí, Ziyi Yang, et al. 2024. Training small multimodal models to bridge biomedical competency gap: A case study in radiology imaging. CoRR.

Jun Chen, Deyao Zhu, Xiaoqian Shen, Xiang Li, Zechun Liu, Pengchuan Zhang, Raghu Ramamritham, Vikas Chandra, Yunyang Xiong, and Mohamed Elhoseiny. 2023. Minigpt-v2: large language model as a unified interface for vision-language multi-task learning. ArXiv, abs/2310.09478.

Qihui Chen and Yi Hong. 2024. Medblip: Bootstrapping language-image pre-training from 3d medical images and texts. In Proceedings of the Asian Conference on Computer Vision, pages 2404–2420.

Zhihong Chen, Maya Varma, Jean-Benoit Delbrouck, Magdalini Paschali, Louis Blankemeier, Dave Van Veen, Jeya Maria Jose Valanarasu, Alaa Youssef, Joseph Paul Cohen, Eduardo Pontes Reis, et al. 2024. CheXagent: Towards a foundation model for chest x-ray interpretation. arXiv preprint arXiv:2401.12208.

Hyung Won Chung, Le Hou, Shayne Longpre, Barret Zoph, Yi Tay, William Fedus, Yunxuan Li, Xuezhi Wang, Mostafa Dehghani, Siddhartha Brahma, et al. 2024. Scaling instruction-finetuned language models. Journal of Machine Learning Research, 25(70):1–53.

Kai He, Rui Mao, Qika Lin, Yucheng Ruan, Xiang Lan, Mengling Feng, and Erik Cambria. 2023. A survey of large language models for healthcare: from data, technology, and applications to accountability and ethics. Preprint, arXiv:2310.05694.

Xuehai He, Yichen Zhang, Luntian Mou, Eric Xing, and Pengtao Xie. 2020. Pathvqa: 30000+ questions for medical visual question answering. Preprint, arXiv:2003.10286.

Albert Q. Jiang, Alexandre Sablayrolles, Arthur Mensch, Chris Bamford, Devendra Singh Chaplot, Diego de las Casas, Florian Bressand, Gianna Lengyel, Guillaume Lample, Lucile Saulnier, Lélio Renard Lavaud, Marie-Anne Lachaux, Pierre Stock, Teven Le Scao, Thibaut Lavril, Thomas Wang, Timothée Lacroix,
and William El Sayed. 2023. Mistral 7b. Preprint, arXiv:2310.06825.

Jason Lau, Sagnik Gayen, Asma Ben Abacha, et al. 2018. A dataset of clinically generated visual questions and answers about radiology images. Scientific Data, 5:180251.

Chunyuan Li, Cliff Wong, Sheng Zhang, Naoto Usuyama, Haotian Liu, Jianwei Yang, Tristan Nau mann, Hoifung Poon, and Jianfeng Gao. 2024. Llava med: Training a large language-and-vision assistant for biomedicine in one day. Advances in Neural Information Processing Systems, 36.

Junnan Li, Dongxu Li, Silvio Savarese, and Steven Hoi. 2023. Bilp-2: Bootstrapping language-image pretraining with frozen image encoders and large language models. In International conference on machine learning, pages 19730–19742. PMLR.

Bo Liu, Li-Ming Zhan, Li Xu, Lin Ma, Yan Yang, and Xiao-Ming Wu. 2021. Slake: A semantically-labeled knowledge-enhanced dataset for medical visual question answering. Preprint, arXiv:2102.09542.

Haotian Liu, Chunyuan Li, Yuheng Li, and Yong Jae Lee. 2023a. Improved baselines with visual instruction tuning. 2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 26286–26296.

Haotian Liu, Chunyuan Li, Qingyang Wu, and Yong Jae Lee. 2024. Visual instruction tuning. Advances in neural information processing systems, 36.

Zhengliang Liu, Hanqi Jiang, Tianyang Zhong, Zihao Wu, Chong Ma, Yiwei Li, Xiaowei Yu, Yutong Zhang, Yi Pan, Peng Shu, Yanjun Lyu, Lu Zhang, Junjie Yao, Peixin Dong, Chao Cao, Zhenxiang Xiao, Jiaqi Wang, Huan Zhao, Shaochen Xu, Yaonai Wei, Jingyuan Chen, Haixing Dai, Peilong Wang, Hao He, Zewei Wang, Xinyu Wang, Xu Zhang, Lin Zhao, Yiheng Liu, Kai Zhang, Liheng Yan, Lichao Sun, Jun Liu, Ning Qiang, Bao Ge, Xiaoyan Cai, Shijie Zhao, Xintao Hu, Yixuan Yuan, Gang Li, Shu Zhang, Xin Zhang, Xi Jiang, Tuo Zhang, Dinggang Shen, Quanzheng Li, Wei Liu, Xiang Li, Daqiang Zhu, and Tianming Liu. 2023b. Holistic evaluation of gpt-4v for biomedical imaging. Preprint, arXiv:2312.05256.

Michael Moor, Qian Huang, Shirley Wu, Michihiro Yasunaga, Yash Dalmia, Jure Leskovec, Cyril Zakkia, Eduardo Pontes Reis, and Pranav Rajpurkar. 2023. Med-flamingo: a multimodal medical few-shot learner. In Machine Learning for Health (MLAH), pages 353–367. PMLR.

OpenAI. 2023. Gpt-4v(ision) technical work and authors. Technical report.

OpenAI. 2024. Gpt-4o system card. Preprint, arXiv:2410.21276.

Cheng Peng, Xi Yang, Aokun Chen, Kaleb E Smith, Nima PourNejatian, Anthony B Costa, Cheryl Martin, Mona G Flores, Ying Zhang, Tanja Magoc, et al. 2023. A study of generative large language model for medical research and healthcare. NPJ digital medicine, 6(1):210.

Machel Reid, Nikolay Savinov, Denis Teplyashin, Dmitry Lepikhin, Timothy P. Lillicrap, Jean-Baptiste Alayrac, Radu Soricut, Angeliki Lazaridou, Orhan Firat, Julian Schrittwieser, Ioannis Antonoglou, Rohan Anil, Sebastian Borgeaud, Andrew M. Dai, Katie Millican, Ethan Dyer, Mia Glaese, Thibault Sottiaux, Benjamin Lee, Fabio Viola, Malcolm Reynolds, Yuanzhong Xu, James Molloy, Jilin Chen, Michael Isard, Paul Barham, Tom Hennigan, Ross McIlroy, Melvin Johnson, Johan Schalkwyk, Eli Collins, Eliza Rutherford, Erica Moreira, Kareem W. Ayoub, Megha Goel, Clemens Meyer, Gregory Thornton, Zhen Yang, Henryk Michalewski, Zaheer Abbas, Nathan Schucher, Ankesh Anand, Richard Ives, James Keeling, Karel Lenc, Salem Haykal, Siamak Shakeri, Pranav Shyam, Aakanksha Chowdhery, Roman Ring, Stephen Spencer, Eren Sezener, Luke Vilnis, Oscar Chang, Nobuyuki Morioka, George Tucker, Ce Zheng, Oliver Woodman, Nithya Atlaturi, Tomáš Kocisky, Evgenii Eltyshev, Xi Chen, Timothy Chung, Vittorio Selo, Siddhartha Brahma, Petko Georgiev, Ambrose Slone, Zhenkai Zhu, James Lottes, Siyuan Qiao, Ben Caine, Sebastian Riedel, Alex Tomala, Martin Chadwick, J Christopher Love, Peter Choy, Sid Mittal, Neil Houlsby, Yunhao Tang, Matthew Lamm, Libin Bai, Qiao Zhang, Luheng He, Yong Cheng, Peter Humphreys, Yuji Li, Sergey Brin, Albin Cassirer, Ying-Qi Miao, Lukas Zilka, Taylor Tobin, Kelvin Xu, Lev Proleev, Daniel Sohn, Alberto Magni, Lisa Anne Hendricks, Isabel Gao, Santiago Ontan’on, Oskar Bunyan, Nathan Byrd, Abhanshu Sharma, Biao Zhang, Mario Pinto, Rishika Sinha, Harsh Mehta, Dawei Jia, Sergi Caelles, Albert Webson, Alex Morris, Becca Roelofs, Yifan Ding, Robin Strudel, Xuehan Xiong, Marvin Ritter, Mostafa Dehghani, Rahma Chaabouni, Abhijit Karmarkar, Guangda Lai, Fabian Mentzer, Bibo Xu, YaGuan Li, Yujing Zhang, Tom Le Paine, Alex Goldin, Behnam Neyshabur, Kate Bauml, Anselm Levskaya, Michael Laskin, Wenhao Jia, Jack W. Rae, Kefan Xiao, Antoine He, Skye Giordano, Lakshman Yagati, Jean-Baptiste Lespiau, Paul Natsev, Sanjay Ganapathy, Fangyu Liu, Danilo Martins, Nanxin Chen, Yunhan Xu, Megan Barnes, Rhys May, Arpi Vezer, Junhyuk Oh, Ken Franko, Sophie Bridgers, Ruizhe Zhao, Boxi Wu, Basil Mustafa, Sean Sechrist, Emilio Parisotto, Thanumalayan Sankaranarayana Pillai, Chris Larkin, Chenjie Gu, Christina Sorokin, Maxim Krikun, Alexey Guseynov, Jessica Landon, Romina Datta, Alexander Pritzel, Phoebe Thacker, Fan Yang, Kevin Hui, A.E. Hauth, Chih-Kuan Yeh, David Barker, Justin Mao-Jones, Sophia Austin, Hannah Sheahan, Parker Schuh, James Svensson, Rohan Jain, Vinay Venkatesh Ramasesh, Anton Briukhov, Da-Woon Chung, Tamara von Glehn, Christina Butterfield, Priya Jhakra, Matt Wiethoff, Justin Frye, Jordan Grimstad, Beer Changpinyo, Charline Le
Lan, Anna Bortsova, Yonghui Wu, Paul Voigtlaender, Tara N. Sainath, Charlotte Smith, Will Hawkins, Kris Cao, James Besley, Srivatsan Srinivasan, Mark Omernick, Colin Gaffney, Gabriela de Castro Surita, Ryan Burnell, Bogdan Damoc, Junwhan Ahn, Andrew Brock, Mantas Pajarskas, Anastasia Petrushkina, Seb Noury, Lorenzo Blanco, Kevin Swersky, Arun Ahuja, Thi Avrahami, Vedant Misra, Raoul de Liedekerke, Mariko Inuma, Alex Polozov, Sarah York, George van den Driessche, Paul Michel, Justin Chiu, Rory Blevins, Zach Gleicher, Adria Recasens, Alban Rustemi, Elena Gribovskaya, Aurko Roy, Wiktor Gworek, Sébastien M. R. Arnold, Lisa Lee, James Lee-Thorp, Marcello Maggioni, Enrique Piqueras, Kartikeya Badola, Sharad Vikram, Lucas Gonzalez, Anirudh Baddepudi, Evan Senter, Jacob Devlin, James Qin, Michael Azzam, Maja Trebacz, Martin Polacek, Kashyap Krishnakumar, Shuo yin Chang, Matthew Tung, Ivo Penchev, Rishabh Joshi, Kate Olszewska, Carrie Muir, Mateo Wirth, Ale Jakse Hartman, Joshua Newlan, Sheleem Kashem, Vijay Bolina, Elahe Dabir, Joost R. van Amersfoort, Zafarali Ahmed, James Cobon-Kerr, Aishwarya B Kamath, Arnar Mar Hrafnkelsson, Le Hou, Ian MacKinnon, Alexandre Frechette, Eric Nolan, Xianze Si, Emanuel Taropa, Dong Li, Phil Creone, Anmol Gulati, Sébastien Cevey, Jonas Adler, Ada Ma, David Silver, Simon Tokumine, Richard Powell, Stephan Lee, Michael B. Chang, Samer Hassan, Diana Minucu, Antoine Yang, Nir Levine, Jenny Brennan, Mingqiu Wang, Sarah Hodkinson, Jeffrey Zhao, Josh Lipshultz, Aedan Pope, Michael B. Chang, Cheng Li, Laurent El Shafey, Michela Paganini, Sholto Douglas, Bernd Boltmet, Fabio Pardo, Seth Odom, Michaela Rosca, Cicero Nogueira dos Santos, Kedar Soparkar, Arthur Guez, Tom Hudson, Steven Hansen, Chulayuth Asawaroengchai, Ravichandra Addanki, Tianhe Yu, Wojciech Stokowiec, Mina Khan, Justin Gilmer, Jaehoon Lee, Carrie Grimes Bostock, Keran Rong, Jonathan Caton, Pedram Pejman, Filip Pavetic, Geoff Brown, Vivek Sharma, Mario Luvić, Rajkumar Samuel, Josip Djolonga, Amol Mandhane, Lars Lowe Sjosund, Elena Buchatskaya, Elspeth White, Natalie Clay, Jiepu Jiang, Hyeontak Lim, Ross Hemslcy, Jane Labanowski, Nicola De Cao, David Steiner, Sayed Hadi Hashemi, Jack Austin, Anita Gergely, Tim Blyth, Joe Stanton, Kaushik Shivakumar, Aditya Siddhant, Anders Andreassen, Carlos L. Araya, Nikhil Sethi, Rakesh Shivanna, Steven Hand, Ankur Bapna, Ali Khodaei, Antoine Miech, Garrett Tanzer, Andy Swing, Shantanu Thakoor, Zhufeng Pan, Zachary Nado, Stephanie Winkler, Dian Yu, Mohammad Saleh, Lorenzo Maggiore, Iain Barr, Minh Giang, Thais Kagohara, Ivo Danihelka, Amit Marathe, Vladimir Feinberg, Mohamed Elhawaty, Nimesh Ghelani, Dan Horgan, Helen Miller, Lexi Walker, Richard Tanburn, Mukarram Tariq, Disha Shrivastava, Fei Xia, Chung-Cheng Chiu, Zoe Ashwood, Khuslen Baatarsukh, Sina Samangooei, Fred Alcober, Axel Stjerngren, Paul Komarek, Katherine Tsihlas, Anudhyan Boral, Ramona Comanescu, Jeremy Chen, Ruibo Liu, Dawn Bloxwich, Charlie Chen, Yanhua Sun, Fangxi aoyu Feng, Matthew Mauger, Xerxes Dotiwalla, Vincent Hellendoorn, Michael Sharman, Ivy Zheng, Krishna Haridasan, Gabriel Barth-Maron, Craig Swanson, Dominika Rogozińska, Alek Andreev, Paul Kishan Rubenstein, Ruoxin Sang, Dan Hurt, Gamaleldin Elsayed, Ren shen Wang, Dave Lacey, Anastasia Il’ic, Yao Zhao, Woohyun Han, Lora Aroyo, Chimezie Iwuanyanwu, Vitaly Nikolaev, Balaji Lakshminarayanan, Sadegh Jazayeri, Raphael Lopez Kaufman, Mani Varadarajan, Chetan Tekur, Doug Fritz, Misha Khalmian, David Reitter, Kingshuk Dasgupta, Shourya Sarcar, T. Ornduff, Javier Snaider, Fantine Huot, Johnson Jia, Rupert Kemp, Nejc Trdin, Anitha Vijayakumar, Lucy Kim, Christof Angermueller, Li Lao, Tianqi Liu, Haibin Zhang, David Engel, Somer Greene, Anais White, Jessica Austin, Lilly Taylor, Shereen Ashraf, Dangyi Liu, Maria Georgaki, Irene Cai, Yana Kulizhskaya, Sonam Goenka, Brennan Saeta, Kiran Vodrahalli, Christian Frank, Dario de Cesare, Brona Robenek, Harry Richardson, Mahmoud Alnahlawi, Christopher Yew, Priya Ponnapalli, Marco Tagliasacchi, Alex Korchenny, Yelin Kim, Dinghua Li, Bill Rosgen, Kyle Levin, Jeremy Wiesner, Praseem Banzal, Praveen Srinivasan, Hongkun Yu, cCauglar Unlu, David Reid, Zora Tung, Daniel F. Finchelstein, Ravin Kumar, Andre Elisseeff, Jin Huang, Ming Zhang, Rui Zhu, Ricardo Aguilar, Mai Gim’enez, Jiawei Xia, Olivier Dousse, Willi Gierke, Soheil Hassas Yeganeh, Damion Yates, Komal Jalan, Lu Li, Eri Latorre-Chimoto, Duc Dung Nguyen, Ken Durden, Praveen Kallakuri, Yaxin Liu, Matthew Johnson, Tomy Tsai, Alice Talbert, Jasmine Liu, Alexander Neitz, Chen Elkind, Marco Selvi, Mimi Jasarevic, Livio Baldini Soares, Albert Cui, Pingdong Wang, Alek Wenjiao Wang, Xinyu Ye, Krystal Kallarackal, Lucia Loher, Hoi Lam, Josef Broder, Daniel Niels Holtmann-Rice, Nina Martin, Bramandia Ramadhana, Daniel Toyama, Mrinal Shukla, Sujoy Basu, Abhi Mohan, Nicholas Fernando, Noah Fiedel, Kim Paterson, Hui Li, Ankush Garg, Jane Park, Donghyun Choi, Diane Wu, Sanakalp Singh, Zhishuai Zhang, Amir Globerson, Lily Yu, John Carpenter, Felix de Chaumont Quity, Carey Radebaugh, Chu-Cheng Lin, Alex Tudor, Prakash Shroff, Drew Garmon, Dayou Du, Neera Vats, Han Lu, Shariq Iqbal, Alexey Yakubovich, Nileshe Tripuraneni, James Manyika, Ha roon Qureshi, Nan Hua, Christel Ngani, Maria Abi Raad, Hannah Forbes, Anna Bulanova, Jeff Stanway, Mukund Sundararajan, Victor Ungureanu, Colton Bishop, Yunjie Li, Balaji Venkatraman, Bo Li, Chloe Thornton, Salvatore Scellato, Nishesh Gupta, Yicheng Wang, Ian Tenney, Xiuhui Wu, Ashish Shenoy, Gabriel Carvalhal, Diana Gage Wright, Ben Bariach, Zhuyun Xiao, Peter Hawkins, Sid Dalmia, Clément Farabet, Pedro Valenzuela, Quan Yuan, Christopher A. Welty, Ananth Agarwal, Mianna Chen, Wooyeol Kim, Brice Hulse, Nandita Dukkipati, Adam Paszke, Andrew Bolt, Elnaz Davoodi, Kiam Choo, Jennifer Beattie, Jennifer Prendki, Harsha Vashisht, Rebeca Santamaría-Fernandez, Luis C. Cobo, Jarek Wilkiewicz, David Madras, Ali Elqursh, Grant Uy, Kevin Ramirez, Matt Harvey, Tyler Liechty, Heiga Zen, Jeff Seibert, Clara Huiyi Hu, A. Ya. Khorlin, Maigo Le, Asaf Aharoni, Megan Li, Lily Wang, Sandeep Ku-
mar. Alejandro Lince, Norman Casagrande, Jay Hoover, Dalia El Badawy, David Soergel, Denis Vnukov, Matt Miecnikowski, Jiří ima, Anna Koop, Praveen Kumar, Thibault Sellam, Daniel Vlasic, Samira Daruki, Nir Shabat, John Zhang, Guolong Su, Kalpesh Krishna, Jiageng Zhang, Jeremiah Liu, Yi Sun, Evan Palmer, Alireza Ghaffarkhah, Xi Xiong, Victor Cotruta, Michael Fink, Lucas Dixon, Ashwin Sreevatsa, Adrian Goedecemeyer, Alek Dimitriev, Mohsen Jafari, Remi Crocker, Nicholas Fitzgerald, Aviral Kumar, Sanjay Ghemawat, Ivan Philips, Frederick Liu, Yannie Liang, Rachel Sterneck, Alena Repina, Marcus Wu, Laura Knight, Marin Georgiev, Hyo Lee, Harry Askham, Abhishek Chakladar, Annie Louis, Carl Crous, Hardie Cate, Dessie Petrova, Michael Quinn, Denese Owusu-Afriyie, Achiptya Singhal, Nan Wei, Solomon Kim, Damien Vincent, Milad Nasr, Ilia Shumailov, Christopher A. Choquette-Choo, Reiko Tojo, Shawn Lu, Diego de Las Casas, Yuchung Cheng, Tolga Bolukbasi, Kather ine Lee, Saaber Fatehi, Rajagopal Ananthanarayanan, Miteyan Patel, Charbel El Kaed, Jing Li, Jakub Sygnowski, Shreyas Rammohan Belle, Zhe Chen, Jaclyn Konzelmann, Siim Pöder, Roopal Garg, Vinod Koverkathu, Adam Brown, Chris Dyer, Rosanne Liu, Azade Nova, Jun Xu, Junwen Bai, Slav Petrov, Demis Hassabis, Koray Kavukcuoglu, Jeffrey Dean, Oriol Vinyals, and Alexandra Chronopoulou. 2024. Gemini 1.5: Unlocking multimodal understanding across millions of tokens of context. ArXiv, abs/2403.05530.

Sanjay Subramanian, Lucy Lu Wang, Sachin Mehta, Ben Bogin, Madeleine van Zuylem, Sravanthi Parasa, Sameer Singh, Matt Gardner, and Hannaneh Hajishirzi. 2020. Medicat: A dataset of medical images, captions, and textual references. ArXiv, abs/2010.06000.

Hugo Touvron, Thibaut Lavril, Gautier Izacard, Xavier Martinet, Marie-Anne Lachaux, Timothée Lacroix, Baptiste Rozière, Naman Goyal, Eric Hambro, Faisal Azhar, Aurelien Rodriguez, Armand Joulin, Edouard Grave, and Guillaume Lample. 2023. Llama: Open and efficient foundation language models. Preprint, arXiv:2302.13971.

Xiaosong Wang, Yifan Peng, Le Lu, Zhiyong Lu, Mohammadadi Bagheri, and Ronald M. Summers. 2017. Chestx-ray8: Hospital-scale chest x-ray database and benchmarks on weakly-supervised classification and localization of common thorax diseases. 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pages 3462–3471.

Xinpeng Wang, Bolei Ma, Chengzhi Hu, Leon Weber-Genzel, Paul Röttger, Frauke Kreuter, Dirk Hovy, and Barbara Plank. 2024. "my answer is c": First-token probabilities do not match text answers in instruction-tuned language models. In Annual Meeting of the Association for Computational Linguistics.

Chaoyi Wu, Jiayu Lei, Qiaoyu Zheng, Weike Zhao, Weixiong Lin, Xiaoman Zhang, Xiao Zhou, Ziheng Zhao, Ya Zhang, Yanfeng Wang, and Weidi Xie. 2023a. Can gpt-4(ision) serve medical applications? case studies on gpt-4v for multimodal medical diagnosis. Preprint, arXiv:2310.09909.

Chaoyi Wu, Xiaoman Zhang, Ya Zhang, Yanfeng Wang, and Weidi Xie. 2023b. Towards generalist foundation model for radiology by leveraging web-scale 2d&3d medical data. Preprint, arXiv:2308.02463.

Zhiling Yan, Kai Zhang, Rong Zhou, Lifang He, Xiang Li, and Lichao Sun. 2023. Multimodal chatgpt for medical applications: an experimental study of gpt-4v. Preprint, arXiv:2310.19061.

Zhengyuan Yang, Linjie Li, Kevin Lin, Jianfeng Wang, Chung-Ching Lin, Zicheng Liu, and Lijuan Wang. 2023. The dawn of lms: Preliminary explorations with gpt-4v(ision). Preprint, arXiv:2309.17421.

Kai Zhang, Jun Yu, Eashan Adhikarla, Rong Zhou, Zhiling Yan, Yixin Liu, Zhengliang Liu, Lifang He, Brian Davison, Xiang Li, Hui Ren, Sunyang Fu, James Zou, Wei Liu, Jing Huang, Chen Chen, Yuyin Zhou, Tianming Liu, Xun Chen, Yong Chen, Quanzheng Li, Hongfang Liu, and Lichao Sun. 2024. Biomedgpt: A unified and generalist biomedical generative pre-trained transformer for vision, language, and multimodal tasks. Preprint, arXiv:2305.17100.

Xiaoman Zhang, Chaoyi Wu, Ziheng Zhao, Weixiong Lin, Ya Zhang, Yanfeng Wang, and Weidi Xie. 2023. Pmc-vqa: Visual instruction tuning for medical visual question answering. Preprint, arXiv:2305.10415.

Chujie Zheng, Hao Zhou, Fandong Meng, Jie Zhou, and Minli Huang. 2024. Large language models are not robust multiple choice selectors. Preprint, arXiv:2309.03882.

Hongjian Zhou, Fenglin Liu, Boyang Gu, Xinyu Zou, Jinfa Huang, Jingfe Wu, Yiru Li, Sam S. Chen, Peilin Zhou, Junling Liu, Yining Hua, Chengfeng Mao, Chenyu You, Xian Wu, Yefeng Zheng, Lei Clifton, Zheng Li, Jiebo Luo, and David A. Clifton. 2024. A survey of large language models in medicine: Progress, application, and challenge. Preprint, arXiv:2311.05112.

Yongshuo Zong, Tingyang Yu, Ruchika Chavhan, Bingchen Zhao, and Timothy Hospedales. 2024. Fool your (Vision and) language model with embarrassingly simple permutations. In Proceedings of the 41st International Conference on Machine Learning, volume 235 of Proceedings of Machine Learning Research, pages 62892–62913. PMLR.
A  Breakdown Results on Different Image Modality and Organ.

This section shows a detailed model performance breakdown by each (imaging modality, organ) pair under five categories.

A.1 Brain CT Scan

Table 5: Results of different models on Brain CT scan in ProbMed. The best-performing model in each question category is **in-bold**, and the second best is *underlined*.

<table>
  <tr>
    <th rowspan="2">Model</th>
    <th colspan="2">General Question</th>
    <th colspan="3">Specialized Question</th>
  </tr>
  <tr>
    <th>Modality</th>
    <th>Organ</th>
    <th>Anomaly/Condition</th>
    <th>Finding</th>
    <th>Position</th>
  </tr>
  <tr>
    <td>LlAVA+V1</td>
    <td>Acc. with adv. pairs</td>
    <td>75</td><td>46.67</td><td>/</td><td>75</td>
    <td>38.46</td>
    <td>*3**
      48.55
      46.06
      43.84
      41.90</td>
  </tr>
  <tr>
    <td>LlAVA+V1</td>
    <td>Avg. acc.</td>
    <td>82.39</td><td>72.22</td><td>11.53</td><td>91.32</td>
    <td>81.06</td>
    <td>*6* 90.13 87.21 84.89 81.86</td>
  </tr>
  <tr>
    <td>SalientGPT</td>
    <td>Acc. with adv. pairs</td>
    <td>/</td><td>65.01</td><td>/</td><td>66.57</td>
    <td>51.79</td>
    <td>56.32</td>
  </tr>
  <tr>
    <td>ManiGPT*v2</td>
    <td>Avg. acc.</td>
    <td>81.91</td><td>99</td><td>50.85</td><td>8.68</td>
    <td>78.47</td>
    <td>*1** 89.33 88.71 85.57 83.44</td>
  </tr>
  <tr>
    <td>GPT-4</td>
    <td>Acc. with adv. pairs</td>
    <td>86.18</td><td>72.16</td><td>/</td><td>86.99</td>
    <td>84.88</td>
    <td>*6* 90.13 90.47 90.18 90.18</td>
  </tr>
  <tr>
    <td>GPT-4V</td>
    <td>Avg. acc.</td>
    <td>88.72</td><td>78.98</td><td>14.90</td><td>82.51</td>
    <td>87.98</td>
    <td>*** 92.03 88.72 88.49 88.49</td>
  </tr>
  <tr>
    <td>Genium Pro</td>
    <td>Acc. with adv. pairs</td>
    <td>89.37</td><td>97.38</td><td>50.85</td><td>10.04</td>
    <td>83.62</td>
    <td>12* 77.92 83.27 83.27 83.54</td>
  </tr>
  <tr>
    <td>hum</td>
    <td>n1</td>
    <td>780</td><td>735</td><td>37</td><td>739</td>
    <td>869</td><td></td>
  </tr>
  <tr>
    <th rowspan="2">Model</th>
    <th colspan="2">General Question</th>
    <th colspan="3">Specialized Question</th>
  </tr>
  <tr>
    <th>Modality</th>
    <th>Organ</th>
    <th>Anomaly/Condition</th>
    <th>Finding</th>
    <th>Position</th>
  </tr>
  <tr>
    <td>LlAVA+V1</td>
    <td>Acc. with adv. pairs</td>
    <td>22</td><td>41</td><td>/</td><td>22</td>
    <td>36.36</td>
    <td>*1* 28.95 28.57 28.57 28.57</td>
  </tr>
  <tr>
    <td>LlAVA+V1</td>
    <td>Avg. acc.</td>
    <td>82.39</td><td>72.22</td><td>11.53</td><td>91.32</td>
    <td>81.06</td>
    <td>*6* 90.13 87.21 84.89 81.86</td>
  </tr>
  <tr>
    <td>SalientGPT</td>
    <td>Avg. acc.</td>
    <td>/</td><td>65.01</td><td>/</td><td>66.57</td>
    <td>51.79</td>
    <td>56.32</td>
  </tr>
  <tr>
    <td>ManiGPT*v2</td>
    <td>Avg. acc.</td>
    <td>81.91</td><td>99</td><td>50.85</td><td>8.68</td>
    <td>78.47</td>
    <td>*1** 89.33 88.71 85.57 83.44</td>
  </tr>
  <tr>
    <td>GPT-4</td>
    <td>Acc. with adv. pairs</td>
    <td>86.18</td><td>72.16</td><td>/</td><td>86.99</td>
    <td>84.88</td>
    <td>*6* 90.13 90.47 90.18 90.18</td>
  </tr>
  <tr>
    <td>GPT-4V</td>
    <td>Avg. acc.</td>
    <td>88.72</td><td>78.98</td><td>14.90</td><td>82.51</td>
    <td>87.98</td>
    <td>*** 92.03 88.72 88.49 88.49</td>
  </tr>
  <tr>
    <td>Genium Pro</td>
    <td>Acc. with adv. pairs</td>
    <td>89.37</td><td>97.38</td><td>50.85</td><td>10.04</td>
    <td>83.62</td>
    <td>12* 77.92 83.27 83.27 83.54</td>
  </tr>
  <tr>
    <td>hum</td>
    <td>n1</td>
    <td>780</td><td>735</td><td>37</td><td>739</td>
    <td>869</td><td></td>
  </tr>
</table>

A.2 Chest CT Scan

Table 6: Results of different models on Chest CT Scan in ProbMed. The best-performing model in each question category is **in-bold**, and the second best is *underlined*.

<table>
  <tr>
    <th rowspan="2">Model</th>
    <th colspan="2">General Question</th>
    <th colspan="3">Specialized Question</th>
  </tr>
  <tr>
    <th>Modality</th>
    <th>Organ</th>
    <th>Anomaly/Condition</th>
    <th>Finding</th>
    <th>Position</th>
  </tr>
  <tr>
    <td>LlAVA+V1</td>
    <td>Acc. with adv. pairs</td>
    <td>27.55</td><td>45.36</td><td>80.56</td><td>4.06</td>
    <td>0.22</td>
    <td>31.06</td>
  </tr>
  <tr>
    <td>LlAVA+V1</td>
    <td>Avg. acc.</td>
    <td>82.80</td><td>80.44</td><td>90.54</td><td>86.90</td>
    <td>88.68</td>
    <td>*14* **18 81.53 80.47 80.37 80.37</td>
  </tr>
  <tr>
    <td>LLAMA-V1.5</td>
    <td>Acc. with adv. pairs</td>
    <td>11.61</td><td>60.48</td><td>70.81</td><td>51.11</td>
    <td>X'</td>
    <td>14**
      62.80
      62.20
      62.06
      61.98</td>
  </tr>
  <tr>
    <td>LLAMA-V1.5</td>
    <td>Avg. acc.</td>
    <td>82.39</td><td>80.23</td><td>90.81</td><td>87.18</td>
    <td>88.76</td>
    <td>*1** 89.25 88.66 88.55 88.90</td>
  </tr>
  <tr>
    <td>ManiGPT*v2</td>
    <td>Acc. with adv. pairs</td>
    <td>/</td><td>85.18</td><td>/</td><td>85.66</td>
    <td>50.45</td>
    <td>57.11</td>
  </tr>
  <tr>
    <td>ManiGPT*v2</td>
    <td>Avg. acc.</td>
    <td>90.88</td><td>68.70</td><td>72.95</td><td>55.54</td>
    <td>1.52</td>
    <td>43.67</td>
  </tr>
  <tr>
    <td>GPT-4V</td>
    <td>Acc. with adv. pairs</td>
    <td>86.67</td><td>74.05</td><td>65.46</td><td>63.07</td>
    <td>66.61</td>
    <td>*2* 90.47 89.70 89.63 89.49</td>
  </tr>
  <tr>
    <td>GPT-4V</td>
    <td>Avg. acc.</td>
    <td>88.76</td><td>79.55</td><td>85.22</td><td>86.20</td>
    <td>87.74</td>
    <td>*** 90.52 90.37 90.37 90.37</td>
  </tr>
  <tr>
    <td>hum</td>
    <td>n3</td>
    <td>780</td><td>780</td><td>780</td><td>780</td>
    <td>780</td>
    <td></td>
  </tr>
</table>

A.3 Spine CT Scan

Table 7: Results of different models on Spine CT Scan in ProbMed. The best-performing model in each question category is **in-bold**, and the second best is *underlined*.

<table>
  <tr>
    <th colspan="8">General Question</th>
    <th colspan="8">Specialized Question</th>
  </tr>
  <tr>
    <th>Random Choice</th>
    <th>-</th>
    <th>--</th>
    <th>--</th>
    <th>s--</th>
    <th>/</th>
    <th>--</th>
    <th>---</th>
    <th>Random Choice</th>
    <th>-</th>
    <th>--</th>
    <th>--</th>
    <th>s--</th>
    <th>/</th>
    <th>--</th>
    <th>---</th>
  </tr>
  <tr>
    <td>LlAVA+V1</td>
    <td>Acc. with adv. pairs</td>
    <td>22.98</td><td>44.82</td><td>80</td><td>0</td><td>0</td><td>0</td>
    <td>LlAVA+V1</td>
    <td>Avg. acc.</td>
    <td>81.49</td><td>70.84</td><td>3.81</td><td>49.47</td><td>50.08</td><td>0</td>
  </tr>
  <tr>
    <td>LlAVA+V1.6</td>
    <td>Acc. with adv. pairs</td>
    <td>4.90</td><td>8.83</td><td>11</td><td>0</td><td>0</td><td>8.28</td>
    <td>LlAVA+V1.6</td>
    <td>Avg. acc.</td>
    <td>52.29</td><td>83.90</td><td>0</td><td>37.66</td><td>41.07</td><td>0</td>
  </tr>
  <tr>
    <td>ManiGPT*v2</td>
    <td>Acc. with adv. pairs</td>
    <td>1.14</td><td>41.37</td><td>0</td><td>12.64</td><td>51.32</td><td>0</td>
    <td>ManiGPT*v2</td>
    <td>Avg. acc.</td>
    <td>50.97</td><td>10.14</td><td>0</td><td>64.61</td><td>41.21</td><td>0</td>
  </tr>
  <tr>
    <td>LlAVA-Med-V1</td>
    <td>Acc. with adv. pairs</td>
    <td>2.29</td><td>11.49</td><td>50</td><td>10.3</td><td>0.4</td><td>0</td>
    <td>LlAVA-Med-V1</td>
    <td>Avg. acc.</td>
    <td>28.47</td><td>30.45</td><td>0</td><td>46.37</td><td>47.87</td><td>0</td>
  </tr>
  <tr>
    <td>LlAVA-Med-V1.5</td>
    <td>Acc. with adv. pairs</td>
    <td>32.18</td><td>67.81</td><td>50.0</td><td>9.19</td><td>0</td><td>14.16</td>
    <td>LlAVA-Med-V1.5</td>
    <td>Avg. acc.</td>
    <td>68.55</td><td>62.73</td><td>0</td><td>52.35</td><td>32.22</td><td>0</td>
  </tr>
  <tr>
    <td>BiomeGPT</td>
    <td>Acc. with adv. pairs</td>
    <td>8.27</td><td>8.40</td><td>0</td><td>0</td><td>0</td><td>2.66</td>
    <td>BiomeGPT</td>
    <td>Avg. acc.</td>
    <td>67.19</td><td>53.44</td><td>0</td><td>508.00</td><td>508.00</td><td>53.27</td>
  </tr>
  <tr>
    <td>Mem-Filangno</td>
    <td>Avg. acc.</td>
    <td>50.99</td><td>69.67</td><td>N.</td><td>0</td><td>50.82</td><td>6.92</td>
    <td>Mem-Filangno</td>
    <td>Avg. acc.</td>
    <td>25.40</td><td>12.18</td><td>0</td><td>76.85</td><td>20.78</td><td>0</td>
  </tr>
  <tr>
    <td>cheXceGate</td>
    <td>Acc. with adv. pairs</td>
    <td>87.15</td><td>74.24</td><td>4.25</td><td>50.92</td><td>8.497</td><td>3.62</td>
    <td>cheXceGate</td>
    <td>Avg. acc.</td>
    <td>77.91</td><td>86.12</td><td>59.00</td><td>7.71</td><td>8.94</td><td>4.52</td>
  </tr>
  <tr>
    <td>GPT-4o</td>
    <td>Acc. with adv. pairs</td>
    <td>898.31</td><td>*.15.19</td><td>100</td><td>89.28</td><td>88.65</td><td>80.87</td>
    <td>GPT-4V</td>
    <td>Acc. with adv. pairs</td>
    <td>83.16</td><td>68.72</td><td>82.36</td><td>72.23</td><td>73.94</td><td>80.27</td>
  </tr>
  <tr>
    <td>GPT-4V</td>
    <td>Avg. acc.</td>
    <td>89.34</td><td>75.92</td><td>13.80</td><td>51.76</td><td>87.38</td><td>31.05</td>
    <td>Gttnbi Pro</td>
    <td>Avg. acc.</td>
    <td>27.29</td><td>80.55</td><td>10.00</td><td>40.85</td><td>78.10</td><td></td>
  </tr>
  <tr>
    <td>Genium Pro</td>
    <td>Acc. with adv. pairs</td>
    <td>81.26</td><td>100</td><td>0</td><td>80.85</td><td>21.80</td><td>0</td>
    <td>Genium Pro</td>
    <td>Avg. acc.</td>
    <td>92.10</td><td>58.62</td><td>0</td><td>58.64</td><td>55.46</td><td>0</td>
  </tr>
  <tr>
    <td>hum</td>
    <td>n4</td>
    <td>56</td><td>55</td><td>2</td><td>60</td><td>56</td><td>56</td>
    <td>hum</td>
    <td>n14</td>
    <td>889</td><td>813</td><td>18.</td><td>885</td><td>93</td><td>93</td>
  </tr>
</table>

A.4 Abdominal CT Scan

Table 8: Results of different models on Abdominal CT Scan in ProbMed. The best-performing model in each question category is **in-bold**, and the second best is *underlined*.

<table>
  <tr>
    <th colspan="8">General Question</th>
    <th colspan="8">Specialized Question</th>
  </tr>
  <tr>
    <th>Random Choice</th>
    <th>-</th>
    <th>--</th>
    <th>-p</th>
    <th>/</th>
    <th>-</th>
    <th>--</th>
    <th>---</th>
    <th>Random Choice</th>
    <th>-</th>
    <th>--</th>
    <th>-p</th>
    <th>/</th>
    <th>--</th>
    <th>---</th>
  </tr>
  <tr>
    <td>LlAVA+V1</td>
    <td>Acc. with adv. pairs</td>
    <td>24.47</td><td>35.59</td><td>8.57</td><td>51.11</td><td>0.54</td><td>0</td>
    <td>LlAVA+V1</td>
    <td>Avg. acc.</td>
    <td>54.19</td><td>50</td><td></td><td>0.8</td><td>0.8</td><td>0.8</td>
  </tr>
  <tr>
    <td>LlAVA+V1.6</td>
    <td>Avg. acc.</td>
    <td>76.12</td><td>80.54</td><td>0.33</td><td>57.83</td><td>70.96</td><td>50.00</td>
    <td>LlAVA+V1.6</td>
    <td>Avg. acc.</td>
    <td>92.82</td><td>46.114</td><td>0.51</td><td>39.76</td><td>38.06</td><td>0.77</td>
  </tr>
  <tr>
    <td>ManiGPT*v2</td>
    <td>Avg. acc.</td>
    <td>50.83</td><td>81.67</td><td>18.83</td><td>45.17</td><td>0.95</td><td>0.88</td>
    <td>ManiGPT*v2</td>
    <td>Avg. acc.</td>
    <td>0</td><td>48.57</td><td>21.01</td><td>31.21</td><td>27.72</td><td>0</td>
  </tr>
  <tr>
    <td>LlAVA-Med-V1</td>
    <td>Acc. with adv. pairs</td>
    <td>2.55</td><td>21.64</td><td>10</td><td>25.42</td><td>14.18</td><td>47.7</td>
    <td>LlAVA-Med-V1</td>
    <td>Avg. acc.</td>
    <td>2.05</td><td>45</td><td>30</td><td>15.04</td><td>5.82</td><td>28.23</td>
  </tr>
  <tr>
    <td>LlAVA-Med-V1.5</td>
    <td>Avg. acc.</td>
    <td>56.07</td><td>76.24</td><td>20.77</td><td>54.81</td><td>0.58</td><td>0.21</td>
    <td>LlAVA-Med-V1.5</td>
    <td>Avg. acc.</td>
    <td>57.51</td><td>10.95</td><td>17.33</td><td>45.49</td><td>42.61</td><td>16.51</td>
  </tr>
  <tr>
    <td>SacintegarP</td>
    <td>Acc. with adv. pairs</td>
    <td>57.99</td><td>56.49</td><td>20.77</td><td>52.64</td><td>0.58</td><td>0.21</td>
    <td>SacintegarP</td>
    <td>Avg. acc.</td>
    <td>57.60</td><td>6.02</td><td>13.71</td><td>44.99</td><td>42.61</td><td>16.48</td>
  </tr>
  <tr>
    <td>SacintegarP</td>
    <td>Avg. acc.</td>
    <td>65.09</td><td>69.61</td><td>50.23</td><td>34.19</td><td>1.72</td><td>4.34</td>
    <td>SacintegarP</td>
    <td>Avg. acc.</td>
    <td>68.05</td><td>67.81</td><td>41.16</td><td>32.85</td><td>7.13</td><td>2.61</td>
  </tr>
  <tr>
    <td>GPT-4A</td>
    <td>Acc. with adv. pairs</td>
    <td>**49.95**</td><td>*65.62**</td><td>27.00</td><td>**80.55**</td><td>0</td><td>**58.41**</td>
    <td>GPT-4A</td>
    <td>Avg. acc.</td>
    <td>99.18</td><td>67.07</td><td>0</td><td>97.12</td><td>61.72</td><td>40.07</td>
  </tr>
  <tr>
    <td>GPT-4V/AV</td>
    <td>Avg. acc.</td>
    <td>98.77</td><td>64.06</td><td>**63.82**</td><td>62.96</td><td>3.52</td><td>6.49</td>
    <td>GPT-4V/AV</td>
    <td>Avg. acc.</td>
    <td>99.18</td><td>67.07</td><td>25.17</td><td>97.16</td><td>61.71</td><td>40.06</td>
  </tr>
  <tr>
    <td>GeminPro AV</td>
    <td>Acc. with adv. pairs</td>
    <td>95.62</td><td>84.38</td><td>13.42</td><td>84.38</td><td>13.42</td><td>40.39</td>
    <td>GeminPro AV</td>
    <td>Acc. with adv. pairs</td>
    <td>98.31</td><td>63.24</td><td>62.96</td><td>59.00</td><td>3.52</td><td>6.18</td>
  </tr>
  <tr>
    <td>humav</td>
    <td>n18</td>
    <td>730</td><td>730</td><td>730</td><td>730</td><td>730</td><td>730</td>
    <td>humav</td>
    <td>n30</td>
    <td>730</td><td>730</td><td>69.71</td><td>70.17</td><td>70.17</td><td>71.67</td>
  </tr>
</table>
A.5 Brain MRI

Table 9: Results of different models on Brain MRI in ProbMed. The best-performing model in each question category is **in-bold**, and the second best is *underlined*.

<table>
  <tr>
    <th rowspan="2">Random Choice</th>
    <th colspan="4">General Question</th>
    <th colspan="4">Specialized Question</th>
  </tr>
  <tr>
    <th>Modality</th>
    <th>Organ</th>
    <th>Anomaly/Prediction</th>
    <th>Position</th>
    <th>Modality</th>
    <th>Organ</th>
    <th>Anomaly/Prediction</th>
    <th>Position</th>
  </tr>
  <tr>
    <td></td>
    <td>Ac.</td><td>37</td><td>57</td><td>57</td><td>8</td><td>37</td><td>57</td><td>57</td><td>0</td>
  </tr>
  <tr>
    <td>LLaVA-v1</td>
    <td>Avg.</td><td>86.92</td><td>82.43</td><td>82.65</td><td>/</td><td>87.73</td><td>84.65</td><td>84.78</td><td>/</td>
  </tr>
  <tr>
    <td>LLaVA-v1.6</td>
    <td>Avg.</td><td>87.00</td><td>82.61</td><td>82.73</td><td>82.87</td><td>87.79</td><td>84.67</td><td>84.79</td><td>84.78</td>
  </tr>
  <tr>
    <td>MiniGPT-v2</td>
    <td>Avg.</td><td>51.51</td><td>50.14</td><td>50.77</td><td>50.77</td><td>51.51</td><td>50.14</td><td>50.77</td><td>50.77</td>
  </tr>
  <tr>
    <td>GPT-to</td>
    <td>Avg.</td><td>98.41</td><td>94.28</td><td>95.20</td><td>95.20</td><td>98.41</td><td>94.28</td><td>95.20</td><td>95.20</td>
  </tr>
  <tr>
    <td>GPT-4V</td>
    <td>Avg.</td><td>86.88</td><td>82.37</td><td>82.61</td><td>82.61</td><td>86.88</td><td>82.37</td><td>82.61</td><td>82.61</td>
  </tr>
  <tr>
    <td>GPT-IV</td>
    <td>Avg.</td><td>86.73</td><td>82.58</td><td>82.61</td><td>82.61</td><td>86.73</td><td>82.58</td><td>82.61</td><td>82.61</td>
  </tr>
  <tr>
    <td>Geniun Pro</td>
    <td>Avg.</td><td>98.12</td><td>94.09</td><td>95.17</td><td>95.17</td><td>98.12</td><td>94.09</td><td>95.17</td><td>95.17</td>
  </tr>
  <tr>
    <td>num</td><td></td><td>40</td><td>40</td><td>40</td><td>40</td><td>40</td><td>40</td><td>40</td><td>40</td>
  </tr>
</table>

A.6 Chest MRI

Table 10: Results of different models on Chest MRI in ProbMed. The best-performing model in each question category is **in-bold**, and the second best is *underlined*.

<table>
  <tr>
    <th rowspan="2">Random Choice</th>
    <th colspan="4">General Question</th>
    <th colspan="4">Specialized Question</th>
  </tr>
  <tr>
    <th>Modality</th>
    <th>Organ</th>
    <th>Anomaly/Prediction</th>
    <th>Position</th>
    <th>Modality</th>
    <th>Organ</th>
    <th>Anomaly/Prediction</th>
    <th>Position</th>
  </tr>
  <tr>
    <td></td>
    <td>Ac.</td><td>432</td><td>416</td><td>416</td><td>0</td><td>432</td><td>416</td><td>416</td><td>0</td>
  </tr>
  <tr>
    <td>LLaVA-v1</td>
    <td>Avg.</td><td>82.75</td><td>82.25</td><td>82.25</td><td>80.50</td><td>82.75</td><td>82.25</td><td>82.25</td><td>80.50</td>
  </tr>
  <tr>
    <td>LLaVA-v1.6</td>
    <td>Avg.</td><td>82.90</td><td>82.40</td><td>82.40</td><td>80.75</td><td>82.90</td><td>82.40</td><td>82.40</td><td>80.75</td>
  </tr>
  <tr>
    <td>MiniGPT-v2</td>
    <td>Avg.</td><td>50.81</td><td>49.97</td><td>50.00</td><td>50.00</td><td>50.81</td><td>49.97</td><td>50.00</td><td>50.00</td>
  </tr>
  <tr>
    <td>GPT-to</td>
    <td>Avg.</td><td>97.84</td><td>94.19</td><td>95.20</td><td>95.20</td><td>97.84</td><td>94.19</td><td>95.20</td><td>95.20</td>
  </tr>
  <tr>
    <td>GPT-4V</td>
    <td>Avg.</td><td>86.88</td><td>82.37</td><td>82.61</td><td>82.61</td><td>86.88</td><td>82.37</td><td>82.61</td><td>82.61</td>
  </tr>
  <tr>
    <td>GPT-IV</td>
    <td>Avg.</td><td>86.73</td><td>82.58</td><td>82.61</td><td>82.61</td><td>86.73</td><td>82.58</td><td>82.61</td><td>82.61</td>
  </tr>
  <tr>
    <td>Geniun Pro</td>
    <td>Avg.</td><td>98.12</td><td>94.09</td><td>95.17</td><td>95.17</td><td>98.12</td><td>94.09</td><td>95.17</td><td>95.17</td>
  </tr>
  <tr>
    <td>num</td><td></td><td>40</td><td>40</td><td>40</td><td>40</td><td>40</td><td>40</td><td>40</td><td>40</td>
  </tr>
</table>

A.7 Spine MRI

Table 11: Results of different models on Spine MRI in ProbMed. The best-performing model in each question category is **in-bold**, and the second best is *underlined*.

<table>
  <tr>
    <th rowspan="2">Random Choice</th>
    <th colspan="4">General Question</th>
    <th colspan="4">Specialized Question</th>
  </tr>
  <tr>
    <th>Modality</th>
    <th>Organ</th>
    <th>Anomaly/Prediction</th>
    <th>Position</th>
    <th>Modality</th>
    <th>Organ</th>
    <th>Anomaly/Prediction</th>
    <th>Position</th>
  </tr>
  <tr>
    <td></td>
    <td>Ac.</td><td>432</td><td>416</td><td>416</td><td>0</td><td>432</td><td>416</td><td>416</td><td>0</td>
  </tr>
  <tr>
    <td>LLaVA-v1</td>
    <td>Avg.</td><td>82.75</td><td>82.25</td><td>82.25</td><td>80.50</td><td>82.75</td><td>82.25</td><td>82.25</td><td>80.50</td>
  </tr>
  <tr>
    <td>LLaVA-v1.6</td>
    <td>Avg.</td><td>82.90</td><td>82.40</td><td>82.40</td><td>80.75</td><td>82.90</td><td>82.40</td><td>82.40</td><td>80.75</td>
  </tr>
  <tr>
    <td>MiniGPT-v2</td>
    <td>Avg.</td><td>50.81</td><td>49.97</td><td>50.00</td><td>50.00</td><td>50.81</td><td>49.97</td><td>50.00</td><td>50.00</td>
  </tr>
  <tr>
    <td>GPT-to</td>
    <td>Avg.</td><td>97.84</td><td>94.19</td><td>95.20</td><td>95.20</td><td>97.84</td><td>94.19</td><td>95.20</td><td>95.20</td>
  </tr>
  <tr>
    <td>GPT-4V</td>
    <td>Avg.</td><td>86.88</td><td>82.37</td><td>82.61</td><td>82.61</td><td>86.88</td><td>82.37</td><td>82.61</td><td>82.61</td>
  </tr>
  <tr>
    <td>GPT-IV</td>
    <td>Avg.</td><td>86.73</td><td>82.58</td><td>82.61</td><td>82.61</td><td>86.73</td><td>82.58</td><td>82.61</td><td>82.61</td>
  </tr>
  <tr>
    <td>Geniun Pro</td>
    <td>Avg.</td><td>98.12</td><td>94.09</td><td>95.17</td><td>95.17</td><td>98.12</td><td>94.09</td><td>95.17</td><td>95.17</td>
  </tr>
  <tr>
    <td>num</td><td></td><td>40</td><td>40</td><td>40</td><td>40</td><td>40</td><td>40</td><td>40</td><td>40</td>
  </tr>
</table>

A.8 Abdominal MRI

Table 12: Results of different models on Abdominal MRI in ProbMed. The best-performing model in each question category is **in-bold**, and the second best is *underlined*.

<table>
  <tr>
    <th rowspan="2">Random Choice</th>
    <th colspan="4">General Question</th>
    <th colspan="4">Specialized Question</th>
  </tr>
  <tr>
    <th>Modality</th>
    <th>Organ</th>
    <th>Anomaly/Prediction</th>
    <th>Position</th>
    <th>Modality</th>
    <th>Organ</th>
    <th>Anomaly/Prediction</th>
    <th>Position</th>
  </tr>
  <tr>
    <td></td>
    <td>Ac.</td><td>432</td><td>416</td><td>416</td><td>0</td><td>432</td><td>416</td><td>416</td><td>0</td>
  </tr>
  <tr>
    <td>LLaVA-v1</td>
    <td>Avg.</td><td>82.75</td><td>82.25</td><td>82.25</td><td>80.50</td><td>82.75</td><td>82.25</td><td>82.25</td><td>80.50</td>
  </tr>
  <tr>
    <td>LLaVA-v1.6</td>
    <td>Avg.</td><td>82.90</td><td>82.40</td><td>82.40</td><td>80.75</td><td>82.90</td><td>82.40</td><td>82.40</td><td>80.75</td>
  </tr>
  <tr>
    <td>MiniGPT-v2</td>
    <td>Avg.</td><td>50.81</td><td>49.97</td><td>50.00</td><td>50.00</td><td>50.81</td><td>49.97</td><td>50.00</td><td>50.00</td>
  </tr>
  <tr>
    <td>GPT-to</td>
    <td>Avg.</td><td>97.84</td><td>94.19</td><td>95.20</td><td>95.20</td><td>97.84</td><td>94.19</td><td>95.20</td><td>95.20</td>
  </tr>
  <tr>
    <td>GPT-4V</td>
    <td>Avg.</td><td>86.88</td><td>82.37</td><td>82.61</td><td>82.61</td><td>86.88</td><td>82.37</td><td>82.61</td><td>82.61</td>
  </tr>
  <tr>
    <td>GPT-IV</td>
    <td>Avg.</td><td>86.73</td><td>82.58</td><td>82.61</td><td>82.61</td><td>86.73</td><td>82.58</td><td>82.61</td><td>82.61</td>
  </tr>
  <tr>
    <td>Geniun Pro</td>
    <td>Avg.</td><td>98.12</td><td>94.09</td><td>95.17</td><td>95.17</td><td>98.12</td><td>94.09</td><td>95.17</td><td>95.17</td>
  </tr>
  <tr>
    <td>num</td><td></td><td>40</td><td>40</td><td>40</td><td>40</td><td>40</td><td>40</td><td>40</td><td>40</td>
  </tr>
</table>
A.9 Brain X-ray

Table 13: Results of different models on Brain X-ray in ProbMed. The best-performing model in each question category is in-bold, and the second best is underlined.

<table>
  <tr>
    <th rowspan="2"> </th>
    <th colspan="7">General Question</th>
    <th colspan="7">Specialized Question</th>
  </tr>
  <tr>
    <th colspan="6">Modality Organ Abnormality Condition/Finding Position</th>
    <th colspan="6">Condition/Finding Position</th>
  </tr>
  <tr>
    <th>Random Choice</th>
    <th>acc.</th><th>pairs</th><th>5</th><th>25</th><th>50</th><th>100</th><th>in</th><th colspan="1"></th><th>acc.</th><th>pairs</th><th>25</th><th>50</th><th>100</th><th>5</th><th>25</th><th>50</th><th>100</th><th>in</th>
  </tr>
  <tr>
    <td>LLaVA-v1</td>
    <td>45.56</td><td>58.51</td><td>50</td><td>/</td><td>48.08</td><td>50.00</td><td></td><td>50.00</td><td colspan="2"></td><td></td><td></td><td></td><td></td><td></td><td>0.04</td>
  </tr>
  <tr>
    <td>LLaVA-v1.6</td>
    <td>72.78</td><td>51.89</td><td>/</td><td>48.10</td><td>50.00</td><td></td><td>50.00</td><td colspan="2"></td><td>58.28</td><td>54.44</td><td>/</td><td>47.32</td><td>43.48</td><td>40.22</td><td>50.00</td><td>49.42</td>
  </tr>
  <tr>
    <td>MiniGPT-v2</td>
    <td>59.55</td><td>51.08</td><td>50</td><td>48.15</td><td>48.44</td><td>40.17</td><td>/</td><td>48.44</td><td colspan="2"></td><td>58.28</td><td>54.44</td><td>/</td><td>47.32</td><td>43.48</td><td>40.22</td><td>50.00</td><td>49.42</td>
  </tr>
  <tr>
    <td>GPT-4o</td>
    <td>88.96</td><td>89.35</td><td>50</td><td>18.72</td><td>17.77</td><td>17.77</td><td>/</td><td>17.77</td><td colspan="2"></td><td>58.28</td><td>54.44</td><td>/</td><td>47.32</td><td>43.48</td><td>40.22</td><td>50.00</td><td>49.42</td>
  </tr>
  <tr>
    <td>GPT-IV</td>
    <td>88.36</td><td>89.77</td><td>50</td><td>18.72</td><td>17.77</td><td>17.77</td><td>/</td><td>17.77</td><td colspan="2"></td><td>58.28</td><td>54.44</td><td>/</td><td>47.32</td><td>43.48</td><td>40.22</td><td>50.00</td><td>49.42</td>
  </tr>
  <tr>
    <td>Gemin Pro</td>
    <td>88.31</td><td>74.05</td><td>50</td><td>/</td><td>61.83</td><td>63.85</td><td></td><td></td><td colspan="2"></td><td>58.28</td><td>54.44</td><td>/</td><td>47.32</td><td>43.48</td><td>40.22</td><td>50.00</td><td>49.42</td>
  </tr>
  <tr>
    <td>num</td>
    <td>79</td><td>79</td><td>6</td><td>79</td><td>79</td><td></td><td></td><td></td><td>291</td><td>291</td><td>24</td><td>291</td><td>291</td><td>24</td>
  </tr>
</table>

A.10 Chest X-ray

Table 14: Results of different models on Chest X-ray in ProbMed. The best-performing model in each question category is in-bold, and the second best is underlined.

<table>
  <tr>
    <th rowspan="2"> </th>
    <th colspan="7">General Question</th>
    <th colspan="7">Specialized Question</th>
  </tr>
  <tr>
    <th colspan="6">Modality Organ Abnormality Condition/Finding Position</th>
    <th colspan="6">Condition/Finding Position</th>
  </tr>
  <tr>
    <th>Random Choice</th>
    <th>acc.</th><th>pairs</th><th>5</th><th>25</th><th>50</th><th>100</th><th>in</th><th colspan="1"></th><th>acc.</th><th>pairs</th><th>25</th><th>50</th><th>100</th><th>5</th><th>25</th><th>50</th><th>100</th><th>in</th>
  </tr>
  <tr>
    <td>LLaVA-v1</td>
    <td>48.72</td><td>56.37</td><td>50</td><td>/</td><td>48.81</td><td>50.00</td><td></td><td>50.00</td><td colspan="2"></td><td></td><td></td><td></td><td></td><td></td><td>0.64</td>
  </tr>
  <tr>
    <td>LLaVA-v1.6</td>
    <td>73.51</td><td>48.97</td><td>50</td><td>48.00</td><td>42.64</td><td>41.04</td><td>/</td><td>40.05</td><td colspan="2"></td><td>58.83</td><td>54.37</td><td>/</td><td>44.62</td><td>41.48</td><td>40.17</td><td>50.00</td><td>49.71</td>
  </tr>
  <tr>
    <td>MiniGPT-v2</td>
    <td>54.57</td><td>53.95</td><td>50</td><td>48.18</td><td>46.67</td><td>40.17</td><td>/</td><td>48.18</td><td colspan="2"></td><td>58.83</td><td>54.37</td><td>/</td><td>44.62</td><td>41.48</td><td>40.17</td><td>50.00</td><td>49.71</td>
  </tr>
  <tr>
    <td>GPT-4o</td>
    <td>97.42</td><td>82.33</td><td>50</td><td>15.47</td><td>13.68</td><td>8.75</td><td>/</td><td>9.46</td><td colspan="2"></td><td>98.25</td><td>81.17</td><td>/</td><td>18.07</td><td>15.28</td><td>7.87</td><td>81.67</td><td>79.71</td>
  </tr>
  <tr>
    <td>GPT-IV</td>
    <td>97.42</td><td>82.33</td><td>50</td><td>15.47</td><td>13.68</td><td>8.75</td><td>/</td><td>9.46</td><td colspan="2"></td><td>98.25</td><td>81.17</td><td>/</td><td>18.07</td><td>15.28</td><td>7.87</td><td>81.67</td><td>79.71</td>
  </tr>
  <tr>
    <td>Gemin Pro</td>
    <td>93.05</td><td>74.05</td><td>50</td><td>/</td><td>61.83</td><td>63.85</td><td></td><td></td><td colspan="2"></td><td>98.25</td><td>81.17</td><td>/</td><td>18.07</td><td>15.28</td><td>7.87</td><td>81.67</td><td>79.71</td>
  </tr>
  <tr>
    <td>num</td>
    <td>3120</td><td>3120</td><td>1974</td><td>3120</td><td>3120</td><td></td><td></td><td></td><td>232</td><td>232</td><td>20</td><td>232</td><td>232</td><td>20</td>
  </tr>
</table>

A.11 Spine X-ray

Table 15: Results of different models on Spine X-ray in ProbMed. The best-performing model in each question category is in-bold, and the second best is underlined.

<table>
  <tr>
    <th rowspan="2"> </th>
    <th colspan="7">General Question</th>
    <th colspan="7">Specialized Question</th>
  </tr>
  <tr>
    <th colspan="6">Modality Organ Abnormality Condition/Finding Position</th>
    <th colspan="6">Organ Condition/Finding Position</th>
  </tr>
  <tr>
    <th>Random Choice</th>
    <th>acc.</th><th>pairs</th><th>25</th><th>50</th><th>100</th><th>in</th><th colspan="1"></th><th>acc.</th><th>pairs</th><th>25</th><th>50</th><th>100</th><th>in</th>
  </tr>
  <tr>
    <td>LLaVA-v1</td>
    <td>45.55</td><td>45.04</td><td>50</td><td>/</td><td>0.49</td><td>0.50</td><td>2.21</td><td>0.15</td>
  </tr>
  <tr>
    <td>LLaVA-v1.6</td>
    <td>72.27</td><td>57.47</td><td>50</td><td>48.75</td><td>47.82</td><td>45.42</td><td>/</td><td>41.48</td><td>40.15</td><td>36.42</td><td>28.77</td>
  </tr>
  <tr>
    <td>MiniGPT-v2</td>
    <td>45.44</td><td>48.27</td><td>25</td><td>33.33</td><td>33.33</td><td>1.48</td><td>0.57</td>
  </tr>
  <tr>
    <td>Unimodal</td>
    <td>52.32</td><td>49.84</td><td>50</td><td>35.87</td><td>35.87</td><td>50.00</td><td>42.02</td>
  </tr>
  <tr>
    <td>MedFlamingo</td>
    <td>51.47</td><td>50.78</td><td>50</td><td>30.83</td><td>28.42</td><td>15.36</td><td>8.02</td>
  </tr>
  <tr>
    <td>ChexGenet</td>
    <td>54.98</td><td>71.78</td><td>/</td><td>53.54</td><td>53.43</td><td>53.52</td><td>53.07</td>
  </tr>
  <tr>
    <td>LLaVA-Med-v1</td>
    <td>8.41</td><td>7.92</td><td>53.33</td><td>17.82</td><td>5.74</td><td>57.48</td>
  </tr>
  <tr>
    <td>LLaVA-Med-v1.5</td>
    <td>52.72</td><td>28.96</td><td>50</td><td>52.82</td><td>47.58</td><td>47.35</td><td>52.23</td>
  </tr>
  <tr>
    <td>BiomeRIPF</td>
    <td>46.66</td><td>50.00</td><td>50.00</td><td>48.81</td><td>40.92</td><td>34.32</td><td>9.27</td>
  </tr>
  <tr>
    <td>BioMed-Flamingo</td>
    <td>47.01</td><td>47.87</td><td>/</td><td>57.89</td><td>57.49</td><td>44.35</td><td>44.97</td>
  </tr>
  <tr>
    <td>ChexGaret</td>
    <td>97.01</td><td>89.60</td><td>89.60</td><td>89.60</td><td>89.60</td><td>89.60</td><td>89.60</td>
  </tr>
  <tr>
    <td>GPT-4o</td>
    <td>95.84</td><td>79.79</td><td>47.82</td><td>34.15</td><td>25.86</td><td>84.59</td>
  </tr>
  <tr>
    <td>GPT-4V</td>
    <td>97.02</td><td>90.80</td><td>/</td><td>86.89</td><td>86.93</td><td>73.67</td><td>80.63</td>
  </tr>
  <tr>
    <td>GPT-4V</td>
    <td>92.03</td><td>85.32</td><td></td><td>88.72</td><td>58.27</td><td>52.11</td><td>71.58</td>
  </tr>
  <tr>
    <td>Gemini Pro</td>
    <td>98.02</td><td>70.14</td><td>70.00</td><td>17.91</td><td>19.07</td><td>19.07</td><td>91.77</td>
  </tr>
  <tr>
    <td>Gemini Pro</td>
    <td>98.02</td><td>44.87</td><td></td><td>99.04</td><td>84.12</td><td>84.82</td><td>81.88</td>
  </tr>
  <tr>
    <td>num</td>
    <td>201</td><td>201</td><td>24</td><td>201</td><td>201</td><td>201</td><td>201</td>
  </tr>
</table>

A.12 Abdominal X-ray

Table 16: Results of different models on Abdominal X-ray in ProbMed. The best-performing model in each question category is in-bold, and the second best is underlined.

<table>
  <tr>
    <th rowspan="2"> </th>
    <th colspan="7">General Question</th>
    <th colspan="7">Specialized Question</th>
  </tr>
  <tr>
    <th colspan="6">Modality Organ Abnormality Condition/Finding Position</th>
    <th colspan="6">Organ Condition/Finding Position</th>
  </tr>
  <tr>
    <th>Random Choice</th>
    <th>acc.</th><th>pairs</th><th>5</th><th>25</th><th>50</th><th>100</th><th>in</th><th colspan="1"></th><th>acc.</th><th>pairs</th><th>25</th><th>50</th><th>100</th><th>5</th><th>25</th><th>50</th><th>100</th><th>in</th>
  </tr>
  <tr>
    <td>LLaVA-v1</td>
    <td>53.87</td><td>53.01</td><td>50</td><td>/</td><td>2.15</td><td>0.56</td><td>0.50</td>
  </tr>
  <tr>
    <td>LLaVA-v1.6</td>
    <td>76.93</td><td>76.60</td><td>50</td><td>_</td><td>4.94</td><td>0.14</td><td>0.00</td>
  </tr>
  <tr>
    <td>MiniGPT-v2</td>
    <td>51.77</td><td>56.46</td><td>46</td><td>/</td><td>6.46</td><td>1.12</td><td>0.52</td>
  </tr>
  <tr>
    <td>GPT-4o</td>
    <td>27.15</td><td>21.37</td><td>/</td><td>47.84</td><td>54.46</td><td>40.46</td><td>42.55</td>
  </tr>
  <tr>
    <td>GPT-4V</td>
    <td>47.44</td><td>38.97</td><td>50</td><td>18.53</td><td>56.4</td><td>50.23</td><td></td>
  </tr>
  <tr>
    <td>LLaVA-Med-v1</td>
    <td>53.87</td><td>53.01</td><td>50</td><td>/</td><td>2.15</td><td>0.56</td><td>0.50</td>
  </tr>
  <tr>
    <td>LLaVA-Med-v1.5</td>
    <td>52.58</td><td>50.86</td><td>50</td><td>/</td><td>6.46</td><td>1.12</td><td>0.52</td>
  </tr>
  <tr>
    <td>BioMed-Flamingo</td>
    <td>97.01</td><td>79.59</td><td>52.94</td><td>/</td><td>29.24</td><td>38.12</td><td>46.69</td>
  </tr>
  <tr>
    <td>GPT-4o</td>
    <td>98.26</td><td>60.91</td><td>60.91</td><td>57.29</td><td>51.78</td><td>52.11</td><td>51.69</td>
  </tr>
  <tr>
    <td>Gemein Pro</td>
    <td>98.26</td><td>45.71</td><td></td><td>95.43</td><td>57.00</td><td>53.13</td><td>92.87</td>
  </tr>
  <tr>
    <td>Gemini Pro</td>
    <td>98.26</td><td>81.36</td><td>81.36</td><td>81.36</td><td>81.36</td><td>81.36</td><td>81.36</td>
  </tr>
  <tr>
    <td>num</td>
    <td>232</td><td>232</td><td>20</td><td>232</td><td>232</td><td>232</td><td>232</td>
  </tr>
</table>

B Dataset Statistics

In this section, we detail the dataset statistics.

Table 17 shows the number of questions across each question type. For each image sample, ground-truth questions were created based on available metadata with "yes" answers. For each ground-truth question, we also created a corresponding adversarial question by selecting random adversarial entities and assigning "no" answers. For an image showing a normal organ without abnormality, since there is no ground-truth information on
the existence of the condition and position, we only construct hallucinated questions for the condition/finding question type. For an image showing abnormality, the number of question pairs per category equals the number of existing conditions or positions.

Table 17: Number of questions across each question type for each image.

<table>
  <tr>
    <th>Question type</th>
    <th>Image with Normal Organ</th>
    <th>Image with Abnormality</th>
  </tr>
  <tr>
    <td>Modality</td>
    <td>2</td>
    <td>2</td>
  </tr>
  <tr>
    <td>Organ</td>
    <td>2</td>
    <td>2</td>
  </tr>
  <tr>
    <td>Abnormality</td>
    <td>1</td>
    <td>1</td>
  </tr>
  <tr>
    <td>Condition/Finding</td>
    <td>1</td>
    <td>2 x number of existing conditions</td>
  </tr>
  <tr>
    <td>Position</td>
    <td>0</td>
    <td>2 x number of existing positions</td>
  </tr>
</table>

C Robustness Study

We conducted an additional robustness analysis using GPT-4o—the best-performing model from our evaluations—on the subset of 100 images and 1,090 VQA pairs used in our expert validation. The following table presents results on randomly sampled test samples from the expert-validated subset (20%, 40%, 60%, 80%, and 100%) and compares accuracy across all samples versus manually validated ("valid") samples:

Table 18: Valid percentage and accuracy under all samples vs. valid samples on different test sample splits. \( N_v \) denotes the number of images in the split, and \( N \) denotes the number of corresponding question-answer pairs.

<table>
  <tr>
    <th>\( N_v, N \)</th>
    <th>Valid QA (%)</th>
    <th>Acc. all samples (%)</th>
    <th>Acc. valid samples (%)</th>
  </tr>
  <tr>
    <td>20, 218</td>
    <td>100.00</td>
    <td>52.63</td>
    <td>52.63</td>
  </tr>
  <tr>
    <td>40, 421</td>
    <td>96.19</td>
    <td>54.36</td>
    <td>55.19</td>
  </tr>
  <tr>
    <td>60, 596</td>
    <td>96.81</td>
    <td>55.32</td>
    <td>55.25</td>
  </tr>
  <tr>
    <td>80, 821</td>
    <td>95.73</td>
    <td>54.21</td>
    <td>54.55</td>
  </tr>
  <tr>
    <td>100, 1090</td>
    <td>97.79</td>
    <td>55.63</td>
    <td>55.59</td>
  </tr>
  <tr>
    <td>6303, 57132</td>
    <td>/</td>
    <td>55.60</td>
    <td>/</td>
  </tr>
</table>

From these results, we observe minimal differences between the accuracy of all samples versus the validated subset. This indicates our evaluation framework is highly robust to the approximately 6% error rate identified during pre-processing. Furthermore, the accuracy of GPT-4o on the full ProbMed dataset (55.60%) closely aligns with the performance on the subset (55.63%), confirming the generalizability of our robustness findings.

D Impact of Chain-of-Thought Prompts and Visual Descriptions on Model Performance

In this section, we illustrate model performance under different categories under the three prompting settings: Vanilla, Chain-of-Thought, and Chain-of-Thought with GPT-4o Visual Understanding.

![Accuracy of the LLaVA-v1 model across five diagnostic categories under three settings: vanilla (blue), chain-of-thought (CoT, red), and CoT with GPT-4o Visual Understanding (green).](page_1012_320_324_324.png)

Figure 6: Accuracy of the LLaVA-v1 model across five diagnostic categories under three settings: vanilla (blue), chain-of-thought (CoT, red), and CoT with GPT-4o Visual Understanding (green).

![Accuracy of the LLaVA-v1.6 model across five diagnostic categories under three settings: vanilla (blue), chain-of-thought (CoT, red), and CoT with GPT-4o Visual Understanding (green).](page_1012_680_324_324.png)

Figure 7: Accuracy of the LLaVA-v1.6 model across five diagnostic categories under three settings: vanilla (blue), chain-of-thought (CoT, red), and CoT with GPT-4o Visual Understanding (green).

![Accuracy of the LLaVA-Med-v1 model across five diagnostic categories under three settings: vanilla (blue), chain-of-thought (CoT, red), and CoT with GPT-4o Visual Understanding (green).](page_1012_1040_324_324.png)

Figure 8: Accuracy of the LLaVA-Med-v1 model across five diagnostic categories under three settings: vanilla (blue), chain-of-thought (CoT, red), and CoT with GPT-4o Visual Understanding (green).
Figure 9: Accuracy of the LLaVA-Med-v1.5 model across five diagnostic categories under three settings: vanilla (blue), chain-of-thought (CoT, red), and CoT with GPT-4o Visual Understanding (green).

Figure 10: Accuracy of the Med-Flamingo model across five diagnostic categories under three settings: vanilla (blue), chain-of-thought (CoT, red), and CoT with GPT-4o Visual Understanding (green).

Figure 11: Accuracy of the GPT-4o model across five diagnostic categories under three settings: vanilla (blue), chain-of-thought (CoT, red), and CoT with GPT-4o Visual Understanding (green).

E Prompt Details

Below is the few-shot model prompt used for GPT-4 to extract medical conditions and their locations from image captions:

You are a helpful assistant and you are given a caption describing a medical image. Extract medical conditions and diseases along with their location, if specified. Do not include any information that cannot be directly inferred from the image, for example, patient status or patient history. Outputs should be in the format: "<condition/disease> <location> <condition/disease> <location>". The condition/location should include at least one positional descriptor and should be explicit in the original caption along with the condition/disease. Otherwise, it should be replaced with "None".

For example, consider the caption: "Fig. 1. MRI abdomen and pelvis showing the cervical mass." The output should be "<cervical mass> : None". For the caption: "Chest radiograph shows enlargement of the hilar mass with spread into the left lower lobe." The output should be "<enlargement of the hilar mass> <left lower lobe>". Similarly, for the caption: "Axial CT scan reveals an enhancing rounded pseudo-aneurysm of the cystic artery, alongside high-density material within the gallbladder's lumen and near the gastrohepatic ligament". The correct output is "<enhancing rounded pseudo-aneurysm of the cystic artery> <high-density material> : <lumen of the gallbladder and region of the gastrohepatic ligament>".

Make sure that the response contains only the information in the original caption without adding extra details.

F Response Distribution Visualization within each Category

Below, we illustrate the distribution of model responses within each category, valid "yes" responses are in red, and valid "no" responses are in blue.

Figure 12: Distribution plot of "yes and "no" ground-truth answers and model responses within the Modality category.

Figure 13: Distribution plot of "yes and "no" ground-truth answers and model responses within the Organ category.
Figure 14: Distribution plot of "yes and "no" ground-truth answers and model responses within the Abnormality category.

Figure 15: Distribution plot of "yes and "no" ground-truth answers and model responses within the Condition/Finding category.

Figure 16: Distribution plot of "yes and "no" ground-truth answers and model responses within the Position category.

Reviewer paper summaries: This paper examines the reliability of Large Multimodal Models (LMMs) in Medical Visual Question Answering (Med-VQA), introducing a dataset called ProbMed that employs adversarial testing to expose significant performance shortcomings in handling specialized medical inquiries, while highlighting the importance of improving visual understanding and incorporating domain-specific training.

This paper introduces ProbMed, a comprehensive evaluation framework for Medical Visual Question Answering (Med-VQA). It aims to uncover the limitations of state-of-the-art large multimodal models (LMMs) in clinical diagnostic tasks and highlight performance gaps in current vision LMMs in a complex set of medical subtasks.

In this paper the authors provide evidence for ineffectiveness of medical VQA benchmarks in tracking diagnostic reliability. A ProbMed dataset of binary question-answer pairs is algorithmically curated from two biomedical datasets using LLM. First, metadata such as abnormality, condition entity, modality are extracted from image and caption pairs from MedICaT and ChestX-ray14. Some of reasoned caption information are related to positional information in the image. These metadata are then used to generate question answer pairs. Corresponding adversarial questions are generated by flipping the entities in the question. During the model evaluation phase, sample evaluation is considered correct, if model correctly evaluates both question and its corresponding adversarial question. The authors show evidence that when adversarial questions are evaluated, some of the models performance lowers by 20-30%, resulting a model performing worse than random,
Summary of strengths: 1. This paper illustrates and proves that poor visual understanding is a key limitation restricting the development of LLMs in Med-VQA.
2. This paper contributes a new dataset.

1. Interesting findings. The paper demonstrates that high benchmark performance can be misleading by introducing adversarial evaluation with hallucinated and negated attributes.

2. The proposed ProbMed is systematically curated from existing biomedical datasets and includes diverse diagnostic dimensions, making the evaluation comprehensive.

- The dataset curation and evaluation method described are simple and well articulated, so that it's easy to reproduce the settings.
- The dataset is useful in enabling future research
- Demonstration of model under performance on adversarial samples on large scale data is important result
Summary of weaknesses: 1. The proposed dataset is built upon a pre-existing dataset, raising concerns about the sufficiency of the contribution.  
2. The dataset mainly consists of binary yes/no questions and is sourced from existing public datasets, making the expert evaluation in Section 3.5 seem less justified, and the criteria for expert evaluation are also unclear.  
3. The dataset's responses are limited to yes/no answers, making the task relatively simple compared to other similar QA tasks.  
4. The prompts used in the evaluation are not specified in the paper, and it is unclear whether the authors have explored the impact of different prompts on performance beyond the discussed CoT prompting.

1. While the "probing" strategy reveals weaknesses, it remains relatively too simplistic to discover in-depth insights. Adversarial-pairing-induced performance drops are not so novel and may not reflect real diagnostic application cases.

2. The study may lack practical insight into how to design or improve models for future improvements. While the dataset curation is extensive, it lacks example showcases and disclosures on how the QA pairs were filtered or validated beyond a small expert check. It remains unclear how the proposed dataset, ProbMed, can serve as a scalable and practical benchmark to improve the robustness and accuracy of future LLM-fostered clinical workflows.

The authors conducted a human evaluation on the correctness of the data on a relatively small sample (n=100) and they show it's about 94% correct. I wish to have seen some elaboration on how dataset error translates to model evaluation robustness. 

The questions were generated from metadata using LLM. It wasn't apparent to me what model was used to generate. There might be some bias if the model used in generating question is also the base model of the evaluated ones.
Comments, suggestions and typos: 1. It seems unusual to place a figure or table directly in the abstract.

1. It would be beneficial to have more justifications for why the contribution would be helpful in real-world medical diagnostic applications.

2. Further discussion and expansion of current experiment designs about how to mitigate the current issue and prevent hallucinations would provide better contributions to this work.

3. More data examples would be beneficial in providing future illustrations of why the model is not working well when providing adversarial information.

NA
Overall assessment scores: 2.5

2.5

4