README.md

---
license: apache-2.0
language:
- en
---

# MedM-VL-2D-3B-en

## Introduction

A medical LVLM, trained on **English** data, accepts text and **a single 2D medical image** as input, and text-based results as output. enabling tasks such as **report generation**, **medical VQA**, **referring expression comprehension**, **referring expression generation** and **image classification**.

Here are the evaluation results on **Uni-Med**:

<table>
  <tr>
    <td align="center"> Method </td>
    <td align="center"> medmnist_derma </td>
    <td align="center"> medmnist_organs </td>
    <td align="center"> medpix </td>
    <td align="center"> mimic </td>
    <td align="center"> pathvqa </td>
    <td align="center"> samed_identify </td>
    <td align="center"> samed_refer </td>
    <td align="center"> slake_identify </td>
    <td align="center"> slake_refer </td>
    <td align="center"> slakevqa </td>
  </tr>
  <tr>
    <td> Med-Flamingo </td>
    <td align="center"> 1.15 </td>
    <td align="center"> 8.90 </td>
    <td align="center"> 8.14 </td>
    <td align="center"> 23.25 </td>
    <td align="center"> 33.38 </td>
    <td align="center"> - </td>
    <td align="center"> - </td>
    <td align="center"> - </td>
    <td align="center"> - </td>
    <td align="center"> 21.51 </td>
  </tr>
  <tr>
    <td> RadFM </td>
    <td align="center"> 5.14 </td>
    <td align="center"> 18.90 </td>
    <td align="center"> - </td>
    <td align="center"> 6.81 </td>
    <td align="center"> 24.83 </td>
    <td align="center"> - </td>
    <td align="center"> - </td>
    <td align="center"> - </td>
    <td align="center"> - </td>
    <td align="center"> 81.66 </td>
  </tr>
  <tr>
    <td> LLaVA-Med </td>
    <td align="center"> 25.84 </td>
    <td align="center"> 66.80 </td>
    <td align="center"> <b>15.11</b> </td>
    <td align="center"> 20.43 </td>
    <td align="center"> 37.79 </td>
    <td align="center"> 45.83 </td>
    <td align="center"> 8.64 </td>
    <td align="center"> 27.21 </td>
    <td align="center"> 4.07 </td>
    <td align="center"> 33.69 </td>
  </tr>
  <tr>
    <td> MedM-VL-2D-3B-en </td>
    <td align="center"> <b>81.05</b> </td>
    <td align="center"> <b>72.14</b> </td>
    <td align="center"> 13.16 </td>
    <td align="center"> <b>22.63</b> </td>
    <td align="center"> <b>62.86</b> </td>
    <td align="center"> <b>70.97</b> </td>
    <td align="center"> <b>20.46</b> </td>
    <td align="center"> <b>68.94</b> </td>
    <td align="center"> <b>31.92</b> </td>
    <td align="center"> <b>84.45</b> </td>
  </tr>
</table>


## Quickstart

Please refer to [MedM-VL](https://github.com/MSIIP/MedM-VL).