Update README.md
Browse files
README.md
CHANGED
|
@@ -96,6 +96,35 @@ The benchmark result in [MTVQA](https://github.com/bytedance/MTVQA/tree/main)
|
|
| 96 |
| MiniCPM-V2.5 | ✓ | 15.3 |
|
| 97 |
| InternVL-V1.5 | ✗ | 12.4 |
|
| 98 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 99 |
We are still working on more detailed benchmarks.
|
| 100 |
|
| 101 |
## Examples
|
|
|
|
| 96 |
| MiniCPM-V2.5 | ✓ | 15.3 |
|
| 97 |
| InternVL-V1.5 | ✗ | 12.4 |
|
| 98 |
|
| 99 |
+
## OpenCompass Benchmark
|
| 100 |
+
|
| 101 |
+
<div align="center">
|
| 102 |
+
<img src="radar_chart.png" width="400"/>
|
| 103 |
+
</div>
|
| 104 |
+
|
| 105 |
+
We evaluate Vintern-1B-v2 on [VLMEvalKit](https://github.com/open-compass/VLMEvalKit). (We use GPT4o-mini for some judge model)
|
| 106 |
+
|
| 107 |
+
The current results are at a quite good level, and we are expanding the training set in English and other languages to approach models within a comparable parameter range.
|
| 108 |
+
|
| 109 |
+
"The table is referenced from the repo [Qwen/Qwen2-VL-2B-Instruct](https://huggingface.co/Qwen/Qwen2-VL-2B-Instruct)."
|
| 110 |
+
|
| 111 |
+
| Benchmark | InternVL2-2B | MiniCPM-V 2.0 | Qwen2-VL-2B | Vintern-1B-v2 |
|
| 112 |
+
|:-----------------|:------------:|:-------------:|:-----------:|:---------------:|
|
| 113 |
+
| MMMUval | 36.3 | 38.2 | 41.1 | 29.56 |
|
| 114 |
+
| DocVQAtest | 86.9 | - | 90.1 | - |
|
| 115 |
+
| InfoVQAtest | 58.9 | - | 65.5 | - |
|
| 116 |
+
| ChartQAtest | 76.2 | - | 73.5 | 34.08 |
|
| 117 |
+
| TextVQAval | 73.4 | - | 79.7 | 64.31 |
|
| 118 |
+
| OCRBench | 781 | 605 | 794 | 628 |
|
| 119 |
+
| MTVQA | 10.9 | 8.8 | 20.0 | 14.63 |
|
| 120 |
+
| Vi-MTVQA | 9.3 | 8.4 | - | 37.9 |
|
| 121 |
+
| RealWorldQA | 57.3 | 55.8 | 62.9 | 51.9 |
|
| 122 |
+
| MMEsum | 1876.8 | 1808.6 | 1872.0 | 1192.9 |
|
| 123 |
+
| MMBench-ENtest | 73.2 | 69.1 | 74.9 | 70.62 |
|
| 124 |
+
| MMStar | 49.8 | 39.1 | 48.0 | 38.67 |
|
| 125 |
+
| HallBenchavg | 38.0 | 36.1 | 41.7 | - |
|
| 126 |
+
| MathVistatestmini| 46.0 | 39.8 | 43.0 | 32.9 |
|
| 127 |
+
|
| 128 |
We are still working on more detailed benchmarks.
|
| 129 |
|
| 130 |
## Examples
|