Commit
•
df35ec4
1
Parent(s):
dcb5a04
add links
Browse files
README.md
CHANGED
|
@@ -72,19 +72,19 @@ IDEFICS-2 exhibits strong performance for a model of its size (8B parameters) wh
|
|
| 72 |
|
| 73 |
| Model | Open weights | Size | # tokens per image | MMMU (val/test) | MathVista (testmini) | TextVQA (val) | MMBench (test)| VQAv2 (test-dev) | DocVQA (test)
|
| 74 |
|--------------|-------------|------|--------------------|-----------|-----------|---------|---------|---------|---------|
|
| 75 |
-
| DeepSeek-VL | ✅ | 7B | 576 | 36.6/- | 36.1 | - | 73.2 | - | - |
|
| 76 |
-
| LLaVa-NeXT-13B | ✅ | 13B | 2880 | 36.2/- | 35.3 | 67.1 | 70.0 | 82.8 | - |
|
| 77 |
-
| LLaVa-NeXT-34B | ✅ | 34B | 2880 | 51.1/44.7 | 46.5 | 69.5 | 79.3 | 83.7 | - | - |
|
| 78 |
| MM1-Chat-7B | ❌ | 7B | 720 | 37.0/35.6 | 35.9 | 72.8 | 72.3 | - | - |
|
| 79 |
| MM1-Chat-30B | ❌ | 30B | 720 | 44.7/40.3 | 39.4 | 73.5 | 75.1 | 83.7 | |
|
| 80 |
| Gemini 1.0 Pro | ❌ | ? | ? | 47.9/- | 45.2 | 74.6 | - | 71.2 | 88.1 |
|
| 81 |
| Gemini 1.5 Pro | ❌ | ? | ? | 58.5/- | 52.1 | 73.5 | - | 73.2 | 86.5 |
|
| 82 |
| Claude 3 Haiku | ❌ |? | ? | 50.2/- | 46.4 | - | - | - | 88.8 |
|
| 83 |
| | | | | | | |
|
| 84 |
-
| IDEFICS-1 instruct (32-shots) | ✅ | 80B | - | - | - | 39.3 | - | 68.8 | - |
|
| 85 |
| | | | | | | |
|
| 86 |
-
| IDEFICS-2 (w/o image splitting) | ✅ | 8B | 64 | 43.5/37.9 | 51.6 | 70.4 | 76.8 | 80.8 | 67.3 |
|
| 87 |
-
| IDEFICS-2 (w/ image splitting) | ✅ | 8B | 320 | 43.0/37.7 | 51.4 | 73.0 | 76.7 | 81.2 | 74.0 |
|
| 88 |
|
| 89 |
</details>
|
| 90 |
|
|
|
|
| 72 |
|
| 73 |
| Model | Open weights | Size | # tokens per image | MMMU (val/test) | MathVista (testmini) | TextVQA (val) | MMBench (test)| VQAv2 (test-dev) | DocVQA (test)
|
| 74 |
|--------------|-------------|------|--------------------|-----------|-----------|---------|---------|---------|---------|
|
| 75 |
+
| [DeepSeek-VL](https://huggingface.co/deepseek-ai/deepseek-vl-7b-chat) | ✅ | 7B | 576 | 36.6/- | 36.1 | - | 73.2 | - | - |
|
| 76 |
+
| [LLaVa-NeXT-13B](https://huggingface.co/liuhaotian/llava-v1.6-vicuna-13b) | ✅ | 13B | 2880 | 36.2/- | 35.3 | 67.1 | 70.0 | 82.8 | - |
|
| 77 |
+
| [LLaVa-NeXT-34B](https://huggingface.co/liuhaotian/llava-v1.6-34b) | ✅ | 34B | 2880 | 51.1/44.7 | 46.5 | 69.5 | 79.3 | 83.7 | - | - |
|
| 78 |
| MM1-Chat-7B | ❌ | 7B | 720 | 37.0/35.6 | 35.9 | 72.8 | 72.3 | - | - |
|
| 79 |
| MM1-Chat-30B | ❌ | 30B | 720 | 44.7/40.3 | 39.4 | 73.5 | 75.1 | 83.7 | |
|
| 80 |
| Gemini 1.0 Pro | ❌ | ? | ? | 47.9/- | 45.2 | 74.6 | - | 71.2 | 88.1 |
|
| 81 |
| Gemini 1.5 Pro | ❌ | ? | ? | 58.5/- | 52.1 | 73.5 | - | 73.2 | 86.5 |
|
| 82 |
| Claude 3 Haiku | ❌ |? | ? | 50.2/- | 46.4 | - | - | - | 88.8 |
|
| 83 |
| | | | | | | |
|
| 84 |
+
| [IDEFICS-1 instruct](https://huggingface.co/HuggingFaceM4/idefics-80b-instruct) (32-shots) | ✅ | 80B | - | - | - | 39.3 | - | 68.8 | - |
|
| 85 |
| | | | | | | |
|
| 86 |
+
| **IDEFICS-2** (w/o image splitting) | ✅ | 8B | 64 | 43.5/37.9 | 51.6 | 70.4 | 76.8 | 80.8 | 67.3 |
|
| 87 |
+
| **IDEFICS-2** (w/ image splitting) | ✅ | 8B | 320 | 43.0/37.7 | 51.4 | 73.0 | 76.7 | 81.2 | 74.0 |
|
| 88 |
|
| 89 |
</details>
|
| 90 |
|