Hugging Face's logo

HuggingFaceM4
/
idefics2-8b-base

Image-Text-to-Text
Transformers
Safetensors
English
idefics2
pretraining
multimodal
vision
text-generation-inference
Inference Endpoints
Model card Files Community
9
Leyo commited on
Commit
55bae09
•
1 Parent(s): 27780fc

update table

Browse files
Files changed (1) hide show
  1. README.md +5 -4
README.md CHANGED
@@ -72,7 +72,8 @@ IDEFICS-2 exhibits strong performance for a model of its size (8B parameters) wh
72
 
73
  | <nobr>Model</nobr> | <nobr>Open <br>weights</nobr> | <nobr>Size</nobr> | <nobr># tokens <br>per image</nobr> | <nobr>MMMU <br>(val/test)</nobr> | <nobr>MathVista <br>(testmini)</nobr> | <nobr>TextVQA <br>(val)</nobr> | <nobr>MMBench <br>(test)</nobr> | <nobr>VQAv2 <br>(test-dev)</nobr> | <nobr>DocVQA <br>(test)</nobr> |
74
  |--------------|-------------|------|--------------------|-----------|-----------|---------|---------|---------|---------|
75
- | [DeepSeek-VL](https://huggingface.co/deepseek-ai/deepseek-vl-7b-chat) | ✅ | 7B | 576 | 36.6/- | 36.1 | - | 73.2 | - | - |
 
76
  | [LLaVa-NeXT-13B](https://huggingface.co/liuhaotian/llava-v1.6-vicuna-13b) | ✅ | 13B | 2880 | 36.2/- | 35.3 | 67.1 | 70.0 | 82.8 | - |
77
  | [LLaVa-NeXT-34B](https://huggingface.co/liuhaotian/llava-v1.6-34b) | ✅ | 34B | 2880 | 51.1/44.7 | 46.5 | 69.5 | 79.3 | 83.7 | - | - |
78
  | MM1-Chat-7B | ❌ | 7B | 720 | 37.0/35.6 | 35.9 | 72.8 | 72.3 | - | - |
@@ -81,10 +82,10 @@ IDEFICS-2 exhibits strong performance for a model of its size (8B parameters) wh
81
  | Gemini 1.5 Pro | ❌ | ? | ? | 58.5/- | 52.1 | 73.5 | - | 73.2 | 86.5 |
82
  | Claude 3 Haiku | ❌ |? | ? | 50.2/- | 46.4 | - | - | - | 88.8 |
83
  | | | | | | | |
84
- | [IDEFICS-1 instruct](https://huggingface.co/HuggingFaceM4/idefics-80b-instruct) (32-shots) | ✅ | 80B | - | - | - | 39.3 | - | 68.8 | - |
85
  | | | | | | | |
86
- | **IDEFICS-2** (w/o im. split) | ✅ | 8B | 64 | 43.5/37.9 | 51.6 | 70.4 | 76.8 | 80.8 | 67.3 |
87
- | **IDEFICS-2** (w/ im. split) | ✅ | 8B | 320 | 43.0/37.7 | 51.4 | 73.0 | 76.7 | 81.2 | 74.0 |
88
 
89
  </details>
90
 
 
72
 
73
  | <nobr>Model</nobr> | <nobr>Open <br>weights</nobr> | <nobr>Size</nobr> | <nobr># tokens <br>per image</nobr> | <nobr>MMMU <br>(val/test)</nobr> | <nobr>MathVista <br>(testmini)</nobr> | <nobr>TextVQA <br>(val)</nobr> | <nobr>MMBench <br>(test)</nobr> | <nobr>VQAv2 <br>(test-dev)</nobr> | <nobr>DocVQA <br>(test)</nobr> |
74
  |--------------|-------------|------|--------------------|-----------|-----------|---------|---------|---------|---------|
75
+ | [DeepSeek-VL](https://huggingface.co/deepseek-ai/deepseek-vl-7b-chat) | ✅ | 7B | 576 | 36.6/- | 36.1 | 64.4 | 73.2 | - | 49.6 |
76
+ | [LLaVa-NeXT-Mistral-7B](https://huggingface.co/liuhaotian/llava-v1.6-mistral-7b) | ✅ | 7B | 2880 | 35.3/- | 37.7 | 65.7 | 68.7 | 82.2 | - |
77
  | [LLaVa-NeXT-13B](https://huggingface.co/liuhaotian/llava-v1.6-vicuna-13b) | ✅ | 13B | 2880 | 36.2/- | 35.3 | 67.1 | 70.0 | 82.8 | - |
78
  | [LLaVa-NeXT-34B](https://huggingface.co/liuhaotian/llava-v1.6-34b) | ✅ | 34B | 2880 | 51.1/44.7 | 46.5 | 69.5 | 79.3 | 83.7 | - | - |
79
  | MM1-Chat-7B | ❌ | 7B | 720 | 37.0/35.6 | 35.9 | 72.8 | 72.3 | - | - |
 
82
  | Gemini 1.5 Pro | ❌ | ? | ? | 58.5/- | 52.1 | 73.5 | - | 73.2 | 86.5 |
83
  | Claude 3 Haiku | ❌ |? | ? | 50.2/- | 46.4 | - | - | - | 88.8 |
84
  | | | | | | | |
85
+ | [Idefics1 instruct](https://huggingface.co/HuggingFaceM4/idefics-80b-instruct) (32-shots) | ✅ | 80B | - | - | - | 39.3 | - | 68.8 | - |
86
  | | | | | | | |
87
+ | **Idefics2** (w/o im. split) | ✅ | 8B | 64 | 43.5/37.9 | 51.6 | 70.4 | 76.8 | 80.8 | 67.3 |
88
+ | **Idefics2** (w/ im. split) | ✅ | 8B | 320 | 43.0/37.7 | 51.4 | 73.0 | 76.7 | 81.2 | 74.0 |
89
 
90
  </details>
91