Salesforce
/

xgen-mm-phi3-mini-instruct-r-v1

Image-Text-to-Text

feature-extraction

Model card Files Files and versions Community

SFXX commited on May 10

Commit

79eb3e7

•

1 Parent(s): 0e92823

Update README.md

Files changed (1) hide show

README.md +2 -2

README.md CHANGED Viewed

@@ -27,7 +27,7 @@ More technical details will come with a technical report soon.
 # Results
-### Pretrain
 | Model       | Shot | COCO (val) | NoCaps (val) | TextCaps (val) | OKVQA  (val) | TextVQA (val) | VizWiz (testdev) | VQAv2 (testdev) |
 |-------------|------|------------|--------------|----------------|--------------|---------------|------------------|-----------------|
 | Flamingo-3B |    4 |       85.0 | -            | -              |         43.3 |          32.7 |               34 |            53.2 |
@@ -39,7 +39,7 @@ More technical details will come with a technical report soon.
 |             |    4 |      110.5 |        **101.7** |           **84.6** |         **49.2** |          **46.1** |             **38.4** |            **63.9** |
 |             |    8 |      112.1 |        104.4 |           87.7 |         **49.1** |          **46.4** |             44.3 |            **63.8** |
-### Instruct
 | Model                      | SEED-IMG | MMBench(dev) | MME-total | MME-P    | MME-C   | MMStar   | MMMU (val) | MMVet    | MathVista (mini) | ScienceQA (test) | POPE      | AI2D     |   |
 |----------------------------|----------|--------------|-----------|----------|---------|----------|------------|----------|------------------|------------------|----------|----------|---|
 | MM1-3B-Chat                | 68.8     | **75.9**         | 1761      | **1482**     | 279     | -        | 33.9       | 43.7     | -                | -                | **87.4**            | -        |   |

 # Results
+### Pretrain (base model without instruction tuning)
 | Model       | Shot | COCO (val) | NoCaps (val) | TextCaps (val) | OKVQA  (val) | TextVQA (val) | VizWiz (testdev) | VQAv2 (testdev) |
 |-------------|------|------------|--------------|----------------|--------------|---------------|------------------|-----------------|
 | Flamingo-3B |    4 |       85.0 | -            | -              |         43.3 |          32.7 |               34 |            53.2 |
 |             |    4 |      110.5 |        **101.7** |           **84.6** |         **49.2** |          **46.1** |             **38.4** |            **63.9** |
 |             |    8 |      112.1 |        104.4 |           87.7 |         **49.1** |          **46.4** |             44.3 |            **63.8** |
+### Instruct (after instruction tuning)
 | Model                      | SEED-IMG | MMBench(dev) | MME-total | MME-P    | MME-C   | MMStar   | MMMU (val) | MMVet    | MathVista (mini) | ScienceQA (test) | POPE      | AI2D     |   |
 |----------------------------|----------|--------------|-----------|----------|---------|----------|------------|----------|------------------|------------------|----------|----------|---|
 | MM1-3B-Chat                | 68.8     | **75.9**         | 1761      | **1482**     | 279     | -        | 33.9       | 43.7     | -                | -                | **87.4**            | -        |   |