switch to OE VQA acc
Browse files
README.md
CHANGED
@@ -171,7 +171,7 @@ We perform checkpoint selection based on validation sets of TODO, and select the
|
|
171 |
|
172 |
TODO: beautiful plots of shots scaling laws.
|
173 |
|
174 |
-
| Model | Shots | VQAv2 (
|
175 |
|:-----------|--------:|-----------------------------:|-----------------------------:|-------------------------------:|------------------------------:|-------------------:|---------------:|-----------------:|-----------------:|------------------------:|-----------------:|-------------------------:|-----------------------:|--------------------------:|----------------------------------------------------:|
|
176 |
| IDEFIX 80B | 0 | 60.1 | 45.2 | 31 | 36 | 56.9 | 91.9 | 65 | 53.7 | 74.3 | 48.9 | 60.7 | 69 | 60.6 | 8 (18.8/22.5) |
|
177 |
| | 4 | 63.5 | 52.4 | 34.7 | 45.9 | 77.9 | 109.3 | 101.1 | 69 | - | 48.7 | 58.7 | 66.3 | 64 | - |
|
|
|
171 |
|
172 |
TODO: beautiful plots of shots scaling laws.
|
173 |
|
174 |
+
| Model | Shots | VQAv2 (OE VQA acc) | OKVQA (OE VQA acc) | TextVQA (OE VQA acc) | VizWiz (OE VQA acc) | TextCaps (CIDEr) | Coco (CIDEr) | NoCaps (CIDEr) | Flickr (CIDEr) | ImageNet1k (accuracy) | VisDial (NDCG) | HatefulMemes (ROC AUC) | ScienceQA (accuracy) | RenderedSST2 (accuracy) | Winoground (group_score (image_score/text_score)) |
|
175 |
|:-----------|--------:|-----------------------------:|-----------------------------:|-------------------------------:|------------------------------:|-------------------:|---------------:|-----------------:|-----------------:|------------------------:|-----------------:|-------------------------:|-----------------------:|--------------------------:|----------------------------------------------------:|
|
176 |
| IDEFIX 80B | 0 | 60.1 | 45.2 | 31 | 36 | 56.9 | 91.9 | 65 | 53.7 | 74.3 | 48.9 | 60.7 | 69 | 60.6 | 8 (18.8/22.5) |
|
177 |
| | 4 | 63.5 | 52.4 | 34.7 | 45.9 | 77.9 | 109.3 | 101.1 | 69 | - | 48.7 | 58.7 | 66.3 | 64 | - |
|