Leyo commited on
Commit
5cc3ed2
1 Parent(s): 0b721e1

switch to OE VQA acc

Browse files
Files changed (1) hide show
  1. README.md +1 -1
README.md CHANGED
@@ -171,7 +171,7 @@ We perform checkpoint selection based on validation sets of TODO, and select the
171
 
172
  TODO: beautiful plots of shots scaling laws.
173
 
174
- | Model | Shots | VQAv2 (open-ended vqa acc) | OKVQA (open-ended vqa acc) | TextVQA (open-ended vqa acc) | VizWiz (open-ended vqa acc) | TextCaps (CIDEr) | Coco (CIDEr) | NoCaps (CIDEr) | Flickr (CIDEr) | ImageNet1k (accuracy) | VisDial (NDCG) | HatefulMemes (ROC AUC) | ScienceQA (accuracy) | RenderedSST2 (accuracy) | Winoground (group_score (image_score/text_score)) |
175
  |:-----------|--------:|-----------------------------:|-----------------------------:|-------------------------------:|------------------------------:|-------------------:|---------------:|-----------------:|-----------------:|------------------------:|-----------------:|-------------------------:|-----------------------:|--------------------------:|----------------------------------------------------:|
176
  | IDEFIX 80B | 0 | 60.1 | 45.2 | 31 | 36 | 56.9 | 91.9 | 65 | 53.7 | 74.3 | 48.9 | 60.7 | 69 | 60.6 | 8 (18.8/22.5) |
177
  | | 4 | 63.5 | 52.4 | 34.7 | 45.9 | 77.9 | 109.3 | 101.1 | 69 | - | 48.7 | 58.7 | 66.3 | 64 | - |
 
171
 
172
  TODO: beautiful plots of shots scaling laws.
173
 
174
+ | Model | Shots | VQAv2 (OE VQA acc) | OKVQA (OE VQA acc) | TextVQA (OE VQA acc) | VizWiz (OE VQA acc) | TextCaps (CIDEr) | Coco (CIDEr) | NoCaps (CIDEr) | Flickr (CIDEr) | ImageNet1k (accuracy) | VisDial (NDCG) | HatefulMemes (ROC AUC) | ScienceQA (accuracy) | RenderedSST2 (accuracy) | Winoground (group_score (image_score/text_score)) |
175
  |:-----------|--------:|-----------------------------:|-----------------------------:|-------------------------------:|------------------------------:|-------------------:|---------------:|-----------------:|-----------------:|------------------------:|-----------------:|-------------------------:|-----------------------:|--------------------------:|----------------------------------------------------:|
176
  | IDEFIX 80B | 0 | 60.1 | 45.2 | 31 | 36 | 56.9 | 91.9 | 65 | 53.7 | 74.3 | 48.9 | 60.7 | 69 | 60.6 | 8 (18.8/22.5) |
177
  | | 4 | 63.5 | 52.4 | 34.7 | 45.9 | 77.9 | 109.3 | 101.1 | 69 | - | 48.7 | 58.7 | 66.3 | 64 | - |