Leyo commited on
Commit
6edaaa6
1 Parent(s): 51b1776

winoground fix

Browse files
Files changed (1) hide show
  1. README.md +1 -1
README.md CHANGED
@@ -171,7 +171,7 @@ We perform checkpoint selection based on validation sets of TODO, and select the
171
 
172
  TODO: beautiful plots of shots scaling laws.
173
 
174
- | Model | Shots | VQAv2 (OE VQA acc) | OKVQA (OE VQA acc) | TextVQA (OE VQA acc) | VizWiz (OE VQA acc) | TextCaps (CIDEr) | Coco (CIDEr) | NoCaps (CIDEr) | Flickr (CIDEr) | ImageNet1k (accuracy) | VisDial (NDCG) | HatefulMemes (ROC AUC) | ScienceQA (accuracy) | RenderedSST2 (accuracy) | Winoground (group_score (image_score/text_score)) |
175
  |:-----------|--------:|---------------------:|---------------------:|-----------------------:|----------------------:|-------------------:|---------------:|-----------------:|-----------------:|------------------------:|-----------------:|-------------------------:|-----------------------:|--------------------------:|----------------------------------------------------:|
176
  | IDEFICS 80B | 0 | 60 | 45.2 | 30.9 | 36 | 56.8 | 91.8 | 65 | 53.7 | 74.3 | 48.8 | 60.6 | 68.9 | 60.5 | 8 (18.8/22.5)|
177
  | | 4 | 63.4 | 52.3 | 34.7 | 45.8 | 77.9 | 109.3 | 101.1 | 68.9 | - | 48.6 | 58.7 | 66.3 | 63.9 | - |
 
171
 
172
  TODO: beautiful plots of shots scaling laws.
173
 
174
+ | Model | Shots | VQAv2 (OE VQA acc) | OKVQA (OE VQA acc) | TextVQA (OE VQA acc) | VizWiz (OE VQA acc) | TextCaps (CIDEr) | Coco (CIDEr) | NoCaps (CIDEr) | Flickr (CIDEr) | ImageNet1k (accuracy) | VisDial (NDCG) | HatefulMemes (ROC AUC) | ScienceQA (accuracy) | RenderedSST2 (accuracy) | Winoground (group (text/image)) |
175
  |:-----------|--------:|---------------------:|---------------------:|-----------------------:|----------------------:|-------------------:|---------------:|-----------------:|-----------------:|------------------------:|-----------------:|-------------------------:|-----------------------:|--------------------------:|----------------------------------------------------:|
176
  | IDEFICS 80B | 0 | 60 | 45.2 | 30.9 | 36 | 56.8 | 91.8 | 65 | 53.7 | 74.3 | 48.8 | 60.6 | 68.9 | 60.5 | 8 (18.8/22.5)|
177
  | | 4 | 63.4 | 52.3 | 34.7 | 45.8 | 77.9 | 109.3 | 101.1 | 68.9 | - | 48.6 | 58.7 | 66.3 | 63.9 | - |