nitzanguetta
commited on
Commit
•
969a984
1
Parent(s):
46b2dc4
Add new leaderboard
Browse files- .DS_Store +0 -0
- WHOOPS-Leaderboard-Full.tsv +12 -0
- app.py +1 -1
- whoops_leaderboard_full.tsv +0 -31
.DS_Store
ADDED
|
Binary file (6.15 kB). View file
|
|
|
WHOOPS-Leaderboard-Full.tsv
ADDED
|
@@ -0,0 +1,12 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
Model Image Captioning Visual Question Answering Image-Text Matching Human Metric - Explanation of Violation Auto Metric - Explanation of Violation identify - Explanation of Violation
|
| 2 |
+
Humans 95 92
|
| 3 |
+
Ground-truth Caption _ GPT3 (Oracle) 68 62 74
|
| 4 |
+
BLIP2 FlanT5-XXL (Fine-tuned) 177 57 84 27 24 73
|
| 5 |
+
BLIP2 FlanT5-XL (Fine-tuned) 174 55 81 15 18 60
|
| 6 |
+
Predicted Caption _ GPT3 33 42 59
|
| 7 |
+
BLIP2 FlanT5-XXL (Zero-shot) 120 55 71 0 0 50
|
| 8 |
+
CLIP ViT-L/14 (Zero-shot) 70
|
| 9 |
+
OFA Large (Zero-shot) 0 38
|
| 10 |
+
CoCa ViT-L-14 MSCOCO (Zero-shot) 102 72
|
| 11 |
+
BLIP Large (Zero-shot) 65 39 77
|
| 12 |
+
BLIP2 FlanT5-XXL (Text only FT) 2 24 94
|
app.py
CHANGED
|
@@ -2,7 +2,7 @@ import gradio as gr
|
|
| 2 |
import pandas as pd
|
| 3 |
|
| 4 |
|
| 5 |
-
df = pd.read_table("
|
| 6 |
headline = """# WHOOPS! Leaderboard Full
|
| 7 |
To submit your results to the leaderboard, please add a "predictions" column to [this csv](https://huggingface.co/datasets/nlphuji/whoops/blob/main/whoops_dataset.csv), and send to [this mail](mailto:yonatanbitton1@gmail.com) or [this mail](mailto:whoops.benchmark@gmail.com,).
|
| 8 |
"""
|
|
|
|
| 2 |
import pandas as pd
|
| 3 |
|
| 4 |
|
| 5 |
+
df = pd.read_table("wWHOOPS-Leaderboard-Full.tsv")
|
| 6 |
headline = """# WHOOPS! Leaderboard Full
|
| 7 |
To submit your results to the leaderboard, please add a "predictions" column to [this csv](https://huggingface.co/datasets/nlphuji/whoops/blob/main/whoops_dataset.csv), and send to [this mail](mailto:yonatanbitton1@gmail.com) or [this mail](mailto:whoops.benchmark@gmail.com,).
|
| 8 |
"""
|
whoops_leaderboard_full.tsv
DELETED
|
@@ -1,31 +0,0 @@
|
|
| 1 |
-
|
| 2 |
-
Model Image Captioning Visual Question Answering Image-Text Matching Human Metric - Explanation of Violation Auto Metric - Explanation of Violation identify - Explanation of Violation
|
| 3 |
-
Humans 95 92
|
| 4 |
-
Ground-truth Caption _ GPT3 (Oracle) 68 62 74
|
| 5 |
-
BLIP2 FlanT5-XXL (Fine-tuned) 177 57 84 27 24 73
|
| 6 |
-
BLIP2 FlanT5-XL (Fine-tuned) 174 55 81 15 18 60
|
| 7 |
-
Predicted Caption _ GPT3 33 42 59
|
| 8 |
-
BLIP2 FlanT5-XXL (Zero-shot) 120 55 71 0 0 50
|
| 9 |
-
CLIP ViT-L/14 (Zero-shot) 70
|
| 10 |
-
OFA Large (Zero-shot) 0 38
|
| 11 |
-
CoCa ViT-L-14 MSCOCO (Zero-shot) 102 72
|
| 12 |
-
BLIP Large (Zero-shot) 65 39 77
|
| 13 |
-
BLIP2 FlanT5-XXL (Text only FT) 2 24 94
|
| 14 |
-
|
| 15 |
-
|
| 16 |
-
|
| 17 |
-
|
| 18 |
-
|
| 19 |
-
|
| 20 |
-
|
| 21 |
-
|
| 22 |
-
|
| 23 |
-
|
| 24 |
-
|
| 25 |
-
|
| 26 |
-
|
| 27 |
-
|
| 28 |
-
|
| 29 |
-
|
| 30 |
-
|
| 31 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|