Update README.md
Browse files
README.md
CHANGED
@@ -51,3 +51,28 @@ hf (pretrained=pszemraj/t5e-mini-nl24-flan,trust_remote_code=True,dtype=bfloat16
|
|
51 |
|tinyHellaswag| 0|none | 10|acc_norm |↑ |0.3538|± | N/A|
|
52 |
|tinyMMLU | 0|none | 0|acc_norm |↑ |0.2551|± | N/A|
|
53 |
|winogrande | 1|none | 0|acc |↑ |0.5217|± |0.0140|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
51 |
|tinyHellaswag| 0|none | 10|acc_norm |↑ |0.3538|± | N/A|
|
52 |
|tinyMMLU | 0|none | 0|acc_norm |↑ |0.2551|± | N/A|
|
53 |
|winogrande | 1|none | 0|acc |↑ |0.5217|± |0.0140|
|
54 |
+
|
55 |
+
|
56 |
+
<details>
|
57 |
+
<summary>base model evals: click to expand</summary>
|
58 |
+
|
59 |
+
Quick eval for: `google/t5-efficient-mini-nl24`
|
60 |
+
|
61 |
+
|
62 |
+
hf (pretrained=google/t5-efficient-mini-nl24,trust_remote_code=True,dtype=bfloat16,trust_remote_code=True), gen_kwargs: (None), limit: None, num_fewshot: None, batch_size: 8
|
63 |
+
| Tasks |Version| Filter |n-shot| Metric | |Value | |Stderr|
|
64 |
+
|-------------|------:|----------------|-----:|-----------|---|-----:|---|------|
|
65 |
+
|boolq | 2|none | 0|acc |↑ |0.3783|± |0.0085|
|
66 |
+
|openbookqa | 1|none | 0|acc |↑ |0.1280|± |0.0150|
|
67 |
+
| | |none | 0|acc_norm |↑ |0.2660|± |0.0198|
|
68 |
+
|piqa | 1|none | 0|acc |↑ |0.5473|± |0.0116|
|
69 |
+
| | |none | 0|acc_norm |↑ |0.5267|± |0.0116|
|
70 |
+
|social_iqa | 0|none | 0|acc |↑ |0.3536|± |0.0108|
|
71 |
+
|tinyArc | 0|none | 25|acc_norm |↑ |0.3101|± | N/A|
|
72 |
+
|tinyGSM8k | 0|flexible-extract| 5|exact_match|↑ |0.0145|± | N/A|
|
73 |
+
| | |strict-match | 5|exact_match|↑ |0.0055|± | N/A|
|
74 |
+
|tinyHellaswag| 0|none | 10|acc_norm |↑ |0.2616|± | N/A|
|
75 |
+
|tinyMMLU | 0|none | 0|acc_norm |↑ |0.2839|± | N/A|
|
76 |
+
|winogrande | 1|none | 0|acc |↑ |0.4996|± |0.0141|
|
77 |
+
|
78 |
+
</details>
|