pszemraj commited on
Commit
34dcd7e
1 Parent(s): 9a6f73e

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +25 -0
README.md CHANGED
@@ -51,3 +51,28 @@ hf (pretrained=pszemraj/t5e-mini-nl24-flan,trust_remote_code=True,dtype=bfloat16
51
  |tinyHellaswag| 0|none | 10|acc_norm |↑ |0.3538|± | N/A|
52
  |tinyMMLU | 0|none | 0|acc_norm |↑ |0.2551|± | N/A|
53
  |winogrande | 1|none | 0|acc |↑ |0.5217|± |0.0140|
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
51
  |tinyHellaswag| 0|none | 10|acc_norm |↑ |0.3538|± | N/A|
52
  |tinyMMLU | 0|none | 0|acc_norm |↑ |0.2551|± | N/A|
53
  |winogrande | 1|none | 0|acc |↑ |0.5217|± |0.0140|
54
+
55
+
56
+ <details>
57
+ <summary>base model evals: click to expand</summary>
58
+
59
+ Quick eval for: `google/t5-efficient-mini-nl24`
60
+
61
+
62
+ hf (pretrained=google/t5-efficient-mini-nl24,trust_remote_code=True,dtype=bfloat16,trust_remote_code=True), gen_kwargs: (None), limit: None, num_fewshot: None, batch_size: 8
63
+ | Tasks |Version| Filter |n-shot| Metric | |Value | |Stderr|
64
+ |-------------|------:|----------------|-----:|-----------|---|-----:|---|------|
65
+ |boolq | 2|none | 0|acc |↑ |0.3783|± |0.0085|
66
+ |openbookqa | 1|none | 0|acc |↑ |0.1280|± |0.0150|
67
+ | | |none | 0|acc_norm |↑ |0.2660|± |0.0198|
68
+ |piqa | 1|none | 0|acc |↑ |0.5473|± |0.0116|
69
+ | | |none | 0|acc_norm |↑ |0.5267|± |0.0116|
70
+ |social_iqa | 0|none | 0|acc |↑ |0.3536|± |0.0108|
71
+ |tinyArc | 0|none | 25|acc_norm |↑ |0.3101|± | N/A|
72
+ |tinyGSM8k | 0|flexible-extract| 5|exact_match|↑ |0.0145|± | N/A|
73
+ | | |strict-match | 5|exact_match|↑ |0.0055|± | N/A|
74
+ |tinyHellaswag| 0|none | 10|acc_norm |↑ |0.2616|± | N/A|
75
+ |tinyMMLU | 0|none | 0|acc_norm |↑ |0.2839|± | N/A|
76
+ |winogrande | 1|none | 0|acc |↑ |0.4996|± |0.0141|
77
+
78
+ </details>