Update README.md
Browse files
README.md
CHANGED
@@ -70,17 +70,26 @@ It achieves the following results on the evaluation set:
|
|
70 |
- Loss: 3.7502
|
71 |
- Accuracy: 0.3650
|
72 |
|
73 |
-
##
|
74 |
-
|
75 |
-
|
76 |
-
|
77 |
-
|
78 |
-
|
79 |
-
|
80 |
-
|
81 |
-
|
82 |
-
|
83 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
84 |
|
85 |
## Training procedure
|
86 |
|
|
|
70 |
- Loss: 3.7502
|
71 |
- Accuracy: 0.3650
|
72 |
|
73 |
+
## eval
|
74 |
+
|
75 |
+
initial 'get the feet wet':
|
76 |
+
|
77 |
+
`hf-causal-experimental (pretrained=pszemraj/mega-ar-small-4096-sw_minipile,revision=main,trust_remote_code=True,dtype='float'), limit: None, provide_description: False, num_fewshot: 0, batch_size: 16`
|
78 |
+
|
79 |
+
| Task |Version| Metric | Value | | Stderr |
|
80 |
+
|--------------|------:|--------|--------:|---|-------:|
|
81 |
+
|arc_easy | 0|acc | 0.3173|± | 0.0096|
|
82 |
+
| | |acc_norm| 0.3022|± | 0.0094|
|
83 |
+
|boolq | 1|acc | 0.4107|± | 0.0086|
|
84 |
+
|lambada_openai| 0|ppl |6843.1824|± |295.0792|
|
85 |
+
| | |acc | 0.0155|± | 0.0017|
|
86 |
+
|openbookqa | 0|acc | 0.1220|± | 0.0147|
|
87 |
+
| | |acc_norm| 0.2480|± | 0.0193|
|
88 |
+
|piqa | 0|acc | 0.5609|± | 0.0116|
|
89 |
+
| | |acc_norm| 0.5566|± | 0.0116|
|
90 |
+
|winogrande | 0|acc | 0.5059|± | 0.0141|
|
91 |
+
|
92 |
+
still some ways to go.
|
93 |
|
94 |
## Training procedure
|
95 |
|