Update README.md
Browse files
README.md
CHANGED
|
@@ -73,4 +73,26 @@ Details:
|
|
| 73 |
- train-from-scratch
|
| 74 |
|
| 75 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 76 |
---
|
|
|
|
| 73 |
- train-from-scratch
|
| 74 |
|
| 75 |
|
| 76 |
+
|
| 77 |
+
## evals
|
| 78 |
+
|
| 79 |
+
Initial data:
|
| 80 |
+
|
| 81 |
+
`hf-causal-experimental (pretrained=BEE-spoke-data/mega-ar-126m-4k,revision=main,trust_remote_code=True,dtype='float'), limit: None, provide_description: False, num_fewshot: 0, batch_size: 4`
|
| 82 |
+
|
| 83 |
+
| Task |Version| Metric | Value | |Stderr|
|
| 84 |
+
|--------------|------:|--------|------:|---|-----:|
|
| 85 |
+
|arc_easy | 0|acc | 0.4415|± |0.0102|
|
| 86 |
+
| | |acc_norm| 0.3969|± |0.0100|
|
| 87 |
+
|boolq | 1|acc | 0.5749|± |0.0086|
|
| 88 |
+
|lambada_openai| 0|ppl |94.9912|± |3.9682|
|
| 89 |
+
| | |acc | 0.2408|± |0.0060|
|
| 90 |
+
|openbookqa | 0|acc | 0.1660|± |0.0167|
|
| 91 |
+
| | |acc_norm| 0.2780|± |0.0201|
|
| 92 |
+
|piqa | 0|acc | 0.5974|± |0.0114|
|
| 93 |
+
| | |acc_norm| 0.5914|± |0.0115|
|
| 94 |
+
|winogrande | 0|acc | 0.4830|± |0.0140|
|
| 95 |
+
|
| 96 |
+
|
| 97 |
+
|
| 98 |
---
|