Update README.md
Browse files
README.md
CHANGED
|
@@ -26,6 +26,27 @@ It achieves the following results on the evaluation set:
|
|
| 26 |
- Accuracy: 0.2326
|
| 27 |
- Num Input Tokens Seen: 798621696
|
| 28 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 29 |
## Training procedure
|
| 30 |
|
| 31 |
### Training hyperparameters
|
|
|
|
| 26 |
- Accuracy: 0.2326
|
| 27 |
- Num Input Tokens Seen: 798621696
|
| 28 |
|
| 29 |
+
|
| 30 |
+
## numbers
|
| 31 |
+
|
| 32 |
+
tl;dr its bad/would need more training:
|
| 33 |
+
|
| 34 |
+
hf (pretrained=pszemraj/griffin-c3t-8L-v0.02-fineweb,trust_remote_code=True,dtype=float), gen_kwargs: (None), limit: None, num_fewshot: None, batch_size: 4
|
| 35 |
+
|
| 36 |
+
| Tasks |Version|Filter|n-shot| Metric | Value | | Stderr |
|
| 37 |
+
|--------------|------:|------|-----:|----------|----------:|---|---------:|
|
| 38 |
+
|winogrande | 1|none | 0|acc | 0.5146|± | 0.0140|
|
| 39 |
+
|piqa | 1|none | 0|acc | 0.5511|± | 0.0116|
|
| 40 |
+
| | |none | 0|acc_norm | 0.5261|± | 0.0116|
|
| 41 |
+
|openbookqa | 1|none | 0|acc | 0.1140|± | 0.0142|
|
| 42 |
+
| | |none | 0|acc_norm | 0.2240|± | 0.0187|
|
| 43 |
+
|lambada_openai| 1|none | 0|perplexity|209503.2246|± |11711.4041|
|
| 44 |
+
| | |none | 0|acc | 0.0000|± | 0.0000|
|
| 45 |
+
|boolq | 2|none | 0|acc | 0.3783|± | 0.0085|
|
| 46 |
+
|arc_easy | 1|none | 0|acc | 0.2593|± | 0.0090|
|
| 47 |
+
| | |none | 0|acc_norm | 0.2774|± | 0.0092|
|
| 48 |
+
|
| 49 |
+
|
| 50 |
## Training procedure
|
| 51 |
|
| 52 |
### Training hyperparameters
|