Update README.md
Browse files
README.md
CHANGED
|
@@ -19,3 +19,27 @@ Fully reproducible finetuning code is available on [GitHub](https://github.com/l
|
|
| 19 |
[wandb log](https://wandb.ai/lauraomahony999/pythia-dpo/runs/blurtl4v)
|
| 20 |
|
| 21 |
See [Pythia-2.8b](https://huggingface.co/EleutherAI/pythia-2.8b) for model details [(paper)](https://arxiv.org/abs/2101.00027).
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 19 |
[wandb log](https://wandb.ai/lauraomahony999/pythia-dpo/runs/blurtl4v)
|
| 20 |
|
| 21 |
See [Pythia-2.8b](https://huggingface.co/EleutherAI/pythia-2.8b) for model details [(paper)](https://arxiv.org/abs/2101.00027).
|
| 22 |
+
|
| 23 |
+
hf (pretrained=lomahony/pythia-2.8b-helpful-dpo), gen_kwargs: (None), limit: None, num_fewshot: 0, batch_size: 16
|
| 24 |
+
| Tasks |Version|Filter|n-shot| Metric | Value | |Stderr|
|
| 25 |
+
|--------------|------:|------|-----:|---------------|------:|---|------|
|
| 26 |
+
|arc_challenge | 1|none | 0|acc | 0.3157|± |0.0136|
|
| 27 |
+
| | |none | 0|acc_norm | 0.3447|± |0.0139|
|
| 28 |
+
|arc_easy | 1|none | 0|acc | 0.6591|± |0.0097|
|
| 29 |
+
| | |none | 0|acc_norm | 0.6002|± |0.0101|
|
| 30 |
+
|boolq | 2|none | 0|acc | 0.6239|± |0.0085|
|
| 31 |
+
|hellaswag | 1|none | 0|acc | 0.4671|± |0.0050|
|
| 32 |
+
| | |none | 0|acc_norm | 0.6107|± |0.0049|
|
| 33 |
+
|lambada_openai| 1|none | 0|perplexity | 4.8811|± |0.1354|
|
| 34 |
+
| | |none | 0|acc | 0.6264|± |0.0067|
|
| 35 |
+
|openbookqa | 1|none | 0|acc | 0.2820|± |0.0201|
|
| 36 |
+
| | |none | 0|acc_norm | 0.4040|± |0.0220|
|
| 37 |
+
|piqa | 1|none | 0|acc | 0.7568|± |0.0100|
|
| 38 |
+
| | |none | 0|acc_norm | 0.7557|± |0.0100|
|
| 39 |
+
|sciq | 1|none | 0|acc | 0.8900|± |0.0099|
|
| 40 |
+
| | |none | 0|acc_norm | 0.8340|± |0.0118|
|
| 41 |
+
|wikitext | 2|none | 0|word_perplexity|13.9186|± |N/A |
|
| 42 |
+
| | |none | 0|byte_perplexity| 1.6363|± |N/A |
|
| 43 |
+
| | |none | 0|bits_per_byte | 0.7104|± |N/A |
|
| 44 |
+
|winogrande | 1|none | 0|acc | 0.6046|± |0.0137|
|
| 45 |
+
|