Add Evaluation Results (#1)
Browse files- Add Evaluation Results (3df871a5124e16645ba8d33c7be9f815627e8487)
- Add Lambada OpenAI to evals (fbf81efd690417bdce94e62a4874d0a324c9d738)
Co-authored-by: USVSN Sai Prashanth <usvsnsp@users.noreply.huggingface.co>
README.md
CHANGED
|
@@ -1 +1,19 @@
|
|
| 1 |
-
Wandb runs: https://wandb.ai/eleutherai/pythia-rlhf/runs/s0qdwbg6?workspace=user-yongzx
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
Wandb runs: https://wandb.ai/eleutherai/pythia-rlhf/runs/s0qdwbg6?workspace=user-yongzx
|
| 2 |
+
|
| 3 |
+
|
| 4 |
+
Evaluation results:
|
| 5 |
+
| Task |Version|Filter| Metric |Value | |Stderr|
|
| 6 |
+
|-------------|-------|------|--------|-----:|---|-----:|
|
| 7 |
+
|arc_challenge|Yaml |none |acc |0.1758|± |0.0111|
|
| 8 |
+
| | |none |acc_norm|0.2176|± |0.0121|
|
| 9 |
+
|arc_easy |Yaml |none |acc |0.3742|± |0.0099|
|
| 10 |
+
| | |none |acc_norm|0.3565|± |0.0098|
|
| 11 |
+
|logiqa |Yaml |none |acc |0.2058|± |0.0159|
|
| 12 |
+
| | |none |acc_norm|0.2412|± |0.0168|
|
| 13 |
+
|piqa |Yaml |none |acc |0.5958|± |0.0114|
|
| 14 |
+
| | |none |acc_norm|0.5941|± |0.0115|
|
| 15 |
+
|sciq |Yaml |none |acc |0.5930|± |0.0155|
|
| 16 |
+
| | |none |acc_norm|0.5720|± |0.0157|
|
| 17 |
+
|winogrande |Yaml |none |acc |0.5154|± |0.0140|
|
| 18 |
+
|lambada_openai|Yaml |none |perplexity|730.2552|± |46.8739|
|
| 19 |
+
| | |none |acc | 0.1316|± | 0.0047|
|