| wandb run: https://wandb.ai/eleutherai/pythia-rlhf/runs/e0drjcsz?workspace=user-yongzx | |
| Model Evals: | |
| | Task |Version|Filter| Metric |Value | |Stderr| | |
| |-------------|-------|------|--------|-----:|---|-----:| | |
| |arc_challenge|Yaml |none |acc |0.1877|± |0.0114| | |
| | | |none |acc_norm|0.2372|± |0.0124| | |
| |arc_easy |Yaml |none |acc |0.4390|± |0.0102| | |
| | | |none |acc_norm|0.4082|± |0.0101| | |
| |logiqa |Yaml |none |acc |0.1889|± |0.0154| | |
| | | |none |acc_norm|0.2473|± |0.0169| | |
| |piqa |Yaml |none |acc |0.6213|± |0.0113| | |
| | | |none |acc_norm|0.6279|± |0.0113| | |
| |sciq |Yaml |none |acc |0.7230|± |0.0142| | |
| | | |none |acc_norm|0.6840|± |0.0147| | |
| |winogrande |Yaml |none |acc |0.5162|± |0.0140| | |
| |wsc |Yaml |none |acc |0.3654|± |0.0474| | |
| |lambada_openai|Yaml |none |perplexity|58.9478|± |2.7662| | |
| | | |none |acc | 0.2602|± |0.0061| | |