Spaces:
Running
on
CPU Upgrade
Running
on
CPU Upgrade
lm-eval-harness commit link appears incorrect
#10
by
dmayhem93
- opened
Under the reproducibility section, there's a link to this commit of the lm-eval-harness: https://github.com/EleutherAI/lm-evaluation-harness/tree/b281b0921b636bc36ad05c0b0b0763bd6dd43463
But I can't find a few of the tasks in this commit (e.g. ifeval)
@dmayhem93
thanks! Indeed, we incorporated most of the tasks we are using into the harness by ourselves.
Our task definitions files are available here: https://huggingface.co/spaces/hallucinations-leaderboard/leaderboard/tree/main/src/backend/tasks (any feedback on these is more than welcome!)
Thanks! I'll give it a try on the main branch
dmayhem93
changed discussion status to
closed