Spaces:
Running
on
CPU Upgrade
Running
on
CPU Upgrade
Any updates on redesigning the leaderboard?
#595
by
TNTOutburst
- opened
I thought around December or so, there were a lot of conversations on ways to redesign how the leaderboard works. I was wondering if you have any updates on what you are working on or if you are still looking for solutions.
Hi!
To list all the topics I remember:
- a number of these conversations were about adding new leaderboards (for example, with rolling updates or private benchmarks), not necessarily changing the Open LLM Leaderboard itself > you can find some of the new leaderboards that were created with partners here.
- wrt contamination, we've had to put these conversations on hold while we were focusing on the release of our mini evaluation suite, lighteval. We'll come back to them as soon as we can!
- we've also increased the efforts on metadata (things like requesting licenses, adding flags for moerges, ...).
We're also looking to hire an intern, because we are stretched quite thin (but hiring takes a lot of time) - once this is done, we'll likely come back in full force on some of these topics :)
clefourrier
changed discussion status to
closed