Leaderboard isn't updating its model list.

#809
by nlpguy - opened

As mentioned in another thread, the leaderboard doesn't update, but unlike what I assumed, a restart didn't fix it. Here is an example model that isn't shown on the leaderboard despite being evaluated:

https://huggingface.co/datasets/open-llm-leaderboard/openchat__openchat-3.6-8b-20240522-details

And on an unrelated note, commit 9b8c53c broke model hiding, as it hides everything but merges when enabling the hide merges option.

Open LLM Leaderboard org
edited 3 days ago

The commit is completely unrelated ^^

The restart is different from the update, as we have a specific parser space which goes from all the result files to the leaderboard contents - we are therefore doing the time consuming step elsewhere, which allows to have a restart in a couple minutes. It's in the parser space that we're having an issue with the model hiding btw, we were discussing it

The Model I mentioned is now displayed, closing for now.

nlpguy changed discussion status to closed

Sign up or log in to comment