Spaces:
Running
on
CPU Upgrade
Result PRs not appearing
Hello! I'm seeing some merge authors getting result PRs and having their models listed, but I haven't seen any in over a week. I know this is a new version - any new critical field in our model cards for us to look after? Any permissions?
Hi! The PRs were opened by a tool that
@Weyaxi
kindly made for the leaderboard, and which is not maintained by the leaderboard authors. It is possible that the tool is not up to date.
@Weyaxi
if the code of your tool is open, maybe members of the community could give you a hand to put it up to date?
Hi @sometimesanotion and @clefourrier ,
For some reason, the following code wasn't working in both spaces for opening PRs:
pd.DataFrame(load_dataset("open-llm-leaderboard/contents", split="train").sort("Average ⬆️", reverse=True))
The strange part is that it works in various environments like Colab or locally.
As a workaround, I had to replace this with the pd.read_parquet function, which ideally shouldn't be preferred:
pd.read_parquet("https://huggingface.co/datasets/open-llm-leaderboard/contents/resolve/main/data/train-00000-of-00001.parquet").sort_values(by="Average ⬆️", ascending=False)
This issue appears to be resolved now, and new model PRs should be opened whenever a new model is added to the leaderboard!
I will track this issue and confirm it is working!
Best,
Weyaxi
Thanks a lot! :)