Spaces:

open-llm-leaderboard

/

open_llm_leaderboard

like 10.2k

Running on CPU Upgrade

same same
It didnt get added to the leaderboard and now says I already submitted when trying to resubmit:
https://huggingface.co/teknium/CollectiveCognition-v1.1-Mistral-7B
https://huggingface.co/teknium/CollectiveCognition-v1-Mistral-7B
https://huggingface.co/teknium/Mistral-Trismegistus-7B

Same for me too:

• https://huggingface.co/datasets/open-llm-leaderboard/requests/blob/main/Weyaxi/OpenOrca-Nebula-7B_eval_request_False_float16_Original.json

• https://huggingface.co/datasets/open-llm-leaderboard/requests/blob/main/Weyaxi/TekniumAiroboros-Nebula-7B_eval_request_False_float16_Original.json

• https://huggingface.co/datasets/open-llm-leaderboard/requests/blob/main/Weyaxi/Dolphin-Nebula-7B_eval_request_False_float16_Original.json

Hi ! I must have missed your thread, I will set all the models here to PENDING and you should quickly see it appear in the leaderboard.

Thanks @SaylorTwift for reply, Just following up on this, How long usually this should take? I can see in requests that it shows PENDING as of 2023-10-16 02:08 UTC and it was submitted at 2023-10-04T13:29

https://huggingface.co/datasets/open-llm-leaderboard/requests/raw/main/pankajmathur/Mistral-7B-model_45k6e2e4_eval_request_False_bfloat16_Original.json

Thanks again for your help on this.

As explained here, we are currently running upgrades to the leaderboard. We expect the queue to be restarted at the end of the month.

I know @SaylorTwift @clefourrier you guys are hard at working new version of LB and I want to say it's fantastic work by adding these new 3 benchmarks. So, Kudos.

Today, i am reaching out again to see, if this model which I submitted for evaluation on Oct 2nd 2023 can be resubmitted for evaluation.

https://huggingface.co/pankajmathur/Mistral-7B-model_45k6e2e4

Thanks again for your handwork on new LB upgrades.

Hi @pankajmathur ,
Thank you for following the leaderboard :)

Why do you need to resubmit it, did it fail? What is the status in its request file?

My hermes 2 and hermes 2.5 models also are gone from the leaderbooard fwiw ;]

Hi @teknium ! Can you provide a link to the request files so we can get more information?

Closing for inactivity, feel free to reopen if needed :)

@clefourrier : Apologies for delayed reply. The model https://huggingface.co/pankajmathur/Mistral-7B-model_45k6e2e4 actually is present in finished evaluation (see screenshot), also shows in the request same thing, https://huggingface.co/datasets/open-llm-leaderboard/requests/blob/main/pankajmathur/Mistral-7B-model_45k6e2e4_eval_request_False_bfloat16_Original.json
However the details (results) page are throwing 404
https://huggingface.co/datasets/open-llm-leaderboard/details_pankajmathur__Mistral-7B-model_45k6e2e4
So really not sure what's going on.

Hi!
It should be good now :)

For future reference, some of our details dataset were published with _public at the end of the name (I haven't had the time to fix it yet), so if you don't find a details dataset, you can try to see if one with _public exists.