open-llm-leaderboard/open_llm_leaderboard · Delete broken model from the HF leaderboard results and re-run Mega-Destroyer-8x7B.

Mar 14

•

Hello,

we've recently released a model called Mega-Destroyer-8x7B and submitted it to the leaderboard. Unfortunately, the merge went wrong, so the results are understandably extremely low.

Now we've uploaded a new repo under the same name and because of that I can't submit it to the HF leaderboard as it's already been submitted.

Can you please delete the old result and re-run https://huggingface.co/FoxEngineAi/Mega-Destroyer-8x7B ?

https://huggingface.co/datasets/open-llm-leaderboard/requests/blob/main/FoxEngineAi/Mega-Destroyer-8x7B_eval_request_False_bfloat16_Original.json

This is the result from the broken merge in question. Thank you!

clefourrier

Open LLM Leaderboard org Mar 14

Hi!
You can always resubmit a new model with the new commit hash if needed, but I can also delete the request file if you want!

Dampfinchen

Mar 14

Hi!
You can always resubmit a new model with the new commit hash if needed, but I can also delete the request file if you want!

Yes, please do. Thank you!

clefourrier

Open LLM Leaderboard org Mar 14

Done! Feel free to resubmit :)

clefourrier changed discussion status to closed Mar 14

Dampfinchen

Mar 14

Done! Feel free to resubmit :)

It seems like it's not enough to delete the request file. The broken result is still visible in the Leaderboard and I'm still not able to submit the new repo under the same name.

clefourrier

Open LLM Leaderboard org Mar 14

Hi! Yes, you'll need to wait for the results to be propagated to the leaderboard, which should be the case after the next rebuild (there is a caching system)