Is leaderboard submission currently available?

#104
by wonhosong - opened

Can I submit my model to the leaderboard now?
Currently, the leaderboard does not appear to be updating for two reasons:

  1. backend system improvements.
  2. re-running all models due to MMLU re-evaluation.

I'm wondering when I can start submitting to the leaderboard.

Open LLM Leaderboard org

Hi! Yes, feel free to submit your model! The leaderboard is not being updated but we still store the queue, and we'll launch all new models quite soon!

@clefourrier Thanks for answering! Can you at least give us a sense of when the new model might be evaluated?

Open LLM Leaderboard org

Tbh we are really trying to push the new update today or tomorrow, we're in the final testing phases - then we'll launch all the new models' training

Thank you for your answers and for running the leaderboard.

Open LLM Leaderboard org
edited Jul 13, 2023

Hi @win7785 ! Leaderboard was updated (here is the announcement), check if your model is in the queue - if not, please resubmit, we fixed the submission system :)

@clefourrier I can see my submission on pending :) Thank you so much the quick update.

SaylorTwift changed discussion status to closed

Sign up or log in to comment