Spaces:
Running
on
CPU Upgrade
[Issue] Skipped evaluation
Hi,
due to recent update of evaluation rule, there are some issues.
At 10:43 pm on Oct 27th, 56 models are in status of finishing evaluation.
But only 27 models are on the board now.
Hope you check the model which didn't get an evaluation but were treated as finished one.
I think there is an error related to the submission date.
Thank you!
Btw, my model is jyoung105/KoR-Orca-Platypus-13B-neft.
Hello.
The above FINISHED status is related to the open-ko-llm-leaderboard/requests repository. The leaderboard reflects based on whether status key of the ...json file is marked as PENDING/RUNNING/FINISHED.
Therefore, we need to change it to PENDING. However, due to commit limitations, we are currently updating them to PENDING in the order they were submitted.
I think we can all change it to PENDING within tomorrow.
If you see anything unusual, please let me know. Thank you.
[Updated]
Oh! Fortunately, everything has been changed to PENDING without locked the commit limit. Now, you should be able to see it correctly displayed on the leaderboard.
Thank you.
First of all, thanks to super fast reply.
Even I'm now in the other team, Smoretalk, I'm always impressed with the passion of Upstage.
Hope you not mind to give this kind of feedback.
And also, if there are some help you need, always ask it to tonylee@smoretalk.io
I will do my best, too to contribute to this amazing square for open source.
Thank you once again.
Thanks