Spaces:
Running
on
CPU Upgrade
About GSM8K Scores
Hi @clefourrier @SaylorTwift ,
Is there an update happening with GSM8K scores? I noticed that the results of some models are being updated. Additionally, the GSM8K score of these updated models are significantly increasing.
Could you please clarify things? Thanks.
Hi @Weyaxi , thanks to the community, we found a bug affecting some models on GSM8K (about 150) and we re-ran GSM8K on those models, this bug made score much lower that they should have been. We will keep everyone informed once we finish re-evaluating the models !
Also, you are so fast at noticing changes in the leaderboard 😅
Hi @Weyaxi , thanks to the community, we found a bug affecting some models on GSM8K (about 150) and we re-ran GSM8K on those models, this bug made score much lower that they should have been. We will keep everyone informed once we finish re-evaluating the models !
Hi, thanks for clarification :)
Also, you are so fast at noticing changes in the leaderboard 😅
Haha, just a lucky coincidence! Thanks! 😄