Spaces:
Running
on
CPU Upgrade
Running
on
CPU Upgrade
Disappointed in the reliability/speed
#330
by
Dampfinchen
- opened
Hello,
while I do like the LLM leaderboard, I don't think it's very reliable. Sometimes (like right now) the whole thing stops for weeks without an apparent reason. The models are just stuck in the pending evaluation queue. Right now its at 69 but I've seen numbers as high as 200 in the past.
What's the matter with that? I would like for this to be fixed. Some people don't have the hardware power to evaluate models, so LLM leaderboard is a nice way to get models evaluated and compare the performance... if it would work correctly.
Please fix, thank you!
Hi, like stated in a few other disscusions, we are preparing an update on the leaderboard. For this reason, we stopped evaluating models until the end of October. Thanks for your patience.
SaylorTwift
changed discussion status to
closed