Spaces:

open-llm-leaderboard

/

open_llm_leaderboard

like 12.1k

Running on CPU Upgrade

Hi! Thanks for the report, this is a bug I believe - all chat models are launched with fewshot as multiturn, but gemma had to be relaunched entirely to include a fix in the harness regarding token management - however, the new results should have overcome the old ones. I'll take a look asap

Hi!
I'm not finding both these results in the leaderboard so I'm going to assume it was a cache problem.
Tell me if you still have the issue.