latest commit breaks ability to submit mistral finetunes

#410
by winglian - opened

this commit https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard/commit/ead4c9640f6462903d8fb9579b6cb5f7ad0a71f7 seems to break the ability to submit mistral finetunes. The tokenizer that tried is the LlamaTokenizer which is pretty standard, but I get the error uses a tokenizer which is not in a transformers release

https://huggingface.co/openaccess-ai-collective/dpopenhermes-v1/blob/main/tokenizer_config.json#L56

Open LLM Leaderboard org

Hi!
Thank you for your report.
Can you please provide the full error message that you got?

Here you go. I've verified the tokenizer loads properly with the autotokenizer
Screenshot 2023-12-01 at 12.33.11 PM.png

Screenshot 2023-12-01 at 12.34.17 PM.png

Open LLM Leaderboard org

Super useful thank you! We were simply missing sentencepiece as a dependency in the front end, I added it - it should be good now, can you give it another go?

thanks for the fix!

winglian changed discussion status to closed

Sign up or log in to comment