Spaces:
Running
on
CPU Upgrade
Feature Request: Multilingual Evaluations 🌐
I've run into a bit of a problem evaluating certain open models in French 😕.
While initiatives like eduagarcia/open_pt_llm_leaderboard and upstage/open-ko-llm-leaderboard have forked the repository to evaluate LLMs in their own language (Portuguese and Korean), it would be so much better if this functionality was built right into this space 🙌. After all, the international community is starting to adopt the best models for their own languages locally! 😊
I really hope this message gets noticed and that you consider implementing this feature. 🤞
Hi @eliot-christon ,
Thank you for your suggestion and for bringing this up! We appreciate your enthusiasm for expanding the evaluation capabilities to other languages like French.
Currently, the focus of the Open LLM Leaderboard is on evaluating models in English. However, we recognize the importance of multilingual support and would be happy to collaborate with the community to create a French LLM Leaderboard. If you're interested in contributing to this effort, we'd be more than willing to assist you in getting it started.
Feel free to reach out if you have any questions or need any guidance on this 👍