Unable to submit Qwen based models.

#427
by xxyyy123 - opened

When I try to submit falcon Qwen model via clicking Submit Eval button, it says

Model "~" needs to be launched with trust_remote_code=True.
For safety reason, we do not allow these models to be automatically submitted to the leaderboard.
It seems this problem has been addressed here and here.

model:https://huggingface.co/AIDC-ai-business/Marcoroni-14B-v2

I would like to ask if you could assist us with conducting an evaluation in the background.

Open LLM Leaderboard org

Hi!
We don't allow models which require trust_remote_code=True to load to be automatically submitted to the leaderboard (as indicated in the message you got and the FAQ in the About tab).
Are you only using model classes that have been integrated in the transformers library ?

xxyyy123 changed discussion status to closed

Sign up or log in to comment