🚩 Report
#1
by
CoreyMorris
- opened
Suspected to be trained on MMLU data. If not, it is fantastic performance and congrats :)
whoops. meant to open this on the open-llm leaderboard. closing.
CoreyMorris
changed discussion status to
closed
CoreyMorris
changed discussion status to
open
There has been an effort to remove models from the open-llm-leaderboard that have evaluation data in their training data. See here https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard/discussions/215 . Another user mentioned that this model config shows it being based on Voicelab/trurl-2-13b . That model was trained on evaluation data so that's probably the issue here. Can you confirm if this model is based on Voicelab/trurl-2-13b?
yes it's a derivative. wasn't aware of the trurl contam