GSM8K contamination
#2
by
Cebtenzzre
- opened
This model doesn't belong on the Open LLM Leaderboard because it has Una-xaberius-34b-v1beta merged in which has been found to be contaminated with GSM8K. See https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard/discussions/444#657abfcca0bd89c1ee8d7861
Yeah I am going to leave it out on the next merge, I don't think 4K models help with the long context coherence anyway.
Still, the Xaberius trainer has a point. I'm pretty sure base Yi itself is contaminated... And I'm not trying to top the leaderboard or anything, just using it as one datapoint of many between merges :P