Spaces:
Running
on
CPU Upgrade
Dataset for models confirmed to have training data contaminated with evaluation data
This would be helpful for folks building off of the hugging face evaluation data https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard/discussions/174 . For now, manual removal is easy because there are just handful, but it would be nice to not have to track this and just have another dataset I can pull from for which models I should remove. If this wouldn't fit well into the hugging face workflow or they do not want to do it for some reason, I can upload a dataset.
Hi
@CoreyMorris
!
We won't go for an automatic process because it could allow people to flag models for the wrong reasons (ex: flag the model of a competitor).
However, we store the list manually here, so every discussion opened will lead to an addition in the file.
Gotcha. That list works. I'll go ahead and close this issue.