What tags need to be existing in a model card to be detected as merger?

#637
by ThiloteE - opened

Currently, there are many models not being detected as merger.

This is just an example and just a substitute for a phenomenon that affects many more models:

Detected as merger: https://huggingface.co/Gille/StrangeMerges_32-7B-slerp
image.png

Not detected as merger: https://huggingface.co/Gille/StrangeMerges_29-7B-dare_ties
image.png

What exactly is the difference?
Is your script faulty? Please update your script or if it is not run automatically, please do let it run again.
Or is it caused by mergekit not adding the correct tags? Does it need to be patched in a way that it automatically adds tags to model cards upon uploading to the open_llm_leaderboard? LazyMergekit by MaximeLabonne seems to be a very influential resource that also many non-specialists use to merge models, so adding tags should by default be done automatically and only with manual intervention be circumvented.

Open LLM Leaderboard org

Hi!
These tags seem correct!
I'll check - if they were added post submission, we could have missed the information (there is a script running, mostly when a model is submitted - it is also normally rerun every x days on all models)

Open LLM Leaderboard org

In general, you'll find detailed model information (notably tags) in this file.
Seems like the tags are correctly detected for both models now, so I'll close, feel free to reopen if needed. (It might take a couple hours to propagate back to the leaderboard)

clefourrier changed discussion status to closed

Sign up or log in to comment