Customizable Model name?

#92
by vaibhavad - opened
Massive Text Embedding Benchmark org

Currently it seems like the model name in the leaderboard is taken from the huggingface identifier. However, in model-index YAML, these is a name attribute. Is that used anywhere? Is it possible to use that for leaderboard model name?

E.g. our model has HF identifier LLM2Vec-Mistral-7B-Instruct-v2-mntp-supervised. but the MTEB README file starts with

model-index:
- name: LLM2Vec-Mistral-7B-supervised
Massive Text Embedding Benchmark org

Yeah if the results are integrated via the results repo instead i.e. https://huggingface.co/datasets/mteb/results everything is customizable. If you want, you can open a PR there with your mteb result files :)

vaibhavad changed discussion status to closed

Sign up or log in to comment