Spaces:
Running
on
CPU Upgrade
New SOTA! Apply for refreshing the results
Hi MTEB team,
We are releasing an embedding model SFR-Embedding-Mistral achieving SOTA on the MTEB benchmark with an average score of 67.56.
Could you please help refresh the leaderboard? Thank you!
Best,
Rui
Also I wonder how to add other meta information such as dimension and length?
- Model Size: 14.22
- Embedding Dimensions: 4096
- Sequence Length: 32768
Thank you!
Hey it will automatically add that metadata once you upload the actual model - do you plan on uploading it?
@Muennighoff yes, we are working on the model uploading. But I don't think it will be available very soon. Would you help us update the leaderboard first? Thank you!
Sounds great & congrats on the performance. If you're not opposed, I recommend we update the LB once the model is released, as it will result in a much better community response.
Currently, users will not be able to do anything once they click on your model. Thus, I think it is better to update the LB once there's a model/api + usage instructions 🙌
I agree as well with @Muennighoff , it's weird to have models that are not released in the leaderboard.
Done, congrats!
Hi @Muennighoff ,
Could you please refresh the LB? We updated the model from 32-bit to 16-bit, resulting in a change in the model size. Thank you!
Done :)