Repetition Improves Language Model Embeddings
Please refer to our paper: https://arxiv.org/abs/2402.15449
And our GitHub: https://github.com/jakespringer/echo-embeddings
We provide a description of the model as well as example usage in the above links.
- Downloads last month
- 2,811
This model does not have enough activity to be deployed to Inference API (serverless) yet.
Increase its social visibility and check back later, or deploy to Inference Endpoints (dedicated)
instead.
Space using jspringer/echo-mistral-7b-instruct-lasttoken 1
Evaluation results
- accuracy on MTEB AmazonCounterfactualClassification (en)test set self-reported82.970
- ap on MTEB AmazonCounterfactualClassification (en)test set self-reported49.629
- f1 on MTEB AmazonCounterfactualClassification (en)test set self-reported77.590
- accuracy on MTEB AmazonPolarityClassificationtest set self-reported90.975
- ap on MTEB AmazonPolarityClassificationtest set self-reported87.573
- f1 on MTEB AmazonPolarityClassificationtest set self-reported90.967
- accuracy on MTEB AmazonReviewsClassification (en)test set self-reported48.708
- f1 on MTEB AmazonReviewsClassification (en)test set self-reported47.736
- map_at_1 on MTEB ArguAnatest set self-reported32.006
- map_at_10 on MTEB ArguAnatest set self-reported49.268