Repetition Improves Language Model Embeddings
Please refer to our paper: https://arxiv.org/abs/2402.15449
And our GitHub: https://github.com/jakespringer/echo-embeddings
We provide a description of the model as well as example usage in the above links.
- Downloads last month
- 5,254
Evaluation results
- accuracy on MTEB AmazonCounterfactualClassification (en)test set self-reported82.970
- ap on MTEB AmazonCounterfactualClassification (en)test set self-reported49.629
- f1 on MTEB AmazonCounterfactualClassification (en)test set self-reported77.590
- accuracy on MTEB AmazonPolarityClassificationtest set self-reported90.975
- ap on MTEB AmazonPolarityClassificationtest set self-reported87.573
- f1 on MTEB AmazonPolarityClassificationtest set self-reported90.967
- accuracy on MTEB AmazonReviewsClassification (en)test set self-reported48.708
- f1 on MTEB AmazonReviewsClassification (en)test set self-reported47.736
- map_at_1 on MTEB ArguAnatest set self-reported32.006
- map_at_10 on MTEB ArguAnatest set self-reported49.268
- map_at_100 on MTEB ArguAnatest set self-reported49.904
- map_at_1000 on MTEB ArguAnatest set self-reported49.909
- map_at_3 on MTEB ArguAnatest set self-reported44.334
- map_at_5 on MTEB ArguAnatest set self-reported47.374
- mrr_at_1 on MTEB ArguAnatest set self-reported32.788
- mrr_at_10 on MTEB ArguAnatest set self-reported49.707
- mrr_at_100 on MTEB ArguAnatest set self-reported50.347
- mrr_at_1000 on MTEB ArguAnatest set self-reported50.352
- mrr_at_3 on MTEB ArguAnatest set self-reported44.950
- mrr_at_5 on MTEB ArguAnatest set self-reported47.767
- ndcg_at_1 on MTEB ArguAnatest set self-reported32.006
- ndcg_at_10 on MTEB ArguAnatest set self-reported58.523
- ndcg_at_100 on MTEB ArguAnatest set self-reported61.095
- ndcg_at_1000 on MTEB ArguAnatest set self-reported61.191
- ndcg_at_3 on MTEB ArguAnatest set self-reported48.431
- ndcg_at_5 on MTEB ArguAnatest set self-reported53.940
- precision_at_1 on MTEB ArguAnatest set self-reported32.006
- precision_at_10 on MTEB ArguAnatest set self-reported8.791
- precision_at_100 on MTEB ArguAnatest set self-reported0.989
- precision_at_1000 on MTEB ArguAnatest set self-reported0.100
- precision_at_3 on MTEB ArguAnatest set self-reported20.104
- precision_at_5 on MTEB ArguAnatest set self-reported14.751
- recall_at_1 on MTEB ArguAnatest set self-reported32.006
- recall_at_10 on MTEB ArguAnatest set self-reported87.909
- recall_at_100 on MTEB ArguAnatest set self-reported98.862
- recall_at_1000 on MTEB ArguAnatest set self-reported99.573
- recall_at_3 on MTEB ArguAnatest set self-reported60.313
- recall_at_5 on MTEB ArguAnatest set self-reported73.755
- v_measure on MTEB ArxivClusteringP2Ptest set self-reported47.015
- v_measure on MTEB ArxivClusteringS2Stest set self-reported43.522
- map on MTEB AskUbuntuDupQuestionstest set self-reported64.135
- mrr on MTEB AskUbuntuDupQuestionstest set self-reported76.938
- cos_sim_pearson on MTEB BIOSSEStest set self-reported87.832
- cos_sim_spearman on MTEB BIOSSEStest set self-reported86.538
- euclidean_pearson on MTEB BIOSSEStest set self-reported86.144
- euclidean_spearman on MTEB BIOSSEStest set self-reported86.708
- manhattan_pearson on MTEB BIOSSEStest set self-reported86.121
- manhattan_spearman on MTEB BIOSSEStest set self-reported86.470
- accuracy on MTEB Banking77Classificationtest set self-reported88.146
- f1 on MTEB Banking77Classificationtest set self-reported88.099
- v_measure on MTEB BiorxivClusteringP2Ptest set self-reported35.530