diff --git "a/README.md" "b/README.md" --- "a/README.md" +++ "b/README.md" @@ -1,51 +1,4217 @@ ---- -pipeline_tag: sentence-similarity -language: en -license: apache-2.0 -tags: -- sentence-transformers -- feature-extraction -- sentence-similarity -- transformers ---- - -# sentence-transformers/sentence-t5-base - -This is a [sentence-transformers](https://www.SBERT.net) model: It maps sentences & paragraphs to a 768 dimensional dense vector space. The model works well for sentence similarity tasks, but doesn't perform that well for semantic search tasks. - -This model was converted from the Tensorflow model [st5-base-1](https://tfhub.dev/google/sentence-t5/st5-base/1) to PyTorch. When using this model, have a look at the publication: [Sentence-T5: Scalable sentence encoders from pre-trained text-to-text models](https://arxiv.org/abs/2108.08877). The tfhub model and this PyTorch model can produce slightly different embeddings, however, when run on the same benchmarks, they produce identical results. - -The model uses only the encoder from a T5-base model. The weights are stored in FP16. - - -## Usage (Sentence-Transformers) - -Using this model becomes easy when you have [sentence-transformers](https://www.SBERT.net) installed: - -``` -pip install -U sentence-transformers -``` - -Then you can use the model like this: - -```python -from sentence_transformers import SentenceTransformer -sentences = ["This is an example sentence", "Each sentence is converted"] - -model = SentenceTransformer('sentence-transformers/sentence-t5-base') -embeddings = model.encode(sentences) -print(embeddings) -``` - -The model requires sentence-transformers version 2.2.0 or newer. - -## Evaluation Results - -For an automated evaluation of this model, see the *Sentence Embeddings Benchmark*: [https://seb.sbert.net](https://seb.sbert.net?model_name=sentence-transformers/sentence-t5-base) - - - -## Citing & Authors - -If you find this model helpful, please cite the respective publication: -[Sentence-T5: Scalable sentence encoders from pre-trained text-to-text models](https://arxiv.org/abs/2108.08877) +--- +pipeline_tag: sentence-similarity +language: en +license: apache-2.0 +tags: +- sentence-transformers +- feature-extraction +- sentence-similarity +- transformers +- mteb +model-index: +- name: sentence-t5-base + results: + - task: + type: Classification + dataset: + type: mteb/amazon_counterfactual + name: MTEB AmazonCounterfactualClassification (en) + config: en + split: test + metrics: + - type: accuracy + value: 75.82089552238807 + - type: ap + value: 40.58809426967639 + - type: f1 + value: 70.5050115572668 + - task: + type: Classification + dataset: + type: mteb/amazon_counterfactual + name: MTEB AmazonCounterfactualClassification (de) + config: de + split: test + metrics: + - type: accuracy + value: 69.97858672376874 + - type: ap + value: 80.89622545806847 + - type: f1 + value: 68.09770164363411 + - task: + type: Classification + dataset: + type: mteb/amazon_counterfactual + name: MTEB AmazonCounterfactualClassification (en-ext) + config: en-ext + split: test + metrics: + - type: accuracy + value: 76.80659670164917 + - type: ap + value: 26.663544686227127 + - type: f1 + value: 64.52406535274052 + - task: + type: Classification + dataset: + type: mteb/amazon_counterfactual + name: MTEB AmazonCounterfactualClassification (ja) + config: ja + split: test + metrics: + - type: accuracy + value: 46.04925053533191 + - type: ap + value: 10.574096802771448 + - type: f1 + value: 36.74441737116304 + - task: + type: Classification + dataset: + type: mteb/amazon_polarity + name: MTEB AmazonPolarityClassification + config: default + split: test + metrics: + - type: accuracy + value: 85.11737500000001 + - type: ap + value: 81.28435308927632 + - type: f1 + value: 85.01612484917347 + - task: + type: Classification + dataset: + type: mteb/amazon_reviews_multi + name: MTEB AmazonReviewsClassification (en) + config: en + split: test + metrics: + - type: accuracy + value: 44.943999999999996 + - type: f1 + value: 42.681783855948844 + - task: + type: Classification + dataset: + type: mteb/amazon_reviews_multi + name: MTEB AmazonReviewsClassification (de) + config: de + split: test + metrics: + - type: accuracy + value: 37.895999999999994 + - type: f1 + value: 35.428429230946115 + - task: + type: Classification + dataset: + type: mteb/amazon_reviews_multi + name: MTEB AmazonReviewsClassification (es) + config: es + split: test + metrics: + - type: accuracy + value: 37.328 + - type: f1 + value: 34.26335456752553 + - task: + type: Classification + dataset: + type: mteb/amazon_reviews_multi + name: MTEB AmazonReviewsClassification (fr) + config: fr + split: test + metrics: + - type: accuracy + value: 37.35 + - type: f1 + value: 34.644931974230495 + - task: + type: Classification + dataset: + type: mteb/amazon_reviews_multi + name: MTEB AmazonReviewsClassification (ja) + config: ja + split: test + metrics: + - type: accuracy + value: 22.290000000000003 + - type: f1 + value: 20.438677904046305 + - task: + type: Classification + dataset: + type: mteb/amazon_reviews_multi + name: MTEB AmazonReviewsClassification (zh) + config: zh + split: test + metrics: + - type: accuracy + value: 21.529999999999998 + - type: f1 + value: 18.273004097867844 + - task: + type: Retrieval + dataset: + type: arguana + name: MTEB ArguAna + config: default + split: test + metrics: + - type: map_at_1 + value: 21.906 + - type: map_at_10 + value: 35.993 + - type: map_at_100 + value: 37.14 + - type: map_at_1000 + value: 37.153999999999996 + - type: map_at_3 + value: 30.642000000000003 + - type: map_at_5 + value: 33.534000000000006 + - type: ndcg_at_1 + value: 21.906 + - type: ndcg_at_10 + value: 44.846000000000004 + - type: ndcg_at_100 + value: 49.95 + - type: ndcg_at_1000 + value: 50.29 + - type: ndcg_at_3 + value: 33.579 + - type: ndcg_at_5 + value: 38.807 + - type: precision_at_1 + value: 21.906 + - type: precision_at_10 + value: 7.367999999999999 + - type: precision_at_100 + value: 0.966 + - type: precision_at_1000 + value: 0.099 + - type: precision_at_3 + value: 14.035 + - type: precision_at_5 + value: 10.967 + - type: recall_at_1 + value: 21.906 + - type: recall_at_10 + value: 73.68400000000001 + - type: recall_at_100 + value: 96.586 + - type: recall_at_1000 + value: 99.14699999999999 + - type: recall_at_3 + value: 42.105 + - type: recall_at_5 + value: 54.836 + - task: + type: Clustering + dataset: + type: mteb/arxiv-clustering-p2p + name: MTEB ArxivClusteringP2P + config: default + split: test + metrics: + - type: v_measure + value: 39.27529166223639 + - task: + type: Clustering + dataset: + type: mteb/arxiv-clustering-s2s + name: MTEB ArxivClusteringS2S + config: default + split: test + metrics: + - type: v_measure + value: 27.261128959373327 + - task: + type: Reranking + dataset: + type: mteb/askubuntudupquestions-reranking + name: MTEB AskUbuntuDupQuestions + config: default + split: test + metrics: + - type: map + value: 59.72875661091822 + - type: mrr + value: 72.76997317856043 + - task: + type: STS + dataset: + type: mteb/biosses-sts + name: MTEB BIOSSES + config: default + split: test + metrics: + - type: cos_sim_pearson + value: 75.50587493517146 + - type: cos_sim_spearman + value: 75.89088585182279 + - type: euclidean_pearson + value: 75.74627833999679 + - type: euclidean_spearman + value: 75.89088585182279 + - type: manhattan_pearson + value: 76.10746255262428 + - type: manhattan_spearman + value: 75.93968214440233 + - task: + type: Classification + dataset: + type: mteb/banking77 + name: MTEB Banking77Classification + config: default + split: test + metrics: + - type: accuracy + value: 76.47727272727273 + - type: f1 + value: 75.41900393828456 + - task: + type: Clustering + dataset: + type: mteb/biorxiv-clustering-p2p + name: MTEB BiorxivClusteringP2P + config: default + split: test + metrics: + - type: v_measure + value: 33.98533095653499 + - task: + type: Clustering + dataset: + type: mteb/biorxiv-clustering-s2s + name: MTEB BiorxivClusteringS2S + config: default + split: test + metrics: + - type: v_measure + value: 22.921149832439514 + - task: + type: Retrieval + dataset: + type: BeIR/cqadupstack + name: MTEB CQADupstackAndroidRetrieval + config: default + split: test + metrics: + - type: map_at_1 + value: 27.97 + - type: map_at_10 + value: 39.523 + - type: map_at_100 + value: 41.101 + - type: map_at_1000 + value: 41.221000000000004 + - type: map_at_3 + value: 36.193999999999996 + - type: map_at_5 + value: 37.952000000000005 + - type: ndcg_at_1 + value: 34.621 + - type: ndcg_at_10 + value: 46.18 + - type: ndcg_at_100 + value: 51.93600000000001 + - type: ndcg_at_1000 + value: 53.833 + - type: ndcg_at_3 + value: 41.091 + - type: ndcg_at_5 + value: 43.230000000000004 + - type: precision_at_1 + value: 34.621 + - type: precision_at_10 + value: 9.041 + - type: precision_at_100 + value: 1.525 + - type: precision_at_1000 + value: 0.19499999999999998 + - type: precision_at_3 + value: 20.029 + - type: precision_at_5 + value: 14.335 + - type: recall_at_1 + value: 27.97 + - type: recall_at_10 + value: 59.325 + - type: recall_at_100 + value: 82.917 + - type: recall_at_1000 + value: 95.175 + - type: recall_at_3 + value: 44.251000000000005 + - type: recall_at_5 + value: 50.383 + - task: + type: Retrieval + dataset: + type: BeIR/cqadupstack + name: MTEB CQADupstackEnglishRetrieval + config: default + split: test + metrics: + - type: map_at_1 + value: 23.701 + - type: map_at_10 + value: 32.094 + - type: map_at_100 + value: 33.293 + - type: map_at_1000 + value: 33.434999999999995 + - type: map_at_3 + value: 29.609999999999996 + - type: map_at_5 + value: 31.16 + - type: ndcg_at_1 + value: 30.573 + - type: ndcg_at_10 + value: 37.031 + - type: ndcg_at_100 + value: 42.001 + - type: ndcg_at_1000 + value: 44.714 + - type: ndcg_at_3 + value: 33.434999999999995 + - type: ndcg_at_5 + value: 35.356 + - type: precision_at_1 + value: 30.573 + - type: precision_at_10 + value: 6.854 + - type: precision_at_100 + value: 1.192 + - type: precision_at_1000 + value: 0.174 + - type: precision_at_3 + value: 16.178 + - type: precision_at_5 + value: 11.567 + - type: recall_at_1 + value: 23.701 + - type: recall_at_10 + value: 45.755 + - type: recall_at_100 + value: 67.035 + - type: recall_at_1000 + value: 84.893 + - type: recall_at_3 + value: 34.977999999999994 + - type: recall_at_5 + value: 40.357 + - task: + type: Retrieval + dataset: + type: BeIR/cqadupstack + name: MTEB CQADupstackGamingRetrieval + config: default + split: test + metrics: + - type: map_at_1 + value: 35.617 + - type: map_at_10 + value: 47.774 + - type: map_at_100 + value: 48.943999999999996 + - type: map_at_1000 + value: 49.007 + - type: map_at_3 + value: 44.214999999999996 + - type: map_at_5 + value: 46.291 + - type: ndcg_at_1 + value: 40.627 + - type: ndcg_at_10 + value: 53.952 + - type: ndcg_at_100 + value: 58.55200000000001 + - type: ndcg_at_1000 + value: 59.824 + - type: ndcg_at_3 + value: 47.911 + - type: ndcg_at_5 + value: 50.966 + - type: precision_at_1 + value: 40.627 + - type: precision_at_10 + value: 8.884 + - type: precision_at_100 + value: 1.213 + - type: precision_at_1000 + value: 0.13699999999999998 + - type: precision_at_3 + value: 21.337999999999997 + - type: precision_at_5 + value: 15.034 + - type: recall_at_1 + value: 35.617 + - type: recall_at_10 + value: 68.73599999999999 + - type: recall_at_100 + value: 88.42999999999999 + - type: recall_at_1000 + value: 97.455 + - type: recall_at_3 + value: 52.915 + - type: recall_at_5 + value: 60.182 + - task: + type: Retrieval + dataset: + type: BeIR/cqadupstack + name: MTEB CQADupstackGisRetrieval + config: default + split: test + metrics: + - type: map_at_1 + value: 21.029999999999998 + - type: map_at_10 + value: 27.915 + - type: map_at_100 + value: 28.924 + - type: map_at_1000 + value: 29.023 + - type: map_at_3 + value: 25.634 + - type: map_at_5 + value: 26.934 + - type: ndcg_at_1 + value: 22.599 + - type: ndcg_at_10 + value: 32.340999999999994 + - type: ndcg_at_100 + value: 37.422 + - type: ndcg_at_1000 + value: 40.014 + - type: ndcg_at_3 + value: 27.604 + - type: ndcg_at_5 + value: 29.872 + - type: precision_at_1 + value: 22.599 + - type: precision_at_10 + value: 5.051 + - type: precision_at_100 + value: 0.799 + - type: precision_at_1000 + value: 0.106 + - type: precision_at_3 + value: 11.562999999999999 + - type: precision_at_5 + value: 8.225999999999999 + - type: recall_at_1 + value: 21.029999999999998 + - type: recall_at_10 + value: 44.226 + - type: recall_at_100 + value: 67.902 + - type: recall_at_1000 + value: 87.497 + - type: recall_at_3 + value: 31.389 + - type: recall_at_5 + value: 36.888 + - task: + type: Retrieval + dataset: + type: BeIR/cqadupstack + name: MTEB CQADupstackMathematicaRetrieval + config: default + split: test + metrics: + - type: map_at_1 + value: 12.592 + - type: map_at_10 + value: 20.054 + - type: map_at_100 + value: 21.384 + - type: map_at_1000 + value: 21.52 + - type: map_at_3 + value: 17.718999999999998 + - type: map_at_5 + value: 19.189999999999998 + - type: ndcg_at_1 + value: 15.299 + - type: ndcg_at_10 + value: 24.698 + - type: ndcg_at_100 + value: 31.080000000000002 + - type: ndcg_at_1000 + value: 34.266000000000005 + - type: ndcg_at_3 + value: 20.331 + - type: ndcg_at_5 + value: 22.735 + - type: precision_at_1 + value: 15.299 + - type: precision_at_10 + value: 4.776 + - type: precision_at_100 + value: 0.928 + - type: precision_at_1000 + value: 0.133 + - type: precision_at_3 + value: 10.033 + - type: precision_at_5 + value: 7.761 + - type: recall_at_1 + value: 12.592 + - type: recall_at_10 + value: 35.386 + - type: recall_at_100 + value: 63.412 + - type: recall_at_1000 + value: 86.20400000000001 + - type: recall_at_3 + value: 23.768 + - type: recall_at_5 + value: 29.557 + - task: + type: Retrieval + dataset: + type: BeIR/cqadupstack + name: MTEB CQADupstackPhysicsRetrieval + config: default + split: test + metrics: + - type: map_at_1 + value: 23.549 + - type: map_at_10 + value: 32.875 + - type: map_at_100 + value: 34.247 + - type: map_at_1000 + value: 34.374 + - type: map_at_3 + value: 29.774 + - type: map_at_5 + value: 31.535000000000004 + - type: ndcg_at_1 + value: 28.874 + - type: ndcg_at_10 + value: 38.801 + - type: ndcg_at_100 + value: 44.727 + - type: ndcg_at_1000 + value: 47.154 + - type: ndcg_at_3 + value: 33.643 + - type: ndcg_at_5 + value: 36.046 + - type: precision_at_1 + value: 28.874 + - type: precision_at_10 + value: 7.305000000000001 + - type: precision_at_100 + value: 1.21 + - type: precision_at_1000 + value: 0.16199999999999998 + - type: precision_at_3 + value: 16.009 + - type: precision_at_5 + value: 11.741999999999999 + - type: recall_at_1 + value: 23.549 + - type: recall_at_10 + value: 51.15 + - type: recall_at_100 + value: 76.32900000000001 + - type: recall_at_1000 + value: 92.167 + - type: recall_at_3 + value: 36.544 + - type: recall_at_5 + value: 42.75 + - task: + type: Retrieval + dataset: + type: BeIR/cqadupstack + name: MTEB CQADupstackProgrammersRetrieval + config: default + split: test + metrics: + - type: map_at_1 + value: 24.524 + - type: map_at_10 + value: 34.288999999999994 + - type: map_at_100 + value: 35.67 + - type: map_at_1000 + value: 35.788 + - type: map_at_3 + value: 31.029 + - type: map_at_5 + value: 32.767 + - type: ndcg_at_1 + value: 29.794999999999998 + - type: ndcg_at_10 + value: 40.164 + - type: ndcg_at_100 + value: 46.278999999999996 + - type: ndcg_at_1000 + value: 48.698 + - type: ndcg_at_3 + value: 34.648 + - type: ndcg_at_5 + value: 36.982 + - type: precision_at_1 + value: 29.794999999999998 + - type: precision_at_10 + value: 7.580000000000001 + - type: precision_at_100 + value: 1.248 + - type: precision_at_1000 + value: 0.165 + - type: precision_at_3 + value: 16.628999999999998 + - type: precision_at_5 + value: 12.055 + - type: recall_at_1 + value: 24.524 + - type: recall_at_10 + value: 52.782 + - type: recall_at_100 + value: 79.108 + - type: recall_at_1000 + value: 95.62899999999999 + - type: recall_at_3 + value: 37.330999999999996 + - type: recall_at_5 + value: 43.502 + - task: + type: Retrieval + dataset: + type: BeIR/cqadupstack + name: MTEB CQADupstackRetrieval + config: default + split: test + metrics: + - type: map_at_1 + value: 21.669083333333333 + - type: map_at_10 + value: 30.095166666666668 + - type: map_at_100 + value: 31.35275 + - type: map_at_1000 + value: 31.476166666666668 + - type: map_at_3 + value: 27.41675 + - type: map_at_5 + value: 28.91216666666667 + - type: ndcg_at_1 + value: 25.666833333333333 + - type: ndcg_at_10 + value: 35.23175 + - type: ndcg_at_100 + value: 40.822833333333335 + - type: ndcg_at_1000 + value: 43.33783333333334 + - type: ndcg_at_3 + value: 30.516333333333336 + - type: ndcg_at_5 + value: 32.723 + - type: precision_at_1 + value: 25.666833333333333 + - type: precision_at_10 + value: 6.345583333333332 + - type: precision_at_100 + value: 1.0886666666666667 + - type: precision_at_1000 + value: 0.14974999999999997 + - type: precision_at_3 + value: 14.185583333333335 + - type: precision_at_5 + value: 10.265333333333334 + - type: recall_at_1 + value: 21.669083333333333 + - type: recall_at_10 + value: 46.69591666666667 + - type: recall_at_100 + value: 71.36999999999999 + - type: recall_at_1000 + value: 88.98216666666666 + - type: recall_at_3 + value: 33.59675 + - type: recall_at_5 + value: 39.2065 + - task: + type: Retrieval + dataset: + type: BeIR/cqadupstack + name: MTEB CQADupstackStatsRetrieval + config: default + split: test + metrics: + - type: map_at_1 + value: 18.587999999999997 + - type: map_at_10 + value: 25.452 + - type: map_at_100 + value: 26.296999999999997 + - type: map_at_1000 + value: 26.394000000000002 + - type: map_at_3 + value: 23.474 + - type: map_at_5 + value: 24.629 + - type: ndcg_at_1 + value: 21.012 + - type: ndcg_at_10 + value: 29.369 + - type: ndcg_at_100 + value: 33.782000000000004 + - type: ndcg_at_1000 + value: 36.406 + - type: ndcg_at_3 + value: 25.45 + - type: ndcg_at_5 + value: 27.384999999999998 + - type: precision_at_1 + value: 21.012 + - type: precision_at_10 + value: 4.723999999999999 + - type: precision_at_100 + value: 0.753 + - type: precision_at_1000 + value: 0.105 + - type: precision_at_3 + value: 11.094 + - type: precision_at_5 + value: 7.914000000000001 + - type: recall_at_1 + value: 18.587999999999997 + - type: recall_at_10 + value: 39.413 + - type: recall_at_100 + value: 59.78 + - type: recall_at_1000 + value: 79.49199999999999 + - type: recall_at_3 + value: 28.485 + - type: recall_at_5 + value: 33.367999999999995 + - task: + type: Retrieval + dataset: + type: BeIR/cqadupstack + name: MTEB CQADupstackTexRetrieval + config: default + split: test + metrics: + - type: map_at_1 + value: 12.76 + - type: map_at_10 + value: 18.859 + - type: map_at_100 + value: 19.865 + - type: map_at_1000 + value: 19.994 + - type: map_at_3 + value: 16.817 + - type: map_at_5 + value: 17.837 + - type: ndcg_at_1 + value: 15.415999999999999 + - type: ndcg_at_10 + value: 23.037 + - type: ndcg_at_100 + value: 28.164 + - type: ndcg_at_1000 + value: 31.404 + - type: ndcg_at_3 + value: 19.134999999999998 + - type: ndcg_at_5 + value: 20.711 + - type: precision_at_1 + value: 15.415999999999999 + - type: precision_at_10 + value: 4.387 + - type: precision_at_100 + value: 0.826 + - type: precision_at_1000 + value: 0.127 + - type: precision_at_3 + value: 9.257 + - type: precision_at_5 + value: 6.696000000000001 + - type: recall_at_1 + value: 12.76 + - type: recall_at_10 + value: 32.657000000000004 + - type: recall_at_100 + value: 56.023 + - type: recall_at_1000 + value: 79.572 + - type: recall_at_3 + value: 21.608 + - type: recall_at_5 + value: 25.726 + - task: + type: Retrieval + dataset: + type: BeIR/cqadupstack + name: MTEB CQADupstackUnixRetrieval + config: default + split: test + metrics: + - type: map_at_1 + value: 22.415 + - type: map_at_10 + value: 29.957 + - type: map_at_100 + value: 31.234 + - type: map_at_1000 + value: 31.351000000000003 + - type: map_at_3 + value: 27.261999999999997 + - type: map_at_5 + value: 28.708 + - type: ndcg_at_1 + value: 26.118999999999996 + - type: ndcg_at_10 + value: 34.961999999999996 + - type: ndcg_at_100 + value: 40.876000000000005 + - type: ndcg_at_1000 + value: 43.586000000000006 + - type: ndcg_at_3 + value: 29.958000000000002 + - type: ndcg_at_5 + value: 32.228 + - type: precision_at_1 + value: 26.118999999999996 + - type: precision_at_10 + value: 6.053999999999999 + - type: precision_at_100 + value: 1.012 + - type: precision_at_1000 + value: 0.13699999999999998 + - type: precision_at_3 + value: 13.65 + - type: precision_at_5 + value: 9.795 + - type: recall_at_1 + value: 22.415 + - type: recall_at_10 + value: 46.339000000000006 + - type: recall_at_100 + value: 72.30799999999999 + - type: recall_at_1000 + value: 91.448 + - type: recall_at_3 + value: 32.673 + - type: recall_at_5 + value: 38.467 + - task: + type: Retrieval + dataset: + type: BeIR/cqadupstack + name: MTEB CQADupstackWebmastersRetrieval + config: default + split: test + metrics: + - type: map_at_1 + value: 21.624 + - type: map_at_10 + value: 30.0 + - type: map_at_100 + value: 31.776 + - type: map_at_1000 + value: 32.005 + - type: map_at_3 + value: 27.314 + - type: map_at_5 + value: 28.741 + - type: ndcg_at_1 + value: 25.691999999999997 + - type: ndcg_at_10 + value: 35.64 + - type: ndcg_at_100 + value: 42.488 + - type: ndcg_at_1000 + value: 44.978 + - type: ndcg_at_3 + value: 31.147000000000002 + - type: ndcg_at_5 + value: 33.241 + - type: precision_at_1 + value: 25.691999999999997 + - type: precision_at_10 + value: 7.0360000000000005 + - type: precision_at_100 + value: 1.547 + - type: precision_at_1000 + value: 0.244 + - type: precision_at_3 + value: 15.02 + - type: precision_at_5 + value: 11.146 + - type: recall_at_1 + value: 21.624 + - type: recall_at_10 + value: 46.415 + - type: recall_at_100 + value: 77.086 + - type: recall_at_1000 + value: 92.72500000000001 + - type: recall_at_3 + value: 33.911 + - type: recall_at_5 + value: 39.116 + - task: + type: Retrieval + dataset: + type: BeIR/cqadupstack + name: MTEB CQADupstackWordpressRetrieval + config: default + split: test + metrics: + - type: map_at_1 + value: 15.659 + - type: map_at_10 + value: 22.35 + - type: map_at_100 + value: 23.498 + - type: map_at_1000 + value: 23.602 + - type: map_at_3 + value: 19.959 + - type: map_at_5 + value: 21.201999999999998 + - type: ndcg_at_1 + value: 17.375 + - type: ndcg_at_10 + value: 26.606 + - type: ndcg_at_100 + value: 32.567 + - type: ndcg_at_1000 + value: 35.177 + - type: ndcg_at_3 + value: 21.843 + - type: ndcg_at_5 + value: 23.924 + - type: precision_at_1 + value: 17.375 + - type: precision_at_10 + value: 4.455 + - type: precision_at_100 + value: 0.8109999999999999 + - type: precision_at_1000 + value: 0.11199999999999999 + - type: precision_at_3 + value: 9.427000000000001 + - type: precision_at_5 + value: 6.912999999999999 + - type: recall_at_1 + value: 15.659 + - type: recall_at_10 + value: 38.167 + - type: recall_at_100 + value: 66.11 + - type: recall_at_1000 + value: 85.529 + - type: recall_at_3 + value: 25.308000000000003 + - type: recall_at_5 + value: 30.182 + - task: + type: Retrieval + dataset: + type: climate-fever + name: MTEB ClimateFEVER + config: default + split: test + metrics: + - type: map_at_1 + value: 3.9469999999999996 + - type: map_at_10 + value: 6.816999999999999 + - type: map_at_100 + value: 7.7170000000000005 + - type: map_at_1000 + value: 7.887 + - type: map_at_3 + value: 5.6739999999999995 + - type: map_at_5 + value: 6.243 + - type: ndcg_at_1 + value: 8.73 + - type: ndcg_at_10 + value: 10.366999999999999 + - type: ndcg_at_100 + value: 15.343000000000002 + - type: ndcg_at_1000 + value: 19.535 + - type: ndcg_at_3 + value: 7.976 + - type: ndcg_at_5 + value: 8.786 + - type: precision_at_1 + value: 8.73 + - type: precision_at_10 + value: 3.3160000000000003 + - type: precision_at_100 + value: 0.857 + - type: precision_at_1000 + value: 0.16199999999999998 + - type: precision_at_3 + value: 5.776 + - type: precision_at_5 + value: 4.534 + - type: recall_at_1 + value: 3.9469999999999996 + - type: recall_at_10 + value: 13.385 + - type: recall_at_100 + value: 31.612000000000002 + - type: recall_at_1000 + value: 56.252 + - type: recall_at_3 + value: 7.686 + - type: recall_at_5 + value: 9.879 + - task: + type: Retrieval + dataset: + type: dbpedia-entity + name: MTEB DBPedia + config: default + split: test + metrics: + - type: map_at_1 + value: 5.75 + - type: map_at_10 + value: 11.632000000000001 + - type: map_at_100 + value: 16.400000000000002 + - type: map_at_1000 + value: 17.580000000000002 + - type: map_at_3 + value: 8.49 + - type: map_at_5 + value: 9.626999999999999 + - type: ndcg_at_1 + value: 35.75 + - type: ndcg_at_10 + value: 27.766000000000002 + - type: ndcg_at_100 + value: 31.424000000000003 + - type: ndcg_at_1000 + value: 38.998 + - type: ndcg_at_3 + value: 30.807000000000002 + - type: ndcg_at_5 + value: 28.62 + - type: precision_at_1 + value: 44.25 + - type: precision_at_10 + value: 22.625 + - type: precision_at_100 + value: 7.163 + - type: precision_at_1000 + value: 1.619 + - type: precision_at_3 + value: 33.75 + - type: precision_at_5 + value: 28.199999999999996 + - type: recall_at_1 + value: 5.75 + - type: recall_at_10 + value: 16.918 + - type: recall_at_100 + value: 37.645 + - type: recall_at_1000 + value: 62.197 + - type: recall_at_3 + value: 9.721 + - type: recall_at_5 + value: 11.974 + - task: + type: Classification + dataset: + type: mteb/emotion + name: MTEB EmotionClassification + config: default + split: test + metrics: + - type: accuracy + value: 51.355000000000004 + - type: f1 + value: 44.27505726378252 + - task: + type: Retrieval + dataset: + type: fever + name: MTEB FEVER + config: default + split: test + metrics: + - type: map_at_1 + value: 13.81 + - type: map_at_10 + value: 21.567 + - type: map_at_100 + value: 22.461000000000002 + - type: map_at_1000 + value: 22.545 + - type: map_at_3 + value: 19.282 + - type: map_at_5 + value: 20.535999999999998 + - type: ndcg_at_1 + value: 15.032 + - type: ndcg_at_10 + value: 26.165 + - type: ndcg_at_100 + value: 30.819999999999997 + - type: ndcg_at_1000 + value: 33.209 + - type: ndcg_at_3 + value: 21.488 + - type: ndcg_at_5 + value: 23.721999999999998 + - type: precision_at_1 + value: 15.032 + - type: precision_at_10 + value: 4.292 + - type: precision_at_100 + value: 0.6779999999999999 + - type: precision_at_1000 + value: 0.09 + - type: precision_at_3 + value: 9.551 + - type: precision_at_5 + value: 6.927999999999999 + - type: recall_at_1 + value: 13.81 + - type: recall_at_10 + value: 39.009 + - type: recall_at_100 + value: 60.99400000000001 + - type: recall_at_1000 + value: 79.703 + - type: recall_at_3 + value: 26.221 + - type: recall_at_5 + value: 31.604 + - task: + type: Retrieval + dataset: + type: fiqa + name: MTEB FiQA2018 + config: default + split: test + metrics: + - type: map_at_1 + value: 16.55 + - type: map_at_10 + value: 27.101 + - type: map_at_100 + value: 28.941 + - type: map_at_1000 + value: 29.137 + - type: map_at_3 + value: 22.926 + - type: map_at_5 + value: 25.217 + - type: ndcg_at_1 + value: 33.951 + - type: ndcg_at_10 + value: 34.832 + - type: ndcg_at_100 + value: 41.989 + - type: ndcg_at_1000 + value: 45.262 + - type: ndcg_at_3 + value: 30.427 + - type: ndcg_at_5 + value: 31.985999999999997 + - type: precision_at_1 + value: 33.951 + - type: precision_at_10 + value: 10.139 + - type: precision_at_100 + value: 1.735 + - type: precision_at_1000 + value: 0.233 + - type: precision_at_3 + value: 20.576 + - type: precision_at_5 + value: 15.556000000000001 + - type: recall_at_1 + value: 16.55 + - type: recall_at_10 + value: 42.153 + - type: recall_at_100 + value: 69.19999999999999 + - type: recall_at_1000 + value: 88.631 + - type: recall_at_3 + value: 27.071 + - type: recall_at_5 + value: 33.432 + - task: + type: Retrieval + dataset: + type: hotpotqa + name: MTEB HotpotQA + config: default + split: test + metrics: + - type: map_at_1 + value: 18.102999999999998 + - type: map_at_10 + value: 26.006 + - type: map_at_100 + value: 27.060000000000002 + - type: map_at_1000 + value: 27.173000000000002 + - type: map_at_3 + value: 23.815 + - type: map_at_5 + value: 24.978 + - type: ndcg_at_1 + value: 36.205 + - type: ndcg_at_10 + value: 33.198 + - type: ndcg_at_100 + value: 37.836999999999996 + - type: ndcg_at_1000 + value: 40.499 + - type: ndcg_at_3 + value: 29.108 + - type: ndcg_at_5 + value: 30.993 + - type: precision_at_1 + value: 36.205 + - type: precision_at_10 + value: 7.404 + - type: precision_at_100 + value: 1.109 + - type: precision_at_1000 + value: 0.146 + - type: precision_at_3 + value: 18.479 + - type: precision_at_5 + value: 12.581000000000001 + - type: recall_at_1 + value: 18.102999999999998 + - type: recall_at_10 + value: 37.022 + - type: recall_at_100 + value: 55.449000000000005 + - type: recall_at_1000 + value: 73.214 + - type: recall_at_3 + value: 27.717999999999996 + - type: recall_at_5 + value: 31.452 + - task: + type: Classification + dataset: + type: mteb/imdb + name: MTEB ImdbClassification + config: default + split: test + metrics: + - type: accuracy + value: 77.3372 + - type: ap + value: 71.64946791935137 + - type: f1 + value: 77.13428403424751 + - task: + type: Retrieval + dataset: + type: msmarco + name: MTEB MSMARCO + config: default + split: dev + metrics: + - type: map_at_1 + value: 9.093 + - type: map_at_10 + value: 16.227 + - type: map_at_100 + value: 17.477999999999998 + - type: map_at_1000 + value: 17.579 + - type: map_at_3 + value: 13.541 + - type: map_at_5 + value: 14.921000000000001 + - type: ndcg_at_1 + value: 9.370000000000001 + - type: ndcg_at_10 + value: 20.705000000000002 + - type: ndcg_at_100 + value: 27.331 + - type: ndcg_at_1000 + value: 30.104 + - type: ndcg_at_3 + value: 15.081 + - type: ndcg_at_5 + value: 17.551 + - type: precision_at_1 + value: 9.370000000000001 + - type: precision_at_10 + value: 3.633 + - type: precision_at_100 + value: 0.7040000000000001 + - type: precision_at_1000 + value: 0.094 + - type: precision_at_3 + value: 6.648 + - type: precision_at_5 + value: 5.241 + - type: recall_at_1 + value: 9.093 + - type: recall_at_10 + value: 34.777 + - type: recall_at_100 + value: 66.673 + - type: recall_at_1000 + value: 88.44999999999999 + - type: recall_at_3 + value: 19.194 + - type: recall_at_5 + value: 25.124999999999996 + - task: + type: Classification + dataset: + type: mteb/mtop_domain + name: MTEB MTOPDomainClassification (en) + config: en + split: test + metrics: + - type: accuracy + value: 90.3374373005016 + - type: f1 + value: 90.25497662319412 + - task: + type: Classification + dataset: + type: mteb/mtop_domain + name: MTEB MTOPDomainClassification (de) + config: de + split: test + metrics: + - type: accuracy + value: 76.98224852071004 + - type: f1 + value: 75.10443724253962 + - task: + type: Classification + dataset: + type: mteb/mtop_domain + name: MTEB MTOPDomainClassification (es) + config: es + split: test + metrics: + - type: accuracy + value: 73.60907271514343 + - type: f1 + value: 73.15530983235772 + - task: + type: Classification + dataset: + type: mteb/mtop_domain + name: MTEB MTOPDomainClassification (fr) + config: fr + split: test + metrics: + - type: accuracy + value: 75.02975258377701 + - type: f1 + value: 75.53083321964739 + - task: + type: Classification + dataset: + type: mteb/mtop_domain + name: MTEB MTOPDomainClassification (hi) + config: hi + split: test + metrics: + - type: accuracy + value: 21.40193617784152 + - type: f1 + value: 15.465217146460256 + - task: + type: Classification + dataset: + type: mteb/mtop_domain + name: MTEB MTOPDomainClassification (th) + config: th + split: test + metrics: + - type: accuracy + value: 16.206148282097647 + - type: f1 + value: 11.580229602870345 + - task: + type: Classification + dataset: + type: mteb/mtop_intent + name: MTEB MTOPIntentClassification (en) + config: en + split: test + metrics: + - type: accuracy + value: 63.32421340629275 + - type: f1 + value: 45.42341063027956 + - task: + type: Classification + dataset: + type: mteb/mtop_intent + name: MTEB MTOPIntentClassification (de) + config: de + split: test + metrics: + - type: accuracy + value: 44.426599041983664 + - type: f1 + value: 27.205947872504428 + - task: + type: Classification + dataset: + type: mteb/mtop_intent + name: MTEB MTOPIntentClassification (es) + config: es + split: test + metrics: + - type: accuracy + value: 42.02801867911942 + - type: f1 + value: 26.314909946795733 + - task: + type: Classification + dataset: + type: mteb/mtop_intent + name: MTEB MTOPIntentClassification (fr) + config: fr + split: test + metrics: + - type: accuracy + value: 43.845912934544316 + - type: f1 + value: 29.519701972859792 + - task: + type: Classification + dataset: + type: mteb/mtop_intent + name: MTEB MTOPIntentClassification (hi) + config: hi + split: test + metrics: + - type: accuracy + value: 3.80064539261384 + - type: f1 + value: 1.2078686392462628 + - task: + type: Classification + dataset: + type: mteb/mtop_intent + name: MTEB MTOPIntentClassification (th) + config: th + split: test + metrics: + - type: accuracy + value: 5.207956600361665 + - type: f1 + value: 1.5365513001536746 + - task: + type: Classification + dataset: + type: mteb/amazon_massive_intent + name: MTEB MassiveIntentClassification (af) + config: af + split: test + metrics: + - type: accuracy + value: 34.32078009414929 + - type: f1 + value: 31.969428974847435 + - task: + type: Classification + dataset: + type: mteb/amazon_massive_intent + name: MTEB MassiveIntentClassification (am) + config: am + split: test + metrics: + - type: accuracy + value: 2.3772696704774714 + - type: f1 + value: 1.027013290806954 + - task: + type: Classification + dataset: + type: mteb/amazon_massive_intent + name: MTEB MassiveIntentClassification (ar) + config: ar + split: test + metrics: + - type: accuracy + value: 4.53261600537996 + - type: f1 + value: 2.793131265571347 + - task: + type: Classification + dataset: + type: mteb/amazon_massive_intent + name: MTEB MassiveIntentClassification (az) + config: az + split: test + metrics: + - type: accuracy + value: 31.758574310692673 + - type: f1 + value: 30.162299253522708 + - task: + type: Classification + dataset: + type: mteb/amazon_massive_intent + name: MTEB MassiveIntentClassification (bn) + config: bn + split: test + metrics: + - type: accuracy + value: 2.5823806321452594 + - type: f1 + value: 1.0918434877949255 + - task: + type: Classification + dataset: + type: mteb/amazon_massive_intent + name: MTEB MassiveIntentClassification (cy) + config: cy + split: test + metrics: + - type: accuracy + value: 28.94418291862811 + - type: f1 + value: 27.498874158049468 + - task: + type: Classification + dataset: + type: mteb/amazon_massive_intent + name: MTEB MassiveIntentClassification (da) + config: da + split: test + metrics: + - type: accuracy + value: 38.81977135171486 + - type: f1 + value: 36.44688565156101 + - task: + type: Classification + dataset: + type: mteb/amazon_massive_intent + name: MTEB MassiveIntentClassification (de) + config: de + split: test + metrics: + - type: accuracy + value: 45.22864828513786 + - type: f1 + value: 41.61460113481098 + - task: + type: Classification + dataset: + type: mteb/amazon_massive_intent + name: MTEB MassiveIntentClassification (el) + config: el + split: test + metrics: + - type: accuracy + value: 10.053799596503026 + - type: f1 + value: 5.1615743271775285 + - task: + type: Classification + dataset: + type: mteb/amazon_massive_intent + name: MTEB MassiveIntentClassification (en) + config: en + split: test + metrics: + - type: accuracy + value: 69.74445191661063 + - type: f1 + value: 67.00099408297854 + - task: + type: Classification + dataset: + type: mteb/amazon_massive_intent + name: MTEB MassiveIntentClassification (es) + config: es + split: test + metrics: + - type: accuracy + value: 45.31943510423672 + - type: f1 + value: 43.92469151179908 + - task: + type: Classification + dataset: + type: mteb/amazon_massive_intent + name: MTEB MassiveIntentClassification (fa) + config: fa + split: test + metrics: + - type: accuracy + value: 3.5810356422326834 + - type: f1 + value: 0.8057464198110936 + - task: + type: Classification + dataset: + type: mteb/amazon_massive_intent + name: MTEB MassiveIntentClassification (fi) + config: fi + split: test + metrics: + - type: accuracy + value: 33.52387357094821 + - type: f1 + value: 30.686159550520415 + - task: + type: Classification + dataset: + type: mteb/amazon_massive_intent + name: MTEB MassiveIntentClassification (fr) + config: fr + split: test + metrics: + - type: accuracy + value: 51.13315400134499 + - type: f1 + value: 48.84533274433444 + - task: + type: Classification + dataset: + type: mteb/amazon_massive_intent + name: MTEB MassiveIntentClassification (he) + config: he + split: test + metrics: + - type: accuracy + value: 2.632817753866846 + - type: f1 + value: 0.7565304035292157 + - task: + type: Classification + dataset: + type: mteb/amazon_massive_intent + name: MTEB MassiveIntentClassification (hi) + config: hi + split: test + metrics: + - type: accuracy + value: 2.6798924008069935 + - type: f1 + value: 1.5577100383199163 + - task: + type: Classification + dataset: + type: mteb/amazon_massive_intent + name: MTEB MassiveIntentClassification (hu) + config: hu + split: test + metrics: + - type: accuracy + value: 32.306657700067255 + - type: f1 + value: 29.508334412971788 + - task: + type: Classification + dataset: + type: mteb/amazon_massive_intent + name: MTEB MassiveIntentClassification (hy) + config: hy + split: test + metrics: + - type: accuracy + value: 3.3254875588433084 + - type: f1 + value: 0.9498561670625558 + - task: + type: Classification + dataset: + type: mteb/amazon_massive_intent + name: MTEB MassiveIntentClassification (id) + config: id + split: test + metrics: + - type: accuracy + value: 35.497646267652996 + - type: f1 + value: 32.919473578262014 + - task: + type: Classification + dataset: + type: mteb/amazon_massive_intent + name: MTEB MassiveIntentClassification (is) + config: is + split: test + metrics: + - type: accuracy + value: 29.818426361802285 + - type: f1 + value: 27.968522255792134 + - task: + type: Classification + dataset: + type: mteb/amazon_massive_intent + name: MTEB MassiveIntentClassification (it) + config: it + split: test + metrics: + - type: accuracy + value: 45.585070611970416 + - type: f1 + value: 43.85609178763681 + - task: + type: Classification + dataset: + type: mteb/amazon_massive_intent + name: MTEB MassiveIntentClassification (ja) + config: ja + split: test + metrics: + - type: accuracy + value: 3.6718224613315398 + - type: f1 + value: 1.5834733153849028 + - task: + type: Classification + dataset: + type: mteb/amazon_massive_intent + name: MTEB MassiveIntentClassification (jv) + config: jv + split: test + metrics: + - type: accuracy + value: 31.149966375252188 + - type: f1 + value: 28.77156087445068 + - task: + type: Classification + dataset: + type: mteb/amazon_massive_intent + name: MTEB MassiveIntentClassification (ka) + config: ka + split: test + metrics: + - type: accuracy + value: 2.767316745124411 + - type: f1 + value: 1.0163373847923576 + - task: + type: Classification + dataset: + type: mteb/amazon_massive_intent + name: MTEB MassiveIntentClassification (km) + config: km + split: test + metrics: + - type: accuracy + value: 5.655682582380632 + - type: f1 + value: 1.6046205246119 + - task: + type: Classification + dataset: + type: mteb/amazon_massive_intent + name: MTEB MassiveIntentClassification (kn) + config: kn + split: test + metrics: + - type: accuracy + value: 2.5924680564895763 + - type: f1 + value: 1.3338404330308657 + - task: + type: Classification + dataset: + type: mteb/amazon_massive_intent + name: MTEB MassiveIntentClassification (ko) + config: ko + split: test + metrics: + - type: accuracy + value: 2.3436449226630804 + - type: f1 + value: 0.5935093070394912 + - task: + type: Classification + dataset: + type: mteb/amazon_massive_intent + name: MTEB MassiveIntentClassification (lv) + config: lv + split: test + metrics: + - type: accuracy + value: 33.97108271687962 + - type: f1 + value: 33.35695453571926 + - task: + type: Classification + dataset: + type: mteb/amazon_massive_intent + name: MTEB MassiveIntentClassification (ml) + config: ml + split: test + metrics: + - type: accuracy + value: 2.5453934095494284 + - type: f1 + value: 0.5515796181696971 + - task: + type: Classification + dataset: + type: mteb/amazon_massive_intent + name: MTEB MassiveIntentClassification (mn) + config: mn + split: test + metrics: + - type: accuracy + value: 14.704102219233356 + - type: f1 + value: 12.444230806799856 + - task: + type: Classification + dataset: + type: mteb/amazon_massive_intent + name: MTEB MassiveIntentClassification (ms) + config: ms + split: test + metrics: + - type: accuracy + value: 33.12037659717552 + - type: f1 + value: 29.867258908899636 + - task: + type: Classification + dataset: + type: mteb/amazon_massive_intent + name: MTEB MassiveIntentClassification (my) + config: my + split: test + metrics: + - type: accuracy + value: 4.421654337592468 + - type: f1 + value: 1.3125497683444283 + - task: + type: Classification + dataset: + type: mteb/amazon_massive_intent + name: MTEB MassiveIntentClassification (nb) + config: nb + split: test + metrics: + - type: accuracy + value: 38.53059852051109 + - type: f1 + value: 35.473185172465996 + - task: + type: Classification + dataset: + type: mteb/amazon_massive_intent + name: MTEB MassiveIntentClassification (nl) + config: nl + split: test + metrics: + - type: accuracy + value: 37.96234028244788 + - type: f1 + value: 34.24786837723274 + - task: + type: Classification + dataset: + type: mteb/amazon_massive_intent + name: MTEB MassiveIntentClassification (pl) + config: pl + split: test + metrics: + - type: accuracy + value: 34.40820443846672 + - type: f1 + value: 32.06121218840769 + - task: + type: Classification + dataset: + type: mteb/amazon_massive_intent + name: MTEB MassiveIntentClassification (pt) + config: pt + split: test + metrics: + - type: accuracy + value: 43.35238735709483 + - type: f1 + value: 41.66578945324342 + - task: + type: Classification + dataset: + type: mteb/amazon_massive_intent + name: MTEB MassiveIntentClassification (ro) + config: ro + split: test + metrics: + - type: accuracy + value: 42.68997982515131 + - type: f1 + value: 40.12799296163562 + - task: + type: Classification + dataset: + type: mteb/amazon_massive_intent + name: MTEB MassiveIntentClassification (ru) + config: ru + split: test + metrics: + - type: accuracy + value: 14.815063887020846 + - type: f1 + value: 14.113147395663178 + - task: + type: Classification + dataset: + type: mteb/amazon_massive_intent + name: MTEB MassiveIntentClassification (sl) + config: sl + split: test + metrics: + - type: accuracy + value: 34.54270342972428 + - type: f1 + value: 32.556771565845736 + - task: + type: Classification + dataset: + type: mteb/amazon_massive_intent + name: MTEB MassiveIntentClassification (sq) + config: sq + split: test + metrics: + - type: accuracy + value: 38.54068594485541 + - type: f1 + value: 34.63112962341627 + - task: + type: Classification + dataset: + type: mteb/amazon_massive_intent + name: MTEB MassiveIntentClassification (sv) + config: sv + split: test + metrics: + - type: accuracy + value: 35.978480161398785 + - type: f1 + value: 32.896757123025 + - task: + type: Classification + dataset: + type: mteb/amazon_massive_intent + name: MTEB MassiveIntentClassification (sw) + config: sw + split: test + metrics: + - type: accuracy + value: 32.135171486213856 + - type: f1 + value: 30.338211408536065 + - task: + type: Classification + dataset: + type: mteb/amazon_massive_intent + name: MTEB MassiveIntentClassification (ta) + config: ta + split: test + metrics: + - type: accuracy + value: 1.4055144586415602 + - type: f1 + value: 0.4241059307791662 + - task: + type: Classification + dataset: + type: mteb/amazon_massive_intent + name: MTEB MassiveIntentClassification (te) + config: te + split: test + metrics: + - type: accuracy + value: 2.501681237390719 + - type: f1 + value: 0.9247691466655827 + - task: + type: Classification + dataset: + type: mteb/amazon_massive_intent + name: MTEB MassiveIntentClassification (th) + config: th + split: test + metrics: + - type: accuracy + value: 3.7088096839273708 + - type: f1 + value: 1.7117159563540418 + - task: + type: Classification + dataset: + type: mteb/amazon_massive_intent + name: MTEB MassiveIntentClassification (tl) + config: tl + split: test + metrics: + - type: accuracy + value: 36.04236718224613 + - type: f1 + value: 31.848270154788448 + - task: + type: Classification + dataset: + type: mteb/amazon_massive_intent + name: MTEB MassiveIntentClassification (tr) + config: tr + split: test + metrics: + - type: accuracy + value: 33.765971755211844 + - type: f1 + value: 31.25189279882076 + - task: + type: Classification + dataset: + type: mteb/amazon_massive_intent + name: MTEB MassiveIntentClassification (ur) + config: ur + split: test + metrics: + - type: accuracy + value: 2.992602555480834 + - type: f1 + value: 1.6243709826215067 + - task: + type: Classification + dataset: + type: mteb/amazon_massive_intent + name: MTEB MassiveIntentClassification (vi) + config: vi + split: test + metrics: + - type: accuracy + value: 22.61936785474109 + - type: f1 + value: 20.616493213131037 + - task: + type: Classification + dataset: + type: mteb/amazon_massive_intent + name: MTEB MassiveIntentClassification (zh-CN) + config: zh-CN + split: test + metrics: + - type: accuracy + value: 1.1163416274377942 + - type: f1 + value: 0.17708079483587733 + - task: + type: Classification + dataset: + type: mteb/amazon_massive_intent + name: MTEB MassiveIntentClassification (zh-TW) + config: zh-TW + split: test + metrics: + - type: accuracy + value: 4.6267652992602555 + - type: f1 + value: 1.8933559386335084 + - task: + type: Classification + dataset: + type: mteb/amazon_massive_scenario + name: MTEB MassiveScenarioClassification (af) + config: af + split: test + metrics: + - type: accuracy + value: 44.45191661062542 + - type: f1 + value: 40.78791900409084 + - task: + type: Classification + dataset: + type: mteb/amazon_massive_scenario + name: MTEB MassiveScenarioClassification (am) + config: am + split: test + metrics: + - type: accuracy + value: 7.508406186953598 + - type: f1 + value: 2.821623266241915 + - task: + type: Classification + dataset: + type: mteb/amazon_massive_scenario + name: MTEB MassiveScenarioClassification (ar) + config: ar + split: test + metrics: + - type: accuracy + value: 12.323470073974445 + - type: f1 + value: 7.033060282091267 + - task: + type: Classification + dataset: + type: mteb/amazon_massive_scenario + name: MTEB MassiveScenarioClassification (az) + config: az + split: test + metrics: + - type: accuracy + value: 38.40954942837929 + - type: f1 + value: 35.118197115254915 + - task: + type: Classification + dataset: + type: mteb/amazon_massive_scenario + name: MTEB MassiveScenarioClassification (bn) + config: bn + split: test + metrics: + - type: accuracy + value: 8.453261600537996 + - type: f1 + value: 3.7954675930409887 + - task: + type: Classification + dataset: + type: mteb/amazon_massive_scenario + name: MTEB MassiveScenarioClassification (cy) + config: cy + split: test + metrics: + - type: accuracy + value: 35.036987222595826 + - type: f1 + value: 32.66245000430742 + - task: + type: Classification + dataset: + type: mteb/amazon_massive_scenario + name: MTEB MassiveScenarioClassification (da) + config: da + split: test + metrics: + - type: accuracy + value: 48.35911230665769 + - type: f1 + value: 45.007102253202646 + - task: + type: Classification + dataset: + type: mteb/amazon_massive_scenario + name: MTEB MassiveScenarioClassification (de) + config: de + split: test + metrics: + - type: accuracy + value: 59.11903160726294 + - type: f1 + value: 56.29359157547811 + - task: + type: Classification + dataset: + type: mteb/amazon_massive_scenario + name: MTEB MassiveScenarioClassification (el) + config: el + split: test + metrics: + - type: accuracy + value: 17.679892400806995 + - type: f1 + value: 12.298039805650598 + - task: + type: Classification + dataset: + type: mteb/amazon_massive_scenario + name: MTEB MassiveScenarioClassification (en) + config: en + split: test + metrics: + - type: accuracy + value: 72.32347007397445 + - type: f1 + value: 71.44743919882103 + - task: + type: Classification + dataset: + type: mteb/amazon_massive_scenario + name: MTEB MassiveScenarioClassification (es) + config: es + split: test + metrics: + - type: accuracy + value: 55.61197041022192 + - type: f1 + value: 53.34238608745496 + - task: + type: Classification + dataset: + type: mteb/amazon_massive_scenario + name: MTEB MassiveScenarioClassification (fa) + config: fa + split: test + metrics: + - type: accuracy + value: 6.859448554135844 + - type: f1 + value: 2.6611096235427603 + - task: + type: Classification + dataset: + type: mteb/amazon_massive_scenario + name: MTEB MassiveScenarioClassification (fi) + config: fi + split: test + metrics: + - type: accuracy + value: 41.34162743779422 + - type: f1 + value: 37.16587245902462 + - task: + type: Classification + dataset: + type: mteb/amazon_massive_scenario + name: MTEB MassiveScenarioClassification (fr) + config: fr + split: test + metrics: + - type: accuracy + value: 59.91930060524545 + - type: f1 + value: 58.82978451275059 + - task: + type: Classification + dataset: + type: mteb/amazon_massive_scenario + name: MTEB MassiveScenarioClassification (he) + config: he + split: test + metrics: + - type: accuracy + value: 7.864828513786147 + - type: f1 + value: 2.1478868713724184 + - task: + type: Classification + dataset: + type: mteb/amazon_massive_scenario + name: MTEB MassiveScenarioClassification (hi) + config: hi + split: test + metrics: + - type: accuracy + value: 7.6328177538668465 + - type: f1 + value: 4.359158983520571 + - task: + type: Classification + dataset: + type: mteb/amazon_massive_scenario + name: MTEB MassiveScenarioClassification (hu) + config: hu + split: test + metrics: + - type: accuracy + value: 41.31136516476127 + - type: f1 + value: 37.56187834379295 + - task: + type: Classification + dataset: + type: mteb/amazon_massive_scenario + name: MTEB MassiveScenarioClassification (hy) + config: hy + split: test + metrics: + - type: accuracy + value: 9.226630800268998 + - type: f1 + value: 4.272151036150232 + - task: + type: Classification + dataset: + type: mteb/amazon_massive_scenario + name: MTEB MassiveScenarioClassification (id) + config: id + split: test + metrics: + - type: accuracy + value: 44.64357767316745 + - type: f1 + value: 42.836643630228295 + - task: + type: Classification + dataset: + type: mteb/amazon_massive_scenario + name: MTEB MassiveScenarioClassification (is) + config: is + split: test + metrics: + - type: accuracy + value: 39.633490248823136 + - type: f1 + value: 35.97498169100485 + - task: + type: Classification + dataset: + type: mteb/amazon_massive_scenario + name: MTEB MassiveScenarioClassification (it) + config: it + split: test + metrics: + - type: accuracy + value: 54.57969065232011 + - type: f1 + value: 51.96887024053593 + - task: + type: Classification + dataset: + type: mteb/amazon_massive_scenario + name: MTEB MassiveScenarioClassification (ja) + config: ja + split: test + metrics: + - type: accuracy + value: 4.956287827841291 + - type: f1 + value: 2.885322950712636 + - task: + type: Classification + dataset: + type: mteb/amazon_massive_scenario + name: MTEB MassiveScenarioClassification (jv) + config: jv + split: test + metrics: + - type: accuracy + value: 40.73301950235374 + - type: f1 + value: 37.77421393601645 + - task: + type: Classification + dataset: + type: mteb/amazon_massive_scenario + name: MTEB MassiveScenarioClassification (ka) + config: ka + split: test + metrics: + - type: accuracy + value: 7.505043712172157 + - type: f1 + value: 2.885798319215117 + - task: + type: Classification + dataset: + type: mteb/amazon_massive_scenario + name: MTEB MassiveScenarioClassification (km) + config: km + split: test + metrics: + - type: accuracy + value: 8.732347007397442 + - type: f1 + value: 3.3227736931128002 + - task: + type: Classification + dataset: + type: mteb/amazon_massive_scenario + name: MTEB MassiveScenarioClassification (kn) + config: kn + split: test + metrics: + - type: accuracy + value: 7.985877605917954 + - type: f1 + value: 4.454873367911008 + - task: + type: Classification + dataset: + type: mteb/amazon_massive_scenario + name: MTEB MassiveScenarioClassification (ko) + config: ko + split: test + metrics: + - type: accuracy + value: 6.028917283120376 + - type: f1 + value: 2.1947301613241064 + - task: + type: Classification + dataset: + type: mteb/amazon_massive_scenario + name: MTEB MassiveScenarioClassification (lv) + config: lv + split: test + metrics: + - type: accuracy + value: 36.415601882985875 + - type: f1 + value: 34.328528362740194 + - task: + type: Classification + dataset: + type: mteb/amazon_massive_scenario + name: MTEB MassiveScenarioClassification (ml) + config: ml + split: test + metrics: + - type: accuracy + value: 6.960322797579018 + - type: f1 + value: 2.5948923502086636 + - task: + type: Classification + dataset: + type: mteb/amazon_massive_scenario + name: MTEB MassiveScenarioClassification (mn) + config: mn + split: test + metrics: + - type: accuracy + value: 19.84868863483524 + - type: f1 + value: 16.571872313821913 + - task: + type: Classification + dataset: + type: mteb/amazon_massive_scenario + name: MTEB MassiveScenarioClassification (ms) + config: ms + split: test + metrics: + - type: accuracy + value: 43.18090114324143 + - type: f1 + value: 39.21386548691514 + - task: + type: Classification + dataset: + type: mteb/amazon_massive_scenario + name: MTEB MassiveScenarioClassification (my) + config: my + split: test + metrics: + - type: accuracy + value: 9.462004034969738 + - type: f1 + value: 4.439617481980993 + - task: + type: Classification + dataset: + type: mteb/amazon_massive_scenario + name: MTEB MassiveScenarioClassification (nb) + config: nb + split: test + metrics: + - type: accuracy + value: 46.60390047074647 + - type: f1 + value: 42.97761990856311 + - task: + type: Classification + dataset: + type: mteb/amazon_massive_scenario + name: MTEB MassiveScenarioClassification (nl) + config: nl + split: test + metrics: + - type: accuracy + value: 50.00336247478143 + - type: f1 + value: 45.623347729802326 + - task: + type: Classification + dataset: + type: mteb/amazon_massive_scenario + name: MTEB MassiveScenarioClassification (pl) + config: pl + split: test + metrics: + - type: accuracy + value: 42.29993275050437 + - type: f1 + value: 39.93988660787354 + - task: + type: Classification + dataset: + type: mteb/amazon_massive_scenario + name: MTEB MassiveScenarioClassification (pt) + config: pt + split: test + metrics: + - type: accuracy + value: 52.23940820443846 + - type: f1 + value: 50.63740128715899 + - task: + type: Classification + dataset: + type: mteb/amazon_massive_scenario + name: MTEB MassiveScenarioClassification (ro) + config: ro + split: test + metrics: + - type: accuracy + value: 53.69535978480161 + - type: f1 + value: 51.48099264354228 + - task: + type: Classification + dataset: + type: mteb/amazon_massive_scenario + name: MTEB MassiveScenarioClassification (ru) + config: ru + split: test + metrics: + - type: accuracy + value: 20.68594485541358 + - type: f1 + value: 19.732532357173614 + - task: + type: Classification + dataset: + type: mteb/amazon_massive_scenario + name: MTEB MassiveScenarioClassification (sl) + config: sl + split: test + metrics: + - type: accuracy + value: 39.788164088769335 + - type: f1 + value: 37.63884763800779 + - task: + type: Classification + dataset: + type: mteb/amazon_massive_scenario + name: MTEB MassiveScenarioClassification (sq) + config: sq + split: test + metrics: + - type: accuracy + value: 50.164761264290526 + - type: f1 + value: 46.40832635120166 + - task: + type: Classification + dataset: + type: mteb/amazon_massive_scenario + name: MTEB MassiveScenarioClassification (sv) + config: sv + split: test + metrics: + - type: accuracy + value: 46.68796234028245 + - type: f1 + value: 42.65714705554807 + - task: + type: Classification + dataset: + type: mteb/amazon_massive_scenario + name: MTEB MassiveScenarioClassification (sw) + config: sw + split: test + metrics: + - type: accuracy + value: 40.480833893745796 + - type: f1 + value: 37.28728896543833 + - task: + type: Classification + dataset: + type: mteb/amazon_massive_scenario + name: MTEB MassiveScenarioClassification (ta) + config: ta + split: test + metrics: + - type: accuracy + value: 7.474781439139207 + - type: f1 + value: 2.114620869965788 + - task: + type: Classification + dataset: + type: mteb/amazon_massive_scenario + name: MTEB MassiveScenarioClassification (te) + config: te + split: test + metrics: + - type: accuracy + value: 6.866173503698722 + - type: f1 + value: 3.0078405064872755 + - task: + type: Classification + dataset: + type: mteb/amazon_massive_scenario + name: MTEB MassiveScenarioClassification (th) + config: th + split: test + metrics: + - type: accuracy + value: 8.258238063214526 + - type: f1 + value: 4.082391072869187 + - task: + type: Classification + dataset: + type: mteb/amazon_massive_scenario + name: MTEB MassiveScenarioClassification (tl) + config: tl + split: test + metrics: + - type: accuracy + value: 48.93745796906523 + - type: f1 + value: 46.427786382184266 + - task: + type: Classification + dataset: + type: mteb/amazon_massive_scenario + name: MTEB MassiveScenarioClassification (tr) + config: tr + split: test + metrics: + - type: accuracy + value: 41.82918628110289 + - type: f1 + value: 40.642360044818325 + - task: + type: Classification + dataset: + type: mteb/amazon_massive_scenario + name: MTEB MassiveScenarioClassification (ur) + config: ur + split: test + metrics: + - type: accuracy + value: 9.767989240080698 + - type: f1 + value: 4.6812848634278925 + - task: + type: Classification + dataset: + type: mteb/amazon_massive_scenario + name: MTEB MassiveScenarioClassification (vi) + config: vi + split: test + metrics: + - type: accuracy + value: 30.013449899125753 + - type: f1 + value: 28.091947569862718 + - task: + type: Classification + dataset: + type: mteb/amazon_massive_scenario + name: MTEB MassiveScenarioClassification (zh-CN) + config: zh-CN + split: test + metrics: + - type: accuracy + value: 4.169468728984533 + - type: f1 + value: 0.8211582847545612 + - task: + type: Classification + dataset: + type: mteb/amazon_massive_scenario + name: MTEB MassiveScenarioClassification (zh-TW) + config: zh-TW + split: test + metrics: + - type: accuracy + value: 7.908540685944855 + - type: f1 + value: 3.5358754800289534 + - task: + type: Clustering + dataset: + type: mteb/medrxiv-clustering-p2p + name: MTEB MedrxivClusteringP2P + config: default + split: test + metrics: + - type: v_measure + value: 33.20164128507875 + - task: + type: Clustering + dataset: + type: mteb/medrxiv-clustering-s2s + name: MTEB MedrxivClusteringS2S + config: default + split: test + metrics: + - type: v_measure + value: 26.13067187536383 + - task: + type: Reranking + dataset: + type: mteb/mind_small + name: MTEB MindSmallReranking + config: default + split: test + metrics: + - type: map + value: 30.196798710053223 + - type: mrr + value: 31.098179519159814 + - task: + type: Retrieval + dataset: + type: nfcorpus + name: MTEB NFCorpus + config: default + split: test + metrics: + - type: map_at_1 + value: 4.7010000000000005 + - type: map_at_10 + value: 9.756 + - type: map_at_100 + value: 12.631999999999998 + - type: map_at_1000 + value: 14.046 + - type: map_at_3 + value: 7.053 + - type: map_at_5 + value: 8.244 + - type: ndcg_at_1 + value: 35.604 + - type: ndcg_at_10 + value: 28.645 + - type: ndcg_at_100 + value: 27.431 + - type: ndcg_at_1000 + value: 36.378 + - type: ndcg_at_3 + value: 32.533 + - type: ndcg_at_5 + value: 30.737 + - type: precision_at_1 + value: 37.771 + - type: precision_at_10 + value: 21.517 + - type: precision_at_100 + value: 7.567 + - type: precision_at_1000 + value: 2.026 + - type: precision_at_3 + value: 30.753000000000004 + - type: precision_at_5 + value: 26.811 + - type: recall_at_1 + value: 4.7010000000000005 + - type: recall_at_10 + value: 14.302999999999999 + - type: recall_at_100 + value: 29.304000000000002 + - type: recall_at_1000 + value: 62.202999999999996 + - type: recall_at_3 + value: 8.419 + - type: recall_at_5 + value: 10.656 + - task: + type: Retrieval + dataset: + type: nq + name: MTEB NQ + config: default + split: test + metrics: + - type: map_at_1 + value: 17.444000000000003 + - type: map_at_10 + value: 29.198 + - type: map_at_100 + value: 30.553 + - type: map_at_1000 + value: 30.614 + - type: map_at_3 + value: 24.992 + - type: map_at_5 + value: 27.322999999999997 + - type: ndcg_at_1 + value: 19.641000000000002 + - type: ndcg_at_10 + value: 36.324 + - type: ndcg_at_100 + value: 42.641 + - type: ndcg_at_1000 + value: 44.089 + - type: ndcg_at_3 + value: 28.000000000000004 + - type: ndcg_at_5 + value: 32.025 + - type: precision_at_1 + value: 19.641000000000002 + - type: precision_at_10 + value: 6.6339999999999995 + - type: precision_at_100 + value: 1.024 + - type: precision_at_1000 + value: 0.116 + - type: precision_at_3 + value: 13.209999999999999 + - type: precision_at_5 + value: 10.162 + - type: recall_at_1 + value: 17.444000000000003 + - type: recall_at_10 + value: 56.230999999999995 + - type: recall_at_100 + value: 84.61800000000001 + - type: recall_at_1000 + value: 95.416 + - type: recall_at_3 + value: 34.245999999999995 + - type: recall_at_5 + value: 43.617 + - task: + type: Retrieval + dataset: + type: quora + name: MTEB QuoraRetrieval + config: default + split: test + metrics: + - type: map_at_1 + value: 67.604 + - type: map_at_10 + value: 81.364 + - type: map_at_100 + value: 82.092 + - type: map_at_1000 + value: 82.112 + - type: map_at_3 + value: 78.321 + - type: map_at_5 + value: 80.203 + - type: ndcg_at_1 + value: 77.92 + - type: ndcg_at_10 + value: 85.491 + - type: ndcg_at_100 + value: 87.102 + - type: ndcg_at_1000 + value: 87.246 + - type: ndcg_at_3 + value: 82.219 + - type: ndcg_at_5 + value: 83.991 + - type: precision_at_1 + value: 77.92 + - type: precision_at_10 + value: 13.065 + - type: precision_at_100 + value: 1.525 + - type: precision_at_1000 + value: 0.157 + - type: precision_at_3 + value: 35.96 + - type: precision_at_5 + value: 23.785999999999998 + - type: recall_at_1 + value: 67.604 + - type: recall_at_10 + value: 93.57 + - type: recall_at_100 + value: 99.20400000000001 + - type: recall_at_1000 + value: 99.958 + - type: recall_at_3 + value: 84.38900000000001 + - type: recall_at_5 + value: 89.223 + - task: + type: Clustering + dataset: + type: mteb/reddit-clustering + name: MTEB RedditClustering + config: default + split: test + metrics: + - type: v_measure + value: 52.930534839708464 + - task: + type: Clustering + dataset: + type: mteb/reddit-clustering-p2p + name: MTEB RedditClusteringP2P + config: default + split: test + metrics: + - type: v_measure + value: 59.6686566444821 + - task: + type: Retrieval + dataset: + type: scidocs + name: MTEB SCIDOCS + config: default + split: test + metrics: + - type: map_at_1 + value: 3.267 + - type: map_at_10 + value: 8.061 + - type: map_at_100 + value: 9.66 + - type: map_at_1000 + value: 9.926 + - type: map_at_3 + value: 5.733 + - type: map_at_5 + value: 6.894 + - type: ndcg_at_1 + value: 16.0 + - type: ndcg_at_10 + value: 14.155000000000001 + - type: ndcg_at_100 + value: 20.973 + - type: ndcg_at_1000 + value: 26.163999999999998 + - type: ndcg_at_3 + value: 12.994 + - type: ndcg_at_5 + value: 11.58 + - type: precision_at_1 + value: 16.0 + - type: precision_at_10 + value: 7.470000000000001 + - type: precision_at_100 + value: 1.7389999999999999 + - type: precision_at_1000 + value: 0.299 + - type: precision_at_3 + value: 12.167 + - type: precision_at_5 + value: 10.280000000000001 + - type: recall_at_1 + value: 3.267 + - type: recall_at_10 + value: 15.152 + - type: recall_at_100 + value: 35.248000000000005 + - type: recall_at_1000 + value: 60.742 + - type: recall_at_3 + value: 7.4319999999999995 + - type: recall_at_5 + value: 10.452 + - task: + type: STS + dataset: + type: mteb/sickr-sts + name: MTEB SICK-R + config: default + split: test + metrics: + - type: cos_sim_pearson + value: 84.12684378047692 + - type: cos_sim_spearman + value: 80.18231249099851 + - type: euclidean_pearson + value: 81.10311004134292 + - type: euclidean_spearman + value: 80.18231162371262 + - type: manhattan_pearson + value: 81.06660654194627 + - type: manhattan_spearman + value: 80.15421301055235 + - task: + type: STS + dataset: + type: mteb/sts12-sts + name: MTEB STS12 + config: default + split: test + metrics: + - type: cos_sim_pearson + value: 86.39022108792102 + - type: cos_sim_spearman + value: 78.0511871449349 + - type: euclidean_pearson + value: 83.55414895785707 + - type: euclidean_spearman + value: 78.04999900363751 + - type: manhattan_pearson + value: 83.58122709700247 + - type: manhattan_spearman + value: 78.09617051485085 + - task: + type: STS + dataset: + type: mteb/sts13-sts + name: MTEB STS13 + config: default + split: test + metrics: + - type: cos_sim_pearson + value: 85.20665643089602 + - type: cos_sim_spearman + value: 85.84897342040492 + - type: euclidean_pearson + value: 85.07344348481206 + - type: euclidean_spearman + value: 85.84897334409469 + - type: manhattan_pearson + value: 85.05095172720918 + - type: manhattan_spearman + value: 85.82539599484174 + - task: + type: STS + dataset: + type: mteb/sts14-sts + name: MTEB STS14 + config: default + split: test + metrics: + - type: cos_sim_pearson + value: 83.89550144541676 + - type: cos_sim_spearman + value: 82.18926664662587 + - type: euclidean_pearson + value: 83.2979886572065 + - type: euclidean_spearman + value: 82.18927470901535 + - type: manhattan_pearson + value: 83.26470031355984 + - type: manhattan_spearman + value: 82.18712042624048 + - task: + type: STS + dataset: + type: mteb/sts15-sts + name: MTEB STS15 + config: default + split: test + metrics: + - type: cos_sim_pearson + value: 86.24309164032012 + - type: cos_sim_spearman + value: 87.45860981918769 + - type: euclidean_pearson + value: 87.04473506428359 + - type: euclidean_spearman + value: 87.45861561864089 + - type: manhattan_pearson + value: 87.02002615328881 + - type: manhattan_spearman + value: 87.43661746711435 + - task: + type: STS + dataset: + type: mteb/sts16-sts + name: MTEB STS16 + config: default + split: test + metrics: + - type: cos_sim_pearson + value: 82.24202172291855 + - type: cos_sim_spearman + value: 84.03233567112525 + - type: euclidean_pearson + value: 83.5361433714169 + - type: euclidean_spearman + value: 84.03233506665642 + - type: manhattan_pearson + value: 83.51738829906122 + - type: manhattan_spearman + value: 84.02036537979589 + - task: + type: STS + dataset: + type: mteb/sts17-crosslingual-sts + name: MTEB STS17 (ko-ko) + config: ko-ko + split: test + metrics: + - type: cos_sim_pearson + value: 9.160685666083912 + - type: cos_sim_spearman + value: 10.0553422118037 + - type: euclidean_pearson + value: 9.589155152132493 + - type: euclidean_spearman + value: 10.215143153291868 + - type: manhattan_pearson + value: 9.570908402796292 + - type: manhattan_spearman + value: 10.214075999964175 + - task: + type: STS + dataset: + type: mteb/sts17-crosslingual-sts + name: MTEB STS17 (ar-ar) + config: ar-ar + split: test + metrics: + - type: cos_sim_pearson + value: 10.60353259635145 + - type: cos_sim_spearman + value: 13.355557088500165 + - type: euclidean_pearson + value: 14.636463268109537 + - type: euclidean_spearman + value: 14.35296057730866 + - type: manhattan_pearson + value: 14.553161459629774 + - type: manhattan_spearman + value: 14.267005982719008 + - task: + type: STS + dataset: + type: mteb/sts17-crosslingual-sts + name: MTEB STS17 (en-ar) + config: en-ar + split: test + metrics: + - type: cos_sim_pearson + value: -4.869359628676264 + - type: cos_sim_spearman + value: -5.6460908267056835 + - type: euclidean_pearson + value: -4.9763689688023245 + - type: euclidean_spearman + value: -5.642707032163295 + - type: manhattan_pearson + value: -2.1980242988428276 + - type: manhattan_spearman + value: -1.854801657544592 + - task: + type: STS + dataset: + type: mteb/sts17-crosslingual-sts + name: MTEB STS17 (en-de) + config: en-de + split: test + metrics: + - type: cos_sim_pearson + value: 66.79239834758799 + - type: cos_sim_spearman + value: 67.11298548130333 + - type: euclidean_pearson + value: 66.77948456698994 + - type: euclidean_spearman + value: 67.11298548130333 + - type: manhattan_pearson + value: 66.5459479074496 + - type: manhattan_spearman + value: 66.85517071449804 + - task: + type: STS + dataset: + type: mteb/sts17-crosslingual-sts + name: MTEB STS17 (en-en) + config: en-en + split: test + metrics: + - type: cos_sim_pearson + value: 89.5692743691406 + - type: cos_sim_spearman + value: 89.56885540021487 + - type: euclidean_pearson + value: 89.78111903652413 + - type: euclidean_spearman + value: 89.56885540021487 + - type: manhattan_pearson + value: 89.68974590722112 + - type: manhattan_spearman + value: 89.40694757290255 + - task: + type: STS + dataset: + type: mteb/sts17-crosslingual-sts + name: MTEB STS17 (en-tr) + config: en-tr + split: test + metrics: + - type: cos_sim_pearson + value: 2.5434531383947427 + - type: cos_sim_spearman + value: -0.015686409614414636 + - type: euclidean_pearson + value: 3.3562612023763454 + - type: euclidean_spearman + value: -0.015686409614414636 + - type: manhattan_pearson + value: 3.06029066490911 + - type: manhattan_spearman + value: 0.9087736864115655 + - task: + type: STS + dataset: + type: mteb/sts17-crosslingual-sts + name: MTEB STS17 (es-en) + config: es-en + split: test + metrics: + - type: cos_sim_pearson + value: 48.19554059380143 + - type: cos_sim_spearman + value: 47.72387836409936 + - type: euclidean_pearson + value: 48.566966490440386 + - type: euclidean_spearman + value: 47.72387836409936 + - type: manhattan_pearson + value: 48.47970171544757 + - type: manhattan_spearman + value: 48.06448477123342 + - task: + type: STS + dataset: + type: mteb/sts17-crosslingual-sts + name: MTEB STS17 (es-es) + config: es-es + split: test + metrics: + - type: cos_sim_pearson + value: 80.72325991736295 + - type: cos_sim_spearman + value: 79.94411571627043 + - type: euclidean_pearson + value: 81.66909260117279 + - type: euclidean_spearman + value: 79.94284742229813 + - type: manhattan_pearson + value: 81.78261278000369 + - type: manhattan_spearman + value: 80.18524960358721 + - task: + type: STS + dataset: + type: mteb/sts17-crosslingual-sts + name: MTEB STS17 (fr-en) + config: fr-en + split: test + metrics: + - type: cos_sim_pearson + value: 57.385906804319916 + - type: cos_sim_spearman + value: 56.60927284389835 + - type: euclidean_pearson + value: 58.220472472555414 + - type: euclidean_spearman + value: 56.60927284389835 + - type: manhattan_pearson + value: 57.974972842168704 + - type: manhattan_spearman + value: 56.38609220634484 + - task: + type: STS + dataset: + type: mteb/sts17-crosslingual-sts + name: MTEB STS17 (it-en) + config: it-en + split: test + metrics: + - type: cos_sim_pearson + value: 33.56148532200449 + - type: cos_sim_spearman + value: 30.46169688812801 + - type: euclidean_pearson + value: 34.03749511332228 + - type: euclidean_spearman + value: 30.46169688812801 + - type: manhattan_pearson + value: 33.51842606041771 + - type: manhattan_spearman + value: 30.87826743052681 + - task: + type: STS + dataset: + type: mteb/sts17-crosslingual-sts + name: MTEB STS17 (nl-en) + config: nl-en + split: test + metrics: + - type: cos_sim_pearson + value: 37.01008110523631 + - type: cos_sim_spearman + value: 36.46124293832437 + - type: euclidean_pearson + value: 37.860431566730725 + - type: euclidean_spearman + value: 36.46124293832437 + - type: manhattan_pearson + value: 37.84974555851177 + - type: manhattan_spearman + value: 37.026498066678556 + - task: + type: STS + dataset: + type: mteb/sts22-crosslingual-sts + name: MTEB STS22 (en) + config: en + split: test + metrics: + - type: cos_sim_pearson + value: 56.563590445291226 + - type: cos_sim_spearman + value: 62.65994888539158 + - type: euclidean_pearson + value: 61.43083003163841 + - type: euclidean_spearman + value: 62.65994888539158 + - type: manhattan_pearson + value: 61.530512036243564 + - type: manhattan_spearman + value: 62.65300646176863 + - task: + type: STS + dataset: + type: mteb/sts22-crosslingual-sts + name: MTEB STS22 (de) + config: de + split: test + metrics: + - type: cos_sim_pearson + value: 28.29024604941182 + - type: cos_sim_spearman + value: 42.084625834786046 + - type: euclidean_pearson + value: 30.271611311423545 + - type: euclidean_spearman + value: 42.084625834786046 + - type: manhattan_pearson + value: 30.19034939394144 + - type: manhattan_spearman + value: 42.02260224541176 + - task: + type: STS + dataset: + type: mteb/sts22-crosslingual-sts + name: MTEB STS22 (es) + config: es + split: test + metrics: + - type: cos_sim_pearson + value: 42.79846662914459 + - type: cos_sim_spearman + value: 53.8129210907069 + - type: euclidean_pearson + value: 48.21779716691527 + - type: euclidean_spearman + value: 53.8129210907069 + - type: manhattan_pearson + value: 48.35900342355713 + - type: manhattan_spearman + value: 53.94896150957018 + - task: + type: STS + dataset: + type: mteb/sts22-crosslingual-sts + name: MTEB STS22 (pl) + config: pl + split: test + metrics: + - type: cos_sim_pearson + value: 9.818882867657061 + - type: cos_sim_spearman + value: 24.41994279795319 + - type: euclidean_pearson + value: 4.813919367736767 + - type: euclidean_spearman + value: 24.41994279795319 + - type: manhattan_pearson + value: 4.602063702670144 + - type: manhattan_spearman + value: 24.218951967147824 + - task: + type: STS + dataset: + type: mteb/sts22-crosslingual-sts + name: MTEB STS22 (tr) + config: tr + split: test + metrics: + - type: cos_sim_pearson + value: 27.410972788257048 + - type: cos_sim_spearman + value: 40.44872572093327 + - type: euclidean_pearson + value: 33.742359285090565 + - type: euclidean_spearman + value: 40.44872572093327 + - type: manhattan_pearson + value: 33.90231904900396 + - type: manhattan_spearman + value: 40.19149257794821 + - task: + type: STS + dataset: + type: mteb/sts22-crosslingual-sts + name: MTEB STS22 (ar) + config: ar + split: test + metrics: + - type: cos_sim_pearson + value: 31.3322380268429 + - type: cos_sim_spearman + value: 31.200490449337714 + - type: euclidean_pearson + value: 32.130848968646525 + - type: euclidean_spearman + value: 31.200490449337714 + - type: manhattan_pearson + value: 32.14834980954443 + - type: manhattan_spearman + value: 31.427049121627025 + - task: + type: STS + dataset: + type: mteb/sts22-crosslingual-sts + name: MTEB STS22 (ru) + config: ru + split: test + metrics: + - type: cos_sim_pearson + value: 3.4365537430337683 + - type: cos_sim_spearman + value: 12.125486695771288 + - type: euclidean_pearson + value: 8.134889656987513 + - type: euclidean_spearman + value: 12.125486695771288 + - type: manhattan_pearson + value: 8.163310600014055 + - type: manhattan_spearman + value: 12.129258700591722 + - task: + type: STS + dataset: + type: mteb/sts22-crosslingual-sts + name: MTEB STS22 (zh) + config: zh + split: test + metrics: + - type: cos_sim_pearson + value: 27.320332340418773 + - type: cos_sim_spearman + value: 32.900042162025 + - type: euclidean_pearson + value: 30.195166197236723 + - type: euclidean_spearman + value: 32.900041812396196 + - type: manhattan_pearson + value: 30.146557575933087 + - type: manhattan_spearman + value: 32.96907086076731 + - task: + type: STS + dataset: + type: mteb/sts22-crosslingual-sts + name: MTEB STS22 (fr) + config: fr + split: test + metrics: + - type: cos_sim_pearson + value: 69.2830779511937 + - type: cos_sim_spearman + value: 77.68846630995027 + - type: euclidean_pearson + value: 73.034747757096 + - type: euclidean_spearman + value: 77.68846630995027 + - type: manhattan_pearson + value: 73.03548141166142 + - type: manhattan_spearman + value: 77.65745427658017 + - task: + type: STS + dataset: + type: mteb/sts22-crosslingual-sts + name: MTEB STS22 (de-en) + config: de-en + split: test + metrics: + - type: cos_sim_pearson + value: 39.71606949409573 + - type: cos_sim_spearman + value: 46.8990508231622 + - type: euclidean_pearson + value: 46.606091669710025 + - type: euclidean_spearman + value: 46.8990508231622 + - type: manhattan_pearson + value: 46.39554347396642 + - type: manhattan_spearman + value: 46.59771734872816 + - task: + type: STS + dataset: + type: mteb/sts22-crosslingual-sts + name: MTEB STS22 (es-en) + config: es-en + split: test + metrics: + - type: cos_sim_pearson + value: 54.53158773665186 + - type: cos_sim_spearman + value: 65.18822674266846 + - type: euclidean_pearson + value: 58.19324925326185 + - type: euclidean_spearman + value: 65.18822674266846 + - type: manhattan_pearson + value: 57.83750769005698 + - type: manhattan_spearman + value: 65.02074812497972 + - task: + type: STS + dataset: + type: mteb/sts22-crosslingual-sts + name: MTEB STS22 (it) + config: it + split: test + metrics: + - type: cos_sim_pearson + value: 56.77648080772914 + - type: cos_sim_spearman + value: 60.64694762935356 + - type: euclidean_pearson + value: 58.1456140359783 + - type: euclidean_spearman + value: 60.64694762935356 + - type: manhattan_pearson + value: 58.03342495626636 + - type: manhattan_spearman + value: 60.384166246014914 + - task: + type: STS + dataset: + type: mteb/sts22-crosslingual-sts + name: MTEB STS22 (pl-en) + config: pl-en + split: test + metrics: + - type: cos_sim_pearson + value: 47.2314368564716 + - type: cos_sim_spearman + value: 42.96651621279448 + - type: euclidean_pearson + value: 47.136522518411184 + - type: euclidean_spearman + value: 42.96651621279448 + - type: manhattan_pearson + value: 48.71469489220069 + - type: manhattan_spearman + value: 44.518895193324646 + - task: + type: STS + dataset: + type: mteb/sts22-crosslingual-sts + name: MTEB STS22 (zh-en) + config: zh-en + split: test + metrics: + - type: cos_sim_pearson + value: 25.589949160802995 + - type: cos_sim_spearman + value: 20.153084379882284 + - type: euclidean_pearson + value: 26.82363451623337 + - type: euclidean_spearman + value: 20.153084379882284 + - type: manhattan_pearson + value: 25.843715884495634 + - type: manhattan_spearman + value: 18.901328744286676 + - task: + type: STS + dataset: + type: mteb/sts22-crosslingual-sts + name: MTEB STS22 (es-it) + config: es-it + split: test + metrics: + - type: cos_sim_pearson + value: 48.45790617159233 + - type: cos_sim_spearman + value: 55.28609467652911 + - type: euclidean_pearson + value: 51.88464425822175 + - type: euclidean_spearman + value: 55.28609467652911 + - type: manhattan_pearson + value: 51.815736921803136 + - type: manhattan_spearman + value: 55.33932627352348 + - task: + type: STS + dataset: + type: mteb/sts22-crosslingual-sts + name: MTEB STS22 (de-fr) + config: de-fr + split: test + metrics: + - type: cos_sim_pearson + value: 44.7093430670243 + - type: cos_sim_spearman + value: 55.04493953270152 + - type: euclidean_pearson + value: 47.90591946944973 + - type: euclidean_spearman + value: 55.04493953270152 + - type: manhattan_pearson + value: 47.964230618301606 + - type: manhattan_spearman + value: 56.09186738739794 + - task: + type: STS + dataset: + type: mteb/sts22-crosslingual-sts + name: MTEB STS22 (de-pl) + config: de-pl + split: test + metrics: + - type: cos_sim_pearson + value: 25.093485946833393 + - type: cos_sim_spearman + value: 33.93510205658959 + - type: euclidean_pearson + value: 27.454896639869027 + - type: euclidean_spearman + value: 33.93510205658959 + - type: manhattan_pearson + value: 24.299109196300538 + - type: manhattan_spearman + value: 32.51857329560673 + - task: + type: STS + dataset: + type: mteb/sts22-crosslingual-sts + name: MTEB STS22 (fr-pl) + config: fr-pl + split: test + metrics: + - type: cos_sim_pearson + value: 40.9753045484768 + - type: cos_sim_spearman + value: 28.17180849095055 + - type: euclidean_pearson + value: 40.382800203298906 + - type: euclidean_spearman + value: 28.17180849095055 + - type: manhattan_pearson + value: 34.084425723423486 + - type: manhattan_spearman + value: 28.17180849095055 + - task: + type: STS + dataset: + type: mteb/stsbenchmark-sts + name: MTEB STSBenchmark + config: default + split: test + metrics: + - type: cos_sim_pearson + value: 84.76003618726351 + - type: cos_sim_spearman + value: 85.52030817522575 + - type: euclidean_pearson + value: 85.5039926987335 + - type: euclidean_spearman + value: 85.52030817522575 + - type: manhattan_pearson + value: 85.51493965359182 + - type: manhattan_spearman + value: 85.52189380846832 + - task: + type: Reranking + dataset: + type: mteb/scidocs-reranking + name: MTEB SciDocsRR + config: default + split: test + metrics: + - type: map + value: 73.96228332723271 + - type: mrr + value: 91.34847813769382 + - task: + type: Retrieval + dataset: + type: scifact + name: MTEB SciFact + config: default + split: test + metrics: + - type: map_at_1 + value: 32.372 + - type: map_at_10 + value: 41.02 + - type: map_at_100 + value: 41.907 + - type: map_at_1000 + value: 41.967 + - type: map_at_3 + value: 38.244 + - type: map_at_5 + value: 39.786 + - type: ndcg_at_1 + value: 34.666999999999994 + - type: ndcg_at_10 + value: 45.76 + - type: ndcg_at_100 + value: 50.163999999999994 + - type: ndcg_at_1000 + value: 51.956 + - type: ndcg_at_3 + value: 40.687 + - type: ndcg_at_5 + value: 43.143 + - type: precision_at_1 + value: 34.666999999999994 + - type: precision_at_10 + value: 6.7 + - type: precision_at_100 + value: 0.907 + - type: precision_at_1000 + value: 0.107 + - type: precision_at_3 + value: 16.667 + - type: precision_at_5 + value: 11.466999999999999 + - type: recall_at_1 + value: 32.372 + - type: recall_at_10 + value: 59.061 + - type: recall_at_100 + value: 79.733 + - type: recall_at_1000 + value: 94.167 + - type: recall_at_3 + value: 45.161 + - type: recall_at_5 + value: 51.439 + - task: + type: PairClassification + dataset: + type: mteb/sprintduplicatequestions-pairclassification + name: MTEB SprintDuplicateQuestions + config: default + split: test + metrics: + - type: cos_sim_accuracy + value: 99.6960396039604 + - type: cos_sim_ap + value: 91.22814257221482 + - type: cos_sim_f1 + value: 84.43775100401606 + - type: cos_sim_precision + value: 84.77822580645162 + - type: cos_sim_recall + value: 84.1 + - type: dot_accuracy + value: 99.6960396039604 + - type: dot_ap + value: 91.22814257221482 + - type: dot_f1 + value: 84.43775100401606 + - type: dot_precision + value: 84.77822580645162 + - type: dot_recall + value: 84.1 + - type: euclidean_accuracy + value: 99.6960396039604 + - type: euclidean_ap + value: 91.22814257221482 + - type: euclidean_f1 + value: 84.43775100401606 + - type: euclidean_precision + value: 84.77822580645162 + - type: euclidean_recall + value: 84.1 + - type: manhattan_accuracy + value: 99.6960396039604 + - type: manhattan_ap + value: 91.18887077921163 + - type: manhattan_f1 + value: 84.27991886409735 + - type: manhattan_precision + value: 85.49382716049382 + - type: manhattan_recall + value: 83.1 + - type: max_accuracy + value: 99.6960396039604 + - type: max_ap + value: 91.22814257221482 + - type: max_f1 + value: 84.43775100401606 + - task: + type: Clustering + dataset: + type: mteb/stackexchange-clustering + name: MTEB StackExchangeClustering + config: default + split: test + metrics: + - type: v_measure + value: 63.13072579524015 + - task: + type: Clustering + dataset: + type: mteb/stackexchange-clustering-p2p + name: MTEB StackExchangeClusteringP2P + config: default + split: test + metrics: + - type: v_measure + value: 35.681141375580225 + - task: + type: Reranking + dataset: + type: mteb/stackoverflowdupquestions-reranking + name: MTEB StackOverflowDupQuestions + config: default + split: test + metrics: + - type: map + value: 48.46269194141537 + - type: mrr + value: 49.11958343943638 + - task: + type: Summarization + dataset: + type: mteb/summeval + name: MTEB SummEval + config: default + split: test + metrics: + - type: cos_sim_pearson + value: 30.709572612837498 + - type: cos_sim_spearman + value: 31.3940211538976 + - type: dot_pearson + value: 30.709578240668765 + - type: dot_spearman + value: 31.3940211538976 + - task: + type: Retrieval + dataset: + type: trec-covid + name: MTEB TRECCOVID + config: default + split: test + metrics: + - type: map_at_1 + value: 0.151 + - type: map_at_10 + value: 0.822 + - type: map_at_100 + value: 4.846 + - type: map_at_1000 + value: 13.117 + - type: map_at_3 + value: 0.349 + - type: map_at_5 + value: 0.49500000000000005 + - type: ndcg_at_1 + value: 48.0 + - type: ndcg_at_10 + value: 40.699000000000005 + - type: ndcg_at_100 + value: 35.455 + - type: ndcg_at_1000 + value: 35.067 + - type: ndcg_at_3 + value: 44.519999999999996 + - type: ndcg_at_5 + value: 42.697 + - type: precision_at_1 + value: 54.0 + - type: precision_at_10 + value: 44.0 + - type: precision_at_100 + value: 37.72 + - type: precision_at_1000 + value: 16.302 + - type: precision_at_3 + value: 50.0 + - type: precision_at_5 + value: 47.199999999999996 + - type: recall_at_1 + value: 0.151 + - type: recall_at_10 + value: 1.109 + - type: recall_at_100 + value: 8.644 + - type: recall_at_1000 + value: 34.566 + - type: recall_at_3 + value: 0.394 + - type: recall_at_5 + value: 0.601 + - task: + type: Retrieval + dataset: + type: webis-touche2020 + name: MTEB Touche2020 + config: default + split: test + metrics: + - type: map_at_1 + value: 1.786 + - type: map_at_10 + value: 8.379 + - type: map_at_100 + value: 13.618 + - type: map_at_1000 + value: 15.15 + - type: map_at_3 + value: 3.7900000000000005 + - type: map_at_5 + value: 6.1530000000000005 + - type: ndcg_at_1 + value: 19.387999999999998 + - type: ndcg_at_10 + value: 20.296 + - type: ndcg_at_100 + value: 31.828 + - type: ndcg_at_1000 + value: 43.968 + - type: ndcg_at_3 + value: 19.583000000000002 + - type: ndcg_at_5 + value: 21.066 + - type: precision_at_1 + value: 22.448999999999998 + - type: precision_at_10 + value: 19.592000000000002 + - type: precision_at_100 + value: 7.041 + - type: precision_at_1000 + value: 1.49 + - type: precision_at_3 + value: 22.448999999999998 + - type: precision_at_5 + value: 24.490000000000002 + - type: recall_at_1 + value: 1.786 + - type: recall_at_10 + value: 14.571000000000002 + - type: recall_at_100 + value: 44.247 + - type: recall_at_1000 + value: 80.36 + - type: recall_at_3 + value: 5.117 + - type: recall_at_5 + value: 9.449 + - task: + type: Classification + dataset: + type: mteb/toxic_conversations_50k + name: MTEB ToxicConversationsClassification + config: default + split: test + metrics: + - type: accuracy + value: 68.19919999999999 + - type: ap + value: 14.328836562980976 + - type: f1 + value: 53.33893474325896 + - task: + type: Classification + dataset: + type: mteb/tweet_sentiment_extraction + name: MTEB TweetSentimentExtractionClassification + config: default + split: test + metrics: + - type: accuracy + value: 62.71080928126768 + - type: f1 + value: 62.35221892617029 + - task: + type: Clustering + dataset: + type: mteb/twentynewsgroups-clustering + name: MTEB TwentyNewsgroupsClustering + config: default + split: test + metrics: + - type: v_measure + value: 48.099101871064484 + - task: + type: PairClassification + dataset: + type: mteb/twittersemeval2015-pairclassification + name: MTEB TwitterSemEval2015 + config: default + split: test + metrics: + - type: cos_sim_accuracy + value: 87.60207426834357 + - type: cos_sim_ap + value: 78.25096573546108 + - type: cos_sim_f1 + value: 71.740233384069 + - type: cos_sim_precision + value: 69.07669760625306 + - type: cos_sim_recall + value: 74.6174142480211 + - type: dot_accuracy + value: 87.60207426834357 + - type: dot_ap + value: 78.25097910093768 + - type: dot_f1 + value: 71.740233384069 + - type: dot_precision + value: 69.07669760625306 + - type: dot_recall + value: 74.6174142480211 + - type: euclidean_accuracy + value: 87.60207426834357 + - type: euclidean_ap + value: 78.25097099603116 + - type: euclidean_f1 + value: 71.740233384069 + - type: euclidean_precision + value: 69.07669760625306 + - type: euclidean_recall + value: 74.6174142480211 + - type: manhattan_accuracy + value: 87.61399535077786 + - type: manhattan_ap + value: 78.238484943708 + - type: manhattan_f1 + value: 71.77797490812317 + - type: manhattan_precision + value: 69.05632772494513 + - type: manhattan_recall + value: 74.72295514511873 + - type: max_accuracy + value: 87.61399535077786 + - type: max_ap + value: 78.25097910093768 + - type: max_f1 + value: 71.77797490812317 + - task: + type: PairClassification + dataset: + type: mteb/twitterurlcorpus-pairclassification + name: MTEB TwitterURLCorpus + config: default + split: test + metrics: + - type: cos_sim_accuracy + value: 89.17413746264602 + - type: cos_sim_ap + value: 86.04575990028458 + - type: cos_sim_f1 + value: 78.52034894604814 + - type: cos_sim_precision + value: 76.42300123897675 + - type: cos_sim_recall + value: 80.73606405913151 + - type: dot_accuracy + value: 89.17413746264602 + - type: dot_ap + value: 86.04575880500646 + - type: dot_f1 + value: 78.52034894604814 + - type: dot_precision + value: 76.42300123897675 + - type: dot_recall + value: 80.73606405913151 + - type: euclidean_accuracy + value: 89.17413746264602 + - type: euclidean_ap + value: 86.04575106124874 + - type: euclidean_f1 + value: 78.52034894604814 + - type: euclidean_precision + value: 76.42300123897675 + - type: euclidean_recall + value: 80.73606405913151 + - type: manhattan_accuracy + value: 89.14891139830016 + - type: manhattan_ap + value: 86.01748033351211 + - type: manhattan_f1 + value: 78.48817724818471 + - type: manhattan_precision + value: 76.00057690920892 + - type: manhattan_recall + value: 81.14413304588851 + - type: max_accuracy + value: 89.17413746264602 + - type: max_ap + value: 86.04575990028458 + - type: max_f1 + value: 78.52034894604814 +--- + +# sentence-transformers/sentence-t5-base + +This is a [sentence-transformers](https://www.SBERT.net) model: It maps sentences & paragraphs to a 768 dimensional dense vector space. The model works well for sentence similarity tasks, but doesn't perform that well for semantic search tasks. + +This model was converted from the Tensorflow model [st5-base-1](https://tfhub.dev/google/sentence-t5/st5-base/1) to PyTorch. When using this model, have a look at the publication: [Sentence-T5: Scalable sentence encoders from pre-trained text-to-text models](https://arxiv.org/abs/2108.08877). The tfhub model and this PyTorch model can produce slightly different embeddings, however, when run on the same benchmarks, they produce identical results. + +The model uses only the encoder from a T5-base model. The weights are stored in FP16. + + +## Usage (Sentence-Transformers) + +Using this model becomes easy when you have [sentence-transformers](https://www.SBERT.net) installed: + +``` +pip install -U sentence-transformers +``` + +Then you can use the model like this: + +```python +from sentence_transformers import SentenceTransformer +sentences = ["This is an example sentence", "Each sentence is converted"] + +model = SentenceTransformer('sentence-transformers/sentence-t5-base') +embeddings = model.encode(sentences) +print(embeddings) +``` + +The model requires sentence-transformers version 2.2.0 or newer. + +## Evaluation Results + +For an automated evaluation of this model, see the *Sentence Embeddings Benchmark*: [https://seb.sbert.net](https://seb.sbert.net?model_name=sentence-transformers/sentence-t5-base) + + + +## Citing & Authors + +If you find this model helpful, please cite the respective publication: +[Sentence-T5: Scalable sentence encoders from pre-trained text-to-text models](https://arxiv.org/abs/2108.08877)