--- library_name: peft license: mit language: - en pipeline_tag: sentence-similarity tags: - text-embedding - embeddings - information-retrieval - beir - text-classification - language-model - text-clustering - text-semantic-similarity - text-evaluation - text-reranking - feature-extraction - sentence-similarity - Sentence Similarity - natural_questions - ms_marco - fever - hotpot_qa - mteb model-index: - name: BinGSE-Meta-Llama-3-8B-Instruct results: - dataset: config: en-ext name: MTEB AmazonCounterfactualClassification (en-ext) revision: e8379541af4e31359cca9fbcf4b00f2671dba205 split: test type: mteb/amazon_counterfactual metrics: - type: accuracy value: 80.7271364317841 - type: ap value: 29.57781615779065 - type: ap_weighted value: 29.57781615779065 - type: f1 value: 67.88722644497633 - type: f1_weighted value: 83.93210384487763 - type: main_score value: 80.7271364317841 task: type: Classification - dataset: config: en name: MTEB AmazonCounterfactualClassification (en) revision: e8379541af4e31359cca9fbcf4b00f2671dba205 split: test type: mteb/amazon_counterfactual metrics: - type: accuracy value: 80.41791044776122 - type: ap value: 44.865115567829 - type: ap_weighted value: 44.865115567829 - type: f1 value: 74.51584838607613 - type: f1_weighted value: 81.95697646844347 - type: main_score value: 80.41791044776122 task: type: Classification - dataset: config: default name: MTEB AmazonPolarityClassification (default) revision: e2d317d38cd51312af73b3d32a06d1a08b442046 split: test type: mteb/amazon_polarity metrics: - type: accuracy value: 91.384925 - type: ap value: 87.67370574947891 - type: ap_weighted value: 87.67370574947891 - type: f1 value: 91.37299490898192 - type: f1_weighted value: 91.37299490898194 - type: main_score value: 91.384925 task: type: Classification - dataset: config: en name: MTEB AmazonReviewsClassification (en) revision: 1399c76144fd37290681b995c656ef9b2e06e26d split: test type: mteb/amazon_reviews_multi metrics: - type: accuracy value: 51.532 - type: f1 value: 49.493931716627664 - type: f1_weighted value: 49.49393171662767 - type: main_score value: 51.532 task: type: Classification - dataset: config: default name: MTEB ArguAna (default) revision: c22ab2a51041ffd869aaddef7af8d8215647e41a split: test type: mteb/arguana metrics: - type: main_score value: 60.614000000000004 - type: map_at_1 value: 36.486000000000004 - type: map_at_10 value: 51.995999999999995 - type: map_at_100 value: 52.662 - type: map_at_1000 value: 52.664 - type: map_at_20 value: 52.563 - type: map_at_3 value: 47.321000000000005 - type: map_at_5 value: 49.864000000000004 - type: mrr_at_1 value: 37.12660028449502 - type: mrr_at_10 value: 52.21801688906961 - type: mrr_at_100 value: 52.896624713480335 - type: mrr_at_1000 value: 52.899139810952356 - type: mrr_at_20 value: 52.798368866276725 - type: mrr_at_3 value: 47.629208155524005 - type: mrr_at_5 value: 50.111427216690494 - type: nauc_map_at_1000_diff1 value: 16.54957264421337 - type: nauc_map_at_1000_max value: -10.64627215347515 - type: nauc_map_at_1000_std value: -14.791627421585465 - type: nauc_map_at_100_diff1 value: 16.55389871501292 - type: nauc_map_at_100_max value: -10.639269449448072 - type: nauc_map_at_100_std value: -14.787356497788629 - type: nauc_map_at_10_diff1 value: 16.260442894074867 - type: nauc_map_at_10_max value: -10.520708455145465 - type: nauc_map_at_10_std value: -14.844381831416225 - type: nauc_map_at_1_diff1 value: 20.37562625243646 - type: nauc_map_at_1_max value: -12.353325207525819 - type: nauc_map_at_1_std value: -14.932651302657812 - type: nauc_map_at_20_diff1 value: 16.511576195172182 - type: nauc_map_at_20_max value: -10.536480484448699 - type: nauc_map_at_20_std value: -14.800613000367957 - type: nauc_map_at_3_diff1 value: 16.457822965971268 - type: nauc_map_at_3_max value: -11.122597198482092 - type: nauc_map_at_3_std value: -15.062841168614415 - type: nauc_map_at_5_diff1 value: 16.14704834191165 - type: nauc_map_at_5_max value: -10.66777729065394 - type: nauc_map_at_5_std value: -15.104801469981211 - type: nauc_mrr_at_1000_diff1 value: 14.93701738531105 - type: nauc_mrr_at_1000_max value: -11.000813726208749 - type: nauc_mrr_at_1000_std value: -14.934720206805101 - type: nauc_mrr_at_100_diff1 value: 14.941465999801288 - type: nauc_mrr_at_100_max value: -10.993801241066862 - type: nauc_mrr_at_100_std value: -14.930449432226991 - type: nauc_mrr_at_10_diff1 value: 14.633163813809874 - type: nauc_mrr_at_10_max value: -10.935512160642284 - type: nauc_mrr_at_10_std value: -15.003188811040838 - type: nauc_mrr_at_1_diff1 value: 18.52403185373208 - type: nauc_mrr_at_1_max value: -12.2706958053697 - type: nauc_mrr_at_1_std value: -15.058866493865686 - type: nauc_mrr_at_20_diff1 value: 14.906670909299184 - type: nauc_mrr_at_20_max value: -10.889049584699048 - type: nauc_mrr_at_20_std value: -14.943130609646857 - type: nauc_mrr_at_3_diff1 value: 14.85258148742534 - type: nauc_mrr_at_3_max value: -11.348348481615957 - type: nauc_mrr_at_3_std value: -15.318961044698176 - type: nauc_mrr_at_5_diff1 value: 14.67396557076444 - type: nauc_mrr_at_5_max value: -10.99093305536558 - type: nauc_mrr_at_5_std value: -15.233042085392126 - type: nauc_ndcg_at_1000_diff1 value: 16.226732398924902 - type: nauc_ndcg_at_1000_max value: -10.065730266576361 - type: nauc_ndcg_at_1000_std value: -14.42407866611729 - type: nauc_ndcg_at_100_diff1 value: 16.35087560433341 - type: nauc_ndcg_at_100_max value: -9.864016297887359 - type: nauc_ndcg_at_100_std value: -14.299246150380066 - type: nauc_ndcg_at_10_diff1 value: 15.05184654417688 - type: nauc_ndcg_at_10_max value: -9.026418167156756 - type: nauc_ndcg_at_10_std value: -14.568745179306841 - type: nauc_ndcg_at_1_diff1 value: 20.37562625243646 - type: nauc_ndcg_at_1_max value: -12.353325207525819 - type: nauc_ndcg_at_1_std value: -14.932651302657812 - type: nauc_ndcg_at_20_diff1 value: 15.988646263128512 - type: nauc_ndcg_at_20_max value: -9.062147071547747 - type: nauc_ndcg_at_20_std value: -14.451291030823132 - type: nauc_ndcg_at_3_diff1 value: 15.538693605375215 - type: nauc_ndcg_at_3_max value: -10.558994635661737 - type: nauc_ndcg_at_3_std value: -15.133606718974256 - type: nauc_ndcg_at_5_diff1 value: 14.879923585596973 - type: nauc_ndcg_at_5_max value: -9.67116891074767 - type: nauc_ndcg_at_5_std value: -15.21913397301524 - type: nauc_precision_at_1000_diff1 value: 24.94025537262749 - type: nauc_precision_at_1000_max value: 7.294585511019708 - type: nauc_precision_at_1000_std value: 77.3551884133584 - type: nauc_precision_at_100_diff1 value: 48.11981108528576 - type: nauc_precision_at_100_max value: 43.70602771306187 - type: nauc_precision_at_100_std value: 56.044797764814234 - type: nauc_precision_at_10_diff1 value: 6.245240526102324 - type: nauc_precision_at_10_max value: 3.7092123005872266 - type: nauc_precision_at_10_std value: -11.966012317352538 - type: nauc_precision_at_1_diff1 value: 20.37562625243646 - type: nauc_precision_at_1_max value: -12.353325207525819 - type: nauc_precision_at_1_std value: -14.932651302657812 - type: nauc_precision_at_20_diff1 value: 10.989453992590228 - type: nauc_precision_at_20_max value: 24.73249690000041 - type: nauc_precision_at_20_std value: -5.731495712436961 - type: nauc_precision_at_3_diff1 value: 12.762541849206247 - type: nauc_precision_at_3_max value: -8.721247930835114 - type: nauc_precision_at_3_std value: -15.367166413221348 - type: nauc_precision_at_5_diff1 value: 10.068145026446269 - type: nauc_precision_at_5_max value: -5.62804283853098 - type: nauc_precision_at_5_std value: -15.68637016204708 - type: nauc_recall_at_1000_diff1 value: 24.940255372624858 - type: nauc_recall_at_1000_max value: 7.2945855110158515 - type: nauc_recall_at_1000_std value: 77.35518841335626 - type: nauc_recall_at_100_diff1 value: 48.119811085282684 - type: nauc_recall_at_100_max value: 43.70602771306109 - type: nauc_recall_at_100_std value: 56.04479776481617 - type: nauc_recall_at_10_diff1 value: 6.2452405261024495 - type: nauc_recall_at_10_max value: 3.7092123005874145 - type: nauc_recall_at_10_std value: -11.966012317352225 - type: nauc_recall_at_1_diff1 value: 20.37562625243646 - type: nauc_recall_at_1_max value: -12.353325207525819 - type: nauc_recall_at_1_std value: -14.932651302657812 - type: nauc_recall_at_20_diff1 value: 10.989453992591187 - type: nauc_recall_at_20_max value: 24.73249690000033 - type: nauc_recall_at_20_std value: -5.73149571243645 - type: nauc_recall_at_3_diff1 value: 12.762541849206283 - type: nauc_recall_at_3_max value: -8.721247930835077 - type: nauc_recall_at_3_std value: -15.367166413221279 - type: nauc_recall_at_5_diff1 value: 10.068145026446292 - type: nauc_recall_at_5_max value: -5.628042838530992 - type: nauc_recall_at_5_std value: -15.686370162047094 - type: ndcg_at_1 value: 36.486000000000004 - type: ndcg_at_10 value: 60.614000000000004 - type: ndcg_at_100 value: 63.243 - type: ndcg_at_1000 value: 63.3 - type: ndcg_at_20 value: 62.598 - type: ndcg_at_3 value: 50.909000000000006 - type: ndcg_at_5 value: 55.47 - type: precision_at_1 value: 36.486000000000004 - type: precision_at_10 value: 8.819 - type: precision_at_100 value: 0.992 - type: precision_at_1000 value: 0.1 - type: precision_at_20 value: 4.7940000000000005 - type: precision_at_3 value: 20.436 - type: precision_at_5 value: 14.466999999999999 - type: recall_at_1 value: 36.486000000000004 - type: recall_at_10 value: 88.193 - type: recall_at_100 value: 99.21799999999999 - type: recall_at_1000 value: 99.644 - type: recall_at_20 value: 95.875 - type: recall_at_3 value: 61.309000000000005 - type: recall_at_5 value: 72.333 task: type: Retrieval - dataset: config: default name: MTEB ArxivClusteringP2P (default) revision: a122ad7f3f0291bf49cc6f4d32aa80929df69d5d split: test type: mteb/arxiv-clustering-p2p metrics: - type: main_score value: 47.89537370883614 - type: v_measure value: 47.89537370883614 - type: v_measure_std value: 13.564912043981685 task: type: Clustering - dataset: config: default name: MTEB ArxivClusteringS2S (default) revision: f910caf1a6075f7329cdf8c1a6135696f37dbd53 split: test type: mteb/arxiv-clustering-s2s metrics: - type: main_score value: 46.316519519112575 - type: v_measure value: 46.316519519112575 - type: v_measure_std value: 14.064564320172318 task: type: Clustering - dataset: config: default name: MTEB AskUbuntuDupQuestions (default) revision: 2000358ca161889fa9c082cb41daa8dcfb161a54 split: test type: mteb/askubuntudupquestions-reranking metrics: - type: main_score value: 65.09483223839607 - type: map value: 65.09483223839607 - type: mrr value: 77.75601283911533 - type: nAUC_map_diff1 value: 12.614852005743735 - type: nAUC_map_max value: 29.257344662071027 - type: nAUC_map_std value: 17.630286672870287 - type: nAUC_mrr_diff1 value: 16.314189417460618 - type: nAUC_mrr_max value: 39.68682288371764 - type: nAUC_mrr_std value: 22.85236267444885 task: type: Reranking - dataset: config: default name: MTEB BIOSSES (default) revision: d3fb88f8f02e40887cd149695127462bbcf29b4a split: test type: mteb/biosses-sts metrics: - type: cosine_pearson value: 87.64275539432272 - type: cosine_spearman value: 86.93752309911496 - type: euclidean_pearson value: 85.76812373084148 - type: euclidean_spearman value: 86.93752309911496 - type: main_score value: 86.93752309911496 - type: manhattan_pearson value: 85.66299640283663 - type: manhattan_spearman value: 86.79053179801122 - type: pearson value: 87.64276222909432 - type: spearman value: 86.93752309911496 task: type: STS - dataset: config: default name: MTEB Banking77Classification (default) revision: 0fd18e25b25c072e09e0d92ab615fda904d66300 split: test type: mteb/banking77 metrics: - type: accuracy value: 84.83116883116884 - type: f1 value: 84.33922428309117 - type: f1_weighted value: 84.33922428309116 - type: main_score value: 84.83116883116884 task: type: Classification - dataset: config: default name: MTEB BiorxivClusteringP2P (default) revision: 65b79d1d13f80053f67aca9498d9402c2d9f1f40 split: test type: mteb/biorxiv-clustering-p2p metrics: - type: main_score value: 36.931855182990965 - type: v_measure value: 36.931855182990965 - type: v_measure_std value: 1.259241362575525 task: type: Clustering - dataset: config: default name: MTEB BiorxivClusteringS2S (default) revision: 258694dd0231531bc1fd9de6ceb52a0853c6d908 split: test type: mteb/biorxiv-clustering-s2s metrics: - type: main_score value: 38.276834717269345 - type: v_measure value: 38.276834717269345 - type: v_measure_std value: 0.8171217218107112 task: type: Clustering - dataset: config: default name: MTEB CQADupstackAndroidRetrieval (default) revision: f46a197baaae43b4f621051089b82a364682dfeb split: test type: mteb/cqadupstack-android metrics: - type: main_score value: 53.199 - type: map_at_1 value: 32.222 - type: map_at_10 value: 45.985 - type: map_at_100 value: 47.781 - type: map_at_1000 value: 47.886 - type: map_at_20 value: 47.14 - type: map_at_3 value: 41.934 - type: map_at_5 value: 44.204 - type: mrr_at_1 value: 40.486409155937054 - type: mrr_at_10 value: 51.97288643640573 - type: mrr_at_100 value: 52.82594639688508 - type: mrr_at_1000 value: 52.84957608989007 - type: mrr_at_20 value: 52.56262908663282 - type: mrr_at_3 value: 49.23700524558894 - type: mrr_at_5 value: 50.939437291368606 - type: nauc_map_at_1000_diff1 value: 48.71274550798233 - type: nauc_map_at_1000_max value: 40.02571684078115 - type: nauc_map_at_1000_std value: -10.240607880266495 - type: nauc_map_at_100_diff1 value: 48.719420702404356 - type: nauc_map_at_100_max value: 40.04375889384823 - type: nauc_map_at_100_std value: -10.189770818022746 - type: nauc_map_at_10_diff1 value: 49.002632608732064 - type: nauc_map_at_10_max value: 39.66152953772287 - type: nauc_map_at_10_std value: -10.928551613658987 - type: nauc_map_at_1_diff1 value: 56.081055621434594 - type: nauc_map_at_1_max value: 37.39452905849337 - type: nauc_map_at_1_std value: -12.23297277896674 - type: nauc_map_at_20_diff1 value: 48.75827704614257 - type: nauc_map_at_20_max value: 40.07875211936587 - type: nauc_map_at_20_std value: -10.4773380783476 - type: nauc_map_at_3_diff1 value: 50.266618647494155 - type: nauc_map_at_3_max value: 38.476151589975906 - type: nauc_map_at_3_std value: -12.63236531657738 - type: nauc_map_at_5_diff1 value: 49.704868442016455 - type: nauc_map_at_5_max value: 39.36443082562045 - type: nauc_map_at_5_std value: -11.503767300692095 - type: nauc_mrr_at_1000_diff1 value: 45.364986177189174 - type: nauc_mrr_at_1000_max value: 39.46602627253264 - type: nauc_mrr_at_1000_std value: -10.02957389947249 - type: nauc_mrr_at_100_diff1 value: 45.357268243765816 - type: nauc_mrr_at_100_max value: 39.47219135072728 - type: nauc_mrr_at_100_std value: -10.026695043309909 - type: nauc_mrr_at_10_diff1 value: 45.244273933517626 - type: nauc_mrr_at_10_max value: 39.20945693162985 - type: nauc_mrr_at_10_std value: -9.90767794899639 - type: nauc_mrr_at_1_diff1 value: 49.20992505931076 - type: nauc_mrr_at_1_max value: 39.46297723636049 - type: nauc_mrr_at_1_std value: -11.981411959321381 - type: nauc_mrr_at_20_diff1 value: 45.227086207075025 - type: nauc_mrr_at_20_max value: 39.41746276244367 - type: nauc_mrr_at_20_std value: -10.052067549951015 - type: nauc_mrr_at_3_diff1 value: 45.911580785062874 - type: nauc_mrr_at_3_max value: 39.424633318900945 - type: nauc_mrr_at_3_std value: -11.410883935710286 - type: nauc_mrr_at_5_diff1 value: 45.55756954242934 - type: nauc_mrr_at_5_max value: 39.40058471175011 - type: nauc_mrr_at_5_std value: -10.214441983669223 - type: nauc_ndcg_at_1000_diff1 value: 46.057039050302976 - type: nauc_ndcg_at_1000_max value: 39.957590491002165 - type: nauc_ndcg_at_1000_std value: -8.201442473200322 - type: nauc_ndcg_at_100_diff1 value: 45.69693794270335 - type: nauc_ndcg_at_100_max value: 40.36215332476892 - type: nauc_ndcg_at_100_std value: -7.820701191661731 - type: nauc_ndcg_at_10_diff1 value: 46.01752520382891 - type: nauc_ndcg_at_10_max value: 39.40935337515678 - type: nauc_ndcg_at_10_std value: -9.196768256986553 - type: nauc_ndcg_at_1_diff1 value: 49.20992505931076 - type: nauc_ndcg_at_1_max value: 39.46297723636049 - type: nauc_ndcg_at_1_std value: -11.981411959321381 - type: nauc_ndcg_at_20_diff1 value: 45.24349036182319 - type: nauc_ndcg_at_20_max value: 40.15373766029114 - type: nauc_ndcg_at_20_std value: -9.033362151661638 - type: nauc_ndcg_at_3_diff1 value: 47.56680584050923 - type: nauc_ndcg_at_3_max value: 39.202195898427604 - type: nauc_ndcg_at_3_std value: -11.785092866176829 - type: nauc_ndcg_at_5_diff1 value: 46.96982794214046 - type: nauc_ndcg_at_5_max value: 39.5663039638241 - type: nauc_ndcg_at_5_std value: -10.04391381207476 - type: nauc_precision_at_1000_diff1 value: -23.484781013384755 - type: nauc_precision_at_1000_max value: -10.775814504019596 - type: nauc_precision_at_1000_std value: -1.3598420879011879 - type: nauc_precision_at_100_diff1 value: -18.04401254090738 - type: nauc_precision_at_100_max value: -1.6777370966059024 - type: nauc_precision_at_100_std value: 9.607002317088027 - type: nauc_precision_at_10_diff1 value: 0.1751093583673089 - type: nauc_precision_at_10_max value: 16.8907295462174 - type: nauc_precision_at_10_std value: 4.728800492032401 - type: nauc_precision_at_1_diff1 value: 49.20992505931076 - type: nauc_precision_at_1_max value: 39.46297723636049 - type: nauc_precision_at_1_std value: -11.981411959321381 - type: nauc_precision_at_20_diff1 value: -9.418679223786228 - type: nauc_precision_at_20_max value: 9.594105141089766 - type: nauc_precision_at_20_std value: 8.983281264416412 - type: nauc_precision_at_3_diff1 value: 22.109157571511133 - type: nauc_precision_at_3_max value: 30.005578686840945 - type: nauc_precision_at_3_std value: -7.6600405648564625 - type: nauc_precision_at_5_diff1 value: 13.044712488888747 - type: nauc_precision_at_5_max value: 26.320322603149442 - type: nauc_precision_at_5_std value: -0.7111330736852223 - type: nauc_recall_at_1000_diff1 value: 24.747399774138678 - type: nauc_recall_at_1000_max value: 49.933416319505206 - type: nauc_recall_at_1000_std value: 65.92307305058418 - type: nauc_recall_at_100_diff1 value: 25.510369747526664 - type: nauc_recall_at_100_max value: 45.23205448057407 - type: nauc_recall_at_100_std value: 16.57376782068154 - type: nauc_recall_at_10_diff1 value: 37.08971306710978 - type: nauc_recall_at_10_max value: 35.425313950091024 - type: nauc_recall_at_10_std value: -4.935344752233474 - type: nauc_recall_at_1_diff1 value: 56.081055621434594 - type: nauc_recall_at_1_max value: 37.39452905849337 - type: nauc_recall_at_1_std value: -12.23297277896674 - type: nauc_recall_at_20_diff1 value: 30.938967468918406 - type: nauc_recall_at_20_max value: 38.252828702839174 - type: nauc_recall_at_20_std value: -3.938967991713323 - type: nauc_recall_at_3_diff1 value: 43.990804769126264 - type: nauc_recall_at_3_max value: 35.2863590405335 - type: nauc_recall_at_3_std value: -12.095798082391006 - type: nauc_recall_at_5_diff1 value: 41.106768176487826 - type: nauc_recall_at_5_max value: 35.66847913821838 - type: nauc_recall_at_5_std value: -7.937702109801549 - type: ndcg_at_1 value: 40.486 - type: ndcg_at_10 value: 53.199 - type: ndcg_at_100 value: 58.901 - type: ndcg_at_1000 value: 60.155 - type: ndcg_at_20 value: 56.068 - type: ndcg_at_3 value: 47.781 - type: ndcg_at_5 value: 50.275000000000006 - type: precision_at_1 value: 40.486 - type: precision_at_10 value: 10.501000000000001 - type: precision_at_100 value: 1.657 - type: precision_at_1000 value: 0.208 - type: precision_at_20 value: 6.438000000000001 - type: precision_at_3 value: 23.51 - type: precision_at_5 value: 16.881 - type: recall_at_1 value: 32.222 - type: recall_at_10 value: 67.36 - type: recall_at_100 value: 90.171 - type: recall_at_1000 value: 97.556 - type: recall_at_20 value: 77.486 - type: recall_at_3 value: 51.298 - type: recall_at_5 value: 58.594 task: type: Retrieval - dataset: config: default name: MTEB CQADupstackEnglishRetrieval (default) revision: ad9991cb51e31e31e430383c75ffb2885547b5f0 split: test type: mteb/cqadupstack-english metrics: - type: main_score value: 52.510999999999996 - type: map_at_1 value: 33.87 - type: map_at_10 value: 46.26 - type: map_at_100 value: 47.587 - type: map_at_1000 value: 47.703 - type: map_at_20 value: 46.997 - type: map_at_3 value: 42.609 - type: map_at_5 value: 44.734 - type: mrr_at_1 value: 42.611464968152866 - type: mrr_at_10 value: 52.04279142654943 - type: mrr_at_100 value: 52.65530619466451 - type: mrr_at_1000 value: 52.68759711579288 - type: mrr_at_20 value: 52.43781678281101 - type: mrr_at_3 value: 49.596602972399204 - type: mrr_at_5 value: 51.08067940552024 - type: nauc_map_at_1000_diff1 value: 51.29568684244711 - type: nauc_map_at_1000_max value: 39.152103977041804 - type: nauc_map_at_1000_std value: -5.181397622387878 - type: nauc_map_at_100_diff1 value: 51.32176208688315 - type: nauc_map_at_100_max value: 39.143917385553216 - type: nauc_map_at_100_std value: -5.307065518594916 - type: nauc_map_at_10_diff1 value: 51.85811251362487 - type: nauc_map_at_10_max value: 39.15129087396741 - type: nauc_map_at_10_std value: -6.955630370961674 - type: nauc_map_at_1_diff1 value: 58.70495869436133 - type: nauc_map_at_1_max value: 32.656882457708946 - type: nauc_map_at_1_std value: -12.67248330614144 - type: nauc_map_at_20_diff1 value: 51.643097152908055 - type: nauc_map_at_20_max value: 39.22360058918707 - type: nauc_map_at_20_std value: -6.214645092428745 - type: nauc_map_at_3_diff1 value: 52.83744216264604 - type: nauc_map_at_3_max value: 38.58401409973185 - type: nauc_map_at_3_std value: -9.412909859337919 - type: nauc_map_at_5_diff1 value: 52.14367884906982 - type: nauc_map_at_5_max value: 38.54068123115184 - type: nauc_map_at_5_std value: -8.113232464503577 - type: nauc_mrr_at_1000_diff1 value: 49.32487078343284 - type: nauc_mrr_at_1000_max value: 39.92763641079578 - type: nauc_mrr_at_1000_std value: -0.1863128505082372 - type: nauc_mrr_at_100_diff1 value: 49.31353319032606 - type: nauc_mrr_at_100_max value: 39.93613554648638 - type: nauc_mrr_at_100_std value: -0.17243538819115126 - type: nauc_mrr_at_10_diff1 value: 49.32045332332676 - type: nauc_mrr_at_10_max value: 39.87692178853214 - type: nauc_mrr_at_10_std value: -0.44124064115092854 - type: nauc_mrr_at_1_diff1 value: 54.263136864624485 - type: nauc_mrr_at_1_max value: 37.608424271153126 - type: nauc_mrr_at_1_std value: -4.123286617311634 - type: nauc_mrr_at_20_diff1 value: 49.33965494151155 - type: nauc_mrr_at_20_max value: 39.990788764214905 - type: nauc_mrr_at_20_std value: -0.22811808153329785 - type: nauc_mrr_at_3_diff1 value: 49.883512068525214 - type: nauc_mrr_at_3_max value: 40.17194283092971 - type: nauc_mrr_at_3_std value: -0.9636794529443634 - type: nauc_mrr_at_5_diff1 value: 49.26161551090732 - type: nauc_mrr_at_5_max value: 39.901214925003316 - type: nauc_mrr_at_5_std value: -0.4175685997548143 - type: nauc_ndcg_at_1000_diff1 value: 48.455643184927396 - type: nauc_ndcg_at_1000_max value: 40.1870726936175 - type: nauc_ndcg_at_1000_std value: 0.3499672464058312 - type: nauc_ndcg_at_100_diff1 value: 48.33565088062914 - type: nauc_ndcg_at_100_max value: 40.380628301131686 - type: nauc_ndcg_at_100_std value: 0.18190729344385695 - type: nauc_ndcg_at_10_diff1 value: 49.097983080020896 - type: nauc_ndcg_at_10_max value: 40.24547596077635 - type: nauc_ndcg_at_10_std value: -2.5843384122543545 - type: nauc_ndcg_at_1_diff1 value: 54.263136864624485 - type: nauc_ndcg_at_1_max value: 37.608424271153126 - type: nauc_ndcg_at_1_std value: -4.123286617311634 - type: nauc_ndcg_at_20_diff1 value: 49.050366938364355 - type: nauc_ndcg_at_20_max value: 40.5591755544406 - type: nauc_ndcg_at_20_std value: -1.7443509016518082 - type: nauc_ndcg_at_3_diff1 value: 49.023964347001325 - type: nauc_ndcg_at_3_max value: 39.69832699379779 - type: nauc_ndcg_at_3_std value: -3.6078096702931903 - type: nauc_ndcg_at_5_diff1 value: 48.83333170864256 - type: nauc_ndcg_at_5_max value: 39.356451836952445 - type: nauc_ndcg_at_5_std value: -3.1730431348280503 - type: nauc_precision_at_1000_diff1 value: -21.24752441716892 - type: nauc_precision_at_1000_max value: -3.2970564624677414 - type: nauc_precision_at_1000_std value: 32.83587840469316 - type: nauc_precision_at_100_diff1 value: -16.87411625785483 - type: nauc_precision_at_100_max value: 4.3740393973447595 - type: nauc_precision_at_100_std value: 38.9914966450695 - type: nauc_precision_at_10_diff1 value: 2.6127578839670504 - type: nauc_precision_at_10_max value: 22.35700751539876 - type: nauc_precision_at_10_std value: 24.336639203743573 - type: nauc_precision_at_1_diff1 value: 54.263136864624485 - type: nauc_precision_at_1_max value: 37.608424271153126 - type: nauc_precision_at_1_std value: -4.123286617311634 - type: nauc_precision_at_20_diff1 value: -5.064146107600653 - type: nauc_precision_at_20_max value: 15.50506761431695 - type: nauc_precision_at_20_std value: 29.30528264329397 - type: nauc_precision_at_3_diff1 value: 22.399165352040196 - type: nauc_precision_at_3_max value: 34.296534980252616 - type: nauc_precision_at_3_std value: 10.236576824533735 - type: nauc_precision_at_5_diff1 value: 12.32160353715643 - type: nauc_precision_at_5_max value: 27.798336788100535 - type: nauc_precision_at_5_std value: 16.873821399205333 - type: nauc_recall_at_1000_diff1 value: 31.4259707156231 - type: nauc_recall_at_1000_max value: 44.08463819190203 - type: nauc_recall_at_1000_std value: 37.27474567851736 - type: nauc_recall_at_100_diff1 value: 33.99638840796772 - type: nauc_recall_at_100_max value: 42.40191106044494 - type: nauc_recall_at_100_std value: 20.546273566477595 - type: nauc_recall_at_10_diff1 value: 42.025657605327964 - type: nauc_recall_at_10_max value: 40.00894880694032 - type: nauc_recall_at_10_std value: -1.4821878062279914 - type: nauc_recall_at_1_diff1 value: 58.70495869436133 - type: nauc_recall_at_1_max value: 32.656882457708946 - type: nauc_recall_at_1_std value: -12.67248330614144 - type: nauc_recall_at_20_diff1 value: 40.16583898880467 - type: nauc_recall_at_20_max value: 41.213580182135864 - type: nauc_recall_at_20_std value: 3.894820891877962 - type: nauc_recall_at_3_diff1 value: 46.31225272715354 - type: nauc_recall_at_3_max value: 39.8572063880317 - type: nauc_recall_at_3_std value: -7.1351511325506145 - type: nauc_recall_at_5_diff1 value: 43.482514898780295 - type: nauc_recall_at_5_max value: 38.82684876892274 - type: nauc_recall_at_5_std value: -4.105895420539952 - type: ndcg_at_1 value: 42.611 - type: ndcg_at_10 value: 52.510999999999996 - type: ndcg_at_100 value: 56.682 - type: ndcg_at_1000 value: 58.370999999999995 - type: ndcg_at_20 value: 54.227000000000004 - type: ndcg_at_3 value: 47.673 - type: ndcg_at_5 value: 50.027 - type: precision_at_1 value: 42.611 - type: precision_at_10 value: 10.102 - type: precision_at_100 value: 1.55 - type: precision_at_1000 value: 0.2 - type: precision_at_20 value: 5.863 - type: precision_at_3 value: 23.311999999999998 - type: precision_at_5 value: 16.713 - type: recall_at_1 value: 33.87 - type: recall_at_10 value: 63.845 - type: recall_at_100 value: 81.40899999999999 - type: recall_at_1000 value: 91.594 - type: recall_at_20 value: 70.26599999999999 - type: recall_at_3 value: 49.225 - type: recall_at_5 value: 55.923 task: type: Retrieval - dataset: config: default name: MTEB CQADupstackGamingRetrieval (default) revision: 4885aa143210c98657558c04aaf3dc47cfb54340 split: test type: mteb/cqadupstack-gaming metrics: - type: main_score value: 62.053999999999995 - type: map_at_1 value: 41.581 - type: map_at_10 value: 55.761 - type: map_at_100 value: 56.769999999999996 - type: map_at_1000 value: 56.815000000000005 - type: map_at_20 value: 56.352000000000004 - type: map_at_3 value: 52.317 - type: map_at_5 value: 54.201 - type: mrr_at_1 value: 47.711598746081506 - type: mrr_at_10 value: 58.90774742498884 - type: mrr_at_100 value: 59.52215701290865 - type: mrr_at_1000 value: 59.54155448334906 - type: mrr_at_20 value: 59.26775351231135 - type: mrr_at_3 value: 56.38453500522475 - type: mrr_at_5 value: 57.792058516196555 - type: nauc_map_at_1000_diff1 value: 49.66704402571169 - type: nauc_map_at_1000_max value: 34.07784555889137 - type: nauc_map_at_1000_std value: -13.406279974115126 - type: nauc_map_at_100_diff1 value: 49.6483910540802 - type: nauc_map_at_100_max value: 34.07530844769798 - type: nauc_map_at_100_std value: -13.392381836896217 - type: nauc_map_at_10_diff1 value: 49.84590031698145 - type: nauc_map_at_10_max value: 33.99868315480276 - type: nauc_map_at_10_std value: -14.27778613279837 - type: nauc_map_at_1_diff1 value: 52.01193128355289 - type: nauc_map_at_1_max value: 27.062473514407742 - type: nauc_map_at_1_std value: -15.421160015795982 - type: nauc_map_at_20_diff1 value: 49.6364415002291 - type: nauc_map_at_20_max value: 34.01621363566466 - type: nauc_map_at_20_std value: -13.65438720684474 - type: nauc_map_at_3_diff1 value: 49.554941849859475 - type: nauc_map_at_3_max value: 32.275840154761816 - type: nauc_map_at_3_std value: -15.719396978373707 - type: nauc_map_at_5_diff1 value: 49.818686804013566 - type: nauc_map_at_5_max value: 33.69544243895378 - type: nauc_map_at_5_std value: -15.180559875074701 - type: nauc_mrr_at_1000_diff1 value: 48.73638344709328 - type: nauc_mrr_at_1000_max value: 34.645421834984106 - type: nauc_mrr_at_1000_std value: -13.196777598631263 - type: nauc_mrr_at_100_diff1 value: 48.73305412075345 - type: nauc_mrr_at_100_max value: 34.6491840682274 - type: nauc_mrr_at_100_std value: -13.17337318790356 - type: nauc_mrr_at_10_diff1 value: 48.69043075114964 - type: nauc_mrr_at_10_max value: 34.65856578634812 - type: nauc_mrr_at_10_std value: -13.389064970520973 - type: nauc_mrr_at_1_diff1 value: 51.41758372767858 - type: nauc_mrr_at_1_max value: 32.6284240446433 - type: nauc_mrr_at_1_std value: -13.709567431810976 - type: nauc_mrr_at_20_diff1 value: 48.64452165935141 - type: nauc_mrr_at_20_max value: 34.62668966253434 - type: nauc_mrr_at_20_std value: -13.188024605275306 - type: nauc_mrr_at_3_diff1 value: 48.6499568195659 - type: nauc_mrr_at_3_max value: 34.65641304748175 - type: nauc_mrr_at_3_std value: -14.687769051774529 - type: nauc_mrr_at_5_diff1 value: 48.52683029728482 - type: nauc_mrr_at_5_max value: 34.8254158226808 - type: nauc_mrr_at_5_std value: -13.90451984500762 - type: nauc_ndcg_at_1000_diff1 value: 48.870406690732 - type: nauc_ndcg_at_1000_max value: 35.2675705133695 - type: nauc_ndcg_at_1000_std value: -11.267586713307322 - type: nauc_ndcg_at_100_diff1 value: 48.61247990333261 - type: nauc_ndcg_at_100_max value: 35.41320042882678 - type: nauc_ndcg_at_100_std value: -10.566461682141593 - type: nauc_ndcg_at_10_diff1 value: 48.865637260995584 - type: nauc_ndcg_at_10_max value: 35.72517255893919 - type: nauc_ndcg_at_10_std value: -12.588636382543378 - type: nauc_ndcg_at_1_diff1 value: 51.41758372767858 - type: nauc_ndcg_at_1_max value: 32.6284240446433 - type: nauc_ndcg_at_1_std value: -13.709567431810976 - type: nauc_ndcg_at_20_diff1 value: 48.40318441649108 - type: nauc_ndcg_at_20_max value: 35.45407262577331 - type: nauc_ndcg_at_20_std value: -11.202317076762835 - type: nauc_ndcg_at_3_diff1 value: 48.32682231800704 - type: nauc_ndcg_at_3_max value: 34.07411362254488 - type: nauc_ndcg_at_3_std value: -15.175358391945245 - type: nauc_ndcg_at_5_diff1 value: 48.67053909730509 - type: nauc_ndcg_at_5_max value: 35.63879009797286 - type: nauc_ndcg_at_5_std value: -14.1986091226612 - type: nauc_precision_at_1000_diff1 value: -12.584898766729482 - type: nauc_precision_at_1000_max value: 6.536079770084931 - type: nauc_precision_at_1000_std value: 21.190453831782 - type: nauc_precision_at_100_diff1 value: -10.561462705003443 - type: nauc_precision_at_100_max value: 11.642503498822627 - type: nauc_precision_at_100_std value: 23.364240847068725 - type: nauc_precision_at_10_diff1 value: 10.428388294290725 - type: nauc_precision_at_10_max value: 26.64778522550707 - type: nauc_precision_at_10_std value: 8.7971175822477 - type: nauc_precision_at_1_diff1 value: 51.41758372767858 - type: nauc_precision_at_1_max value: 32.6284240446433 - type: nauc_precision_at_1_std value: -13.709567431810976 - type: nauc_precision_at_20_diff1 value: 1.4153146296498802 - type: nauc_precision_at_20_max value: 21.863221595402713 - type: nauc_precision_at_20_std value: 17.674002200344756 - type: nauc_precision_at_3_diff1 value: 28.16052495795576 - type: nauc_precision_at_3_max value: 33.70008398113751 - type: nauc_precision_at_3_std value: -5.5970612296749644 - type: nauc_precision_at_5_diff1 value: 21.040002038966264 - type: nauc_precision_at_5_max value: 32.38215846452138 - type: nauc_precision_at_5_std value: -0.10361675057694844 - type: nauc_recall_at_1000_diff1 value: 38.213914895059546 - type: nauc_recall_at_1000_max value: 53.09481512653444 - type: nauc_recall_at_1000_std value: 58.24000755818375 - type: nauc_recall_at_100_diff1 value: 38.923229187547136 - type: nauc_recall_at_100_max value: 43.21492770348412 - type: nauc_recall_at_100_std value: 25.990956494703887 - type: nauc_recall_at_10_diff1 value: 44.22168583553731 - type: nauc_recall_at_10_max value: 38.73910397467968 - type: nauc_recall_at_10_std value: -8.031921888246933 - type: nauc_recall_at_1_diff1 value: 52.01193128355289 - type: nauc_recall_at_1_max value: 27.062473514407742 - type: nauc_recall_at_1_std value: -15.421160015795982 - type: nauc_recall_at_20_diff1 value: 40.93219002597565 - type: nauc_recall_at_20_max value: 38.885811403988754 - type: nauc_recall_at_20_std value: 2.162197854553678 - type: nauc_recall_at_3_diff1 value: 44.78642856115218 - type: nauc_recall_at_3_max value: 33.75567422950951 - type: nauc_recall_at_3_std value: -16.234655637869064 - type: nauc_recall_at_5_diff1 value: 44.58830235836066 - type: nauc_recall_at_5_max value: 37.64654838955664 - type: nauc_recall_at_5_std value: -13.950938477657312 - type: ndcg_at_1 value: 47.711999999999996 - type: ndcg_at_10 value: 62.053999999999995 - type: ndcg_at_100 value: 65.83200000000001 - type: ndcg_at_1000 value: 66.599 - type: ndcg_at_20 value: 63.674 - type: ndcg_at_3 value: 56.318999999999996 - type: ndcg_at_5 value: 58.987 - type: precision_at_1 value: 47.711999999999996 - type: precision_at_10 value: 10.125 - type: precision_at_100 value: 1.29 - type: precision_at_1000 value: 0.13899999999999998 - type: precision_at_20 value: 5.574 - type: precision_at_3 value: 25.392 - type: precision_at_5 value: 17.329 - type: recall_at_1 value: 41.581 - type: recall_at_10 value: 77.269 - type: recall_at_100 value: 93.379 - type: recall_at_1000 value: 98.584 - type: recall_at_20 value: 83.313 - type: recall_at_3 value: 62.078 - type: recall_at_5 value: 68.529 task: type: Retrieval - dataset: config: default name: MTEB CQADupstackGisRetrieval (default) revision: 5003b3064772da1887988e05400cf3806fe491f2 split: test type: mteb/cqadupstack-gis metrics: - type: main_score value: 39.748 - type: map_at_1 value: 25.285000000000004 - type: map_at_10 value: 34.371 - type: map_at_100 value: 35.485 - type: map_at_1000 value: 35.569 - type: map_at_20 value: 34.969 - type: map_at_3 value: 31.374000000000002 - type: map_at_5 value: 33.013999999999996 - type: mrr_at_1 value: 27.231638418079097 - type: mrr_at_10 value: 36.34171823154871 - type: mrr_at_100 value: 37.28803039784218 - type: mrr_at_1000 value: 37.35200816522978 - type: mrr_at_20 value: 36.87875687036097 - type: mrr_at_3 value: 33.4463276836158 - type: mrr_at_5 value: 35.11864406779658 - type: nauc_map_at_1000_diff1 value: 40.43809736439773 - type: nauc_map_at_1000_max value: 27.387952141662304 - type: nauc_map_at_1000_std value: -5.018693002046338 - type: nauc_map_at_100_diff1 value: 40.423409205985465 - type: nauc_map_at_100_max value: 27.374411655772974 - type: nauc_map_at_100_std value: -5.017320012911967 - type: nauc_map_at_10_diff1 value: 40.79762636533108 - type: nauc_map_at_10_max value: 27.556132024797297 - type: nauc_map_at_10_std value: -5.236320099160553 - type: nauc_map_at_1_diff1 value: 46.32621190623603 - type: nauc_map_at_1_max value: 27.981895930162796 - type: nauc_map_at_1_std value: -9.286669055239507 - type: nauc_map_at_20_diff1 value: 40.482647454700675 - type: nauc_map_at_20_max value: 27.380748864511183 - type: nauc_map_at_20_std value: -5.157566207786265 - type: nauc_map_at_3_diff1 value: 41.35030270597686 - type: nauc_map_at_3_max value: 26.401420529038973 - type: nauc_map_at_3_std value: -6.825550510798991 - type: nauc_map_at_5_diff1 value: 40.52643403126109 - type: nauc_map_at_5_max value: 26.93344961565937 - type: nauc_map_at_5_std value: -5.671091539711291 - type: nauc_mrr_at_1000_diff1 value: 38.7554936147589 - type: nauc_mrr_at_1000_max value: 26.386180348217618 - type: nauc_mrr_at_1000_std value: -4.033494049459254 - type: nauc_mrr_at_100_diff1 value: 38.742235245990756 - type: nauc_mrr_at_100_max value: 26.38418505089963 - type: nauc_mrr_at_100_std value: -4.005903349074146 - type: nauc_mrr_at_10_diff1 value: 38.967468013959234 - type: nauc_mrr_at_10_max value: 26.48492137182557 - type: nauc_mrr_at_10_std value: -4.037721014953263 - type: nauc_mrr_at_1_diff1 value: 44.016377125107745 - type: nauc_mrr_at_1_max value: 27.47170497073298 - type: nauc_mrr_at_1_std value: -8.306440293433809 - type: nauc_mrr_at_20_diff1 value: 38.68609604223464 - type: nauc_mrr_at_20_max value: 26.374572361531012 - type: nauc_mrr_at_20_std value: -4.086775263524302 - type: nauc_mrr_at_3_diff1 value: 39.53070405624054 - type: nauc_mrr_at_3_max value: 25.480788199729943 - type: nauc_mrr_at_3_std value: -5.541804979531871 - type: nauc_mrr_at_5_diff1 value: 38.83669655976299 - type: nauc_mrr_at_5_max value: 26.078917244246803 - type: nauc_mrr_at_5_std value: -4.422360384815965 - type: nauc_ndcg_at_1000_diff1 value: 38.20327968355583 - type: nauc_ndcg_at_1000_max value: 27.77087169770278 - type: nauc_ndcg_at_1000_std value: -1.8590069773956301 - type: nauc_ndcg_at_100_diff1 value: 37.63890958457494 - type: nauc_ndcg_at_100_max value: 27.575979785801763 - type: nauc_ndcg_at_100_std value: -1.5647502256699413 - type: nauc_ndcg_at_10_diff1 value: 38.799402965520834 - type: nauc_ndcg_at_10_max value: 28.053616957262488 - type: nauc_ndcg_at_10_std value: -2.462939350230248 - type: nauc_ndcg_at_1_diff1 value: 44.016377125107745 - type: nauc_ndcg_at_1_max value: 27.47170497073298 - type: nauc_ndcg_at_1_std value: -8.306440293433809 - type: nauc_ndcg_at_20_diff1 value: 37.680127012971724 - type: nauc_ndcg_at_20_max value: 27.459359553937663 - type: nauc_ndcg_at_20_std value: -2.4173192913196315 - type: nauc_ndcg_at_3_diff1 value: 39.61534122530668 - type: nauc_ndcg_at_3_max value: 25.959631957021195 - type: nauc_ndcg_at_3_std value: -5.802892483091432 - type: nauc_ndcg_at_5_diff1 value: 38.30836310533911 - type: nauc_ndcg_at_5_max value: 26.685143198949117 - type: nauc_ndcg_at_5_std value: -3.5679681994101937 - type: nauc_precision_at_1000_diff1 value: -9.443320801858167 - type: nauc_precision_at_1000_max value: 7.690361600365471 - type: nauc_precision_at_1000_std value: 13.589977869393783 - type: nauc_precision_at_100_diff1 value: 2.1947422010292104 - type: nauc_precision_at_100_max value: 16.350359954039558 - type: nauc_precision_at_100_std value: 13.644626517618743 - type: nauc_precision_at_10_diff1 value: 24.477554964465178 - type: nauc_precision_at_10_max value: 27.26464347335663 - type: nauc_precision_at_10_std value: 7.193678024742925 - type: nauc_precision_at_1_diff1 value: 44.016377125107745 - type: nauc_precision_at_1_max value: 27.47170497073298 - type: nauc_precision_at_1_std value: -8.306440293433809 - type: nauc_precision_at_20_diff1 value: 17.09558791693914 - type: nauc_precision_at_20_max value: 23.205921916753045 - type: nauc_precision_at_20_std value: 8.391846895459514 - type: nauc_precision_at_3_diff1 value: 31.491657526795937 - type: nauc_precision_at_3_max value: 24.710273068738818 - type: nauc_precision_at_3_std value: -1.693177986898931 - type: nauc_precision_at_5_diff1 value: 25.161361863151804 - type: nauc_precision_at_5_max value: 24.936103838639553 - type: nauc_precision_at_5_std value: 2.511934162124435 - type: nauc_recall_at_1000_diff1 value: 25.712209121566048 - type: nauc_recall_at_1000_max value: 39.5293074298725 - type: nauc_recall_at_1000_std value: 38.05519778739929 - type: nauc_recall_at_100_diff1 value: 25.057155432360677 - type: nauc_recall_at_100_max value: 28.588891322745607 - type: nauc_recall_at_100_std value: 15.8128582042832 - type: nauc_recall_at_10_diff1 value: 33.47997633215991 - type: nauc_recall_at_10_max value: 30.028194042399264 - type: nauc_recall_at_10_std value: 4.90810499546238 - type: nauc_recall_at_1_diff1 value: 46.32621190623603 - type: nauc_recall_at_1_max value: 27.981895930162796 - type: nauc_recall_at_1_std value: -9.286669055239507 - type: nauc_recall_at_20_diff1 value: 28.21634195917718 - type: nauc_recall_at_20_max value: 27.48943367590963 - type: nauc_recall_at_20_std value: 5.458479523399421 - type: nauc_recall_at_3_diff1 value: 36.12326469567289 - type: nauc_recall_at_3_max value: 24.6932937032956 - type: nauc_recall_at_3_std value: -3.8068753598947076 - type: nauc_recall_at_5_diff1 value: 33.24253180673521 - type: nauc_recall_at_5_max value: 26.392055872830365 - type: nauc_recall_at_5_std value: 1.6197798374296963 - type: ndcg_at_1 value: 27.232 - type: ndcg_at_10 value: 39.748 - type: ndcg_at_100 value: 45.187 - type: ndcg_at_1000 value: 47.099000000000004 - type: ndcg_at_20 value: 41.811 - type: ndcg_at_3 value: 33.854 - type: ndcg_at_5 value: 36.665 - type: precision_at_1 value: 27.232 - type: precision_at_10 value: 6.271 - type: precision_at_100 value: 0.955 - type: precision_at_1000 value: 0.11399999999999999 - type: precision_at_20 value: 3.627 - type: precision_at_3 value: 14.35 - type: precision_at_5 value: 10.305 - type: recall_at_1 value: 25.285000000000004 - type: recall_at_10 value: 54.466 - type: recall_at_100 value: 79.335 - type: recall_at_1000 value: 93.503 - type: recall_at_20 value: 62.248999999999995 - type: recall_at_3 value: 38.558 - type: recall_at_5 value: 45.249 task: type: Retrieval - dataset: config: default name: MTEB CQADupstackMathematicaRetrieval (default) revision: 90fceea13679c63fe563ded68f3b6f06e50061de split: test type: mteb/cqadupstack-mathematica metrics: - type: main_score value: 34.977000000000004 - type: map_at_1 value: 18.829 - type: map_at_10 value: 28.837000000000003 - type: map_at_100 value: 30.325999999999997 - type: map_at_1000 value: 30.425 - type: map_at_20 value: 29.654999999999998 - type: map_at_3 value: 25.576 - type: map_at_5 value: 27.314 - type: mrr_at_1 value: 23.756218905472636 - type: mrr_at_10 value: 33.77798507462685 - type: mrr_at_100 value: 34.80299385576722 - type: mrr_at_1000 value: 34.85573965593251 - type: mrr_at_20 value: 34.37419603705533 - type: mrr_at_3 value: 30.825041459369828 - type: mrr_at_5 value: 32.317578772802655 - type: nauc_map_at_1000_diff1 value: 33.656933860084195 - type: nauc_map_at_1000_max value: 14.881459199563402 - type: nauc_map_at_1000_std value: -2.0348305200086783 - type: nauc_map_at_100_diff1 value: 33.66651078503352 - type: nauc_map_at_100_max value: 14.87213429316907 - type: nauc_map_at_100_std value: -2.028596921387645 - type: nauc_map_at_10_diff1 value: 33.94706898755689 - type: nauc_map_at_10_max value: 14.335396314155227 - type: nauc_map_at_10_std value: -2.6787096254787643 - type: nauc_map_at_1_diff1 value: 37.51403858840109 - type: nauc_map_at_1_max value: 11.61504414020275 - type: nauc_map_at_1_std value: -5.525240654285529 - type: nauc_map_at_20_diff1 value: 33.80608063225281 - type: nauc_map_at_20_max value: 14.72683703113937 - type: nauc_map_at_20_std value: -2.231105659027114 - type: nauc_map_at_3_diff1 value: 32.96013232027734 - type: nauc_map_at_3_max value: 14.034798307623674 - type: nauc_map_at_3_std value: -3.6600209654787457 - type: nauc_map_at_5_diff1 value: 33.736274045673056 - type: nauc_map_at_5_max value: 14.945515366610651 - type: nauc_map_at_5_std value: -3.4321884525525497 - type: nauc_mrr_at_1000_diff1 value: 30.885842250649926 - type: nauc_mrr_at_1000_max value: 16.199596875238367 - type: nauc_mrr_at_1000_std value: -2.5546416285540445 - type: nauc_mrr_at_100_diff1 value: 30.878602027701735 - type: nauc_mrr_at_100_max value: 16.187973739576613 - type: nauc_mrr_at_100_std value: -2.5665377844879447 - type: nauc_mrr_at_10_diff1 value: 30.978901404792953 - type: nauc_mrr_at_10_max value: 16.05102544616689 - type: nauc_mrr_at_10_std value: -2.7986105722922194 - type: nauc_mrr_at_1_diff1 value: 34.475194079761046 - type: nauc_mrr_at_1_max value: 14.478568759719856 - type: nauc_mrr_at_1_std value: -5.966667936903906 - type: nauc_mrr_at_20_diff1 value: 30.902359160235775 - type: nauc_mrr_at_20_max value: 16.092370727624722 - type: nauc_mrr_at_20_std value: -2.5750922611706475 - type: nauc_mrr_at_3_diff1 value: 30.187778758856542 - type: nauc_mrr_at_3_max value: 16.20340255462948 - type: nauc_mrr_at_3_std value: -3.350161462382174 - type: nauc_mrr_at_5_diff1 value: 30.809795610717135 - type: nauc_mrr_at_5_max value: 16.50725846620521 - type: nauc_mrr_at_5_std value: -3.2069455353142073 - type: nauc_ndcg_at_1000_diff1 value: 32.17770608068487 - type: nauc_ndcg_at_1000_max value: 16.397446092461408 - type: nauc_ndcg_at_1000_std value: 0.8389837771762243 - type: nauc_ndcg_at_100_diff1 value: 31.854138951308812 - type: nauc_ndcg_at_100_max value: 15.912981808544819 - type: nauc_ndcg_at_100_std value: 1.0700284773280755 - type: nauc_ndcg_at_10_diff1 value: 32.78528103960234 - type: nauc_ndcg_at_10_max value: 14.571451424136237 - type: nauc_ndcg_at_10_std value: -1.1491077559945655 - type: nauc_ndcg_at_1_diff1 value: 34.475194079761046 - type: nauc_ndcg_at_1_max value: 14.478568759719856 - type: nauc_ndcg_at_1_std value: -5.966667936903906 - type: nauc_ndcg_at_20_diff1 value: 32.506431715886144 - type: nauc_ndcg_at_20_max value: 15.27762186541485 - type: nauc_ndcg_at_20_std value: 0.04042340992370987 - type: nauc_ndcg_at_3_diff1 value: 30.459324411439447 - type: nauc_ndcg_at_3_max value: 15.203613562825236 - type: nauc_ndcg_at_3_std value: -3.1029884286745637 - type: nauc_ndcg_at_5_diff1 value: 32.163485899012194 - type: nauc_ndcg_at_5_max value: 15.991393277231944 - type: nauc_ndcg_at_5_std value: -2.703797170364151 - type: nauc_precision_at_1000_diff1 value: -2.117125700655664 - type: nauc_precision_at_1000_max value: 6.828373938777722 - type: nauc_precision_at_1000_std value: 2.045215933888246 - type: nauc_precision_at_100_diff1 value: 4.099056472939519 - type: nauc_precision_at_100_max value: 10.875767592976803 - type: nauc_precision_at_100_std value: 8.037114075345642 - type: nauc_precision_at_10_diff1 value: 21.29748789641324 - type: nauc_precision_at_10_max value: 14.672125367272775 - type: nauc_precision_at_10_std value: 3.257317203722233 - type: nauc_precision_at_1_diff1 value: 34.475194079761046 - type: nauc_precision_at_1_max value: 14.478568759719856 - type: nauc_precision_at_1_std value: -5.966667936903906 - type: nauc_precision_at_20_diff1 value: 16.54291297888785 - type: nauc_precision_at_20_max value: 15.697453848166546 - type: nauc_precision_at_20_std value: 6.351013088635058 - type: nauc_precision_at_3_diff1 value: 22.641030059570497 - type: nauc_precision_at_3_max value: 17.94566325867016 - type: nauc_precision_at_3_std value: -0.9418165192444523 - type: nauc_precision_at_5_diff1 value: 24.48631540013291 - type: nauc_precision_at_5_max value: 19.842549668284164 - type: nauc_precision_at_5_std value: 0.31588910564250605 - type: nauc_recall_at_1000_diff1 value: 26.43600927381693 - type: nauc_recall_at_1000_max value: 34.4720828624886 - type: nauc_recall_at_1000_std value: 43.19733828046197 - type: nauc_recall_at_100_diff1 value: 24.9155648703279 - type: nauc_recall_at_100_max value: 16.493541112065078 - type: nauc_recall_at_100_std value: 16.890004919920262 - type: nauc_recall_at_10_diff1 value: 30.406733074151916 - type: nauc_recall_at_10_max value: 12.099008104801277 - type: nauc_recall_at_10_std value: 2.961933769227566 - type: nauc_recall_at_1_diff1 value: 37.51403858840109 - type: nauc_recall_at_1_max value: 11.61504414020275 - type: nauc_recall_at_1_std value: -5.525240654285529 - type: nauc_recall_at_20_diff1 value: 29.02862744695312 - type: nauc_recall_at_20_max value: 13.187402837996709 - type: nauc_recall_at_20_std value: 7.074583238493351 - type: nauc_recall_at_3_diff1 value: 26.87292224049686 - type: nauc_recall_at_3_max value: 14.350499088093164 - type: nauc_recall_at_3_std value: -1.5820898683181281 - type: nauc_recall_at_5_diff1 value: 29.52812156986102 - type: nauc_recall_at_5_max value: 16.36238447178588 - type: nauc_recall_at_5_std value: -1.1333086160805699 - type: ndcg_at_1 value: 23.756 - type: ndcg_at_10 value: 34.977000000000004 - type: ndcg_at_100 value: 41.404999999999994 - type: ndcg_at_1000 value: 43.797000000000004 - type: ndcg_at_20 value: 37.552 - type: ndcg_at_3 value: 29.153000000000002 - type: ndcg_at_5 value: 31.628 - type: precision_at_1 value: 23.756 - type: precision_at_10 value: 6.704000000000001 - type: precision_at_100 value: 1.142 - type: precision_at_1000 value: 0.147 - type: precision_at_20 value: 4.098 - type: precision_at_3 value: 14.469000000000001 - type: precision_at_5 value: 10.423 - type: recall_at_1 value: 18.829 - type: recall_at_10 value: 48.898 - type: recall_at_100 value: 76.12299999999999 - type: recall_at_1000 value: 93.125 - type: recall_at_20 value: 58.013000000000005 - type: recall_at_3 value: 32.628 - type: recall_at_5 value: 39.226 task: type: Retrieval - dataset: config: default name: MTEB CQADupstackPhysicsRetrieval (default) revision: 79531abbd1fb92d06c6d6315a0cbbbf5bb247ea4 split: test type: mteb/cqadupstack-physics metrics: - type: main_score value: 51.841 - type: map_at_1 value: 31.546000000000003 - type: map_at_10 value: 45.062000000000005 - type: map_at_100 value: 46.404 - type: map_at_1000 value: 46.5 - type: map_at_20 value: 45.827 - type: map_at_3 value: 41.285 - type: map_at_5 value: 43.195 - type: mrr_at_1 value: 38.40230991337825 - type: mrr_at_10 value: 49.8141528026032 - type: mrr_at_100 value: 50.57447408080726 - type: mrr_at_1000 value: 50.5993052428773 - type: mrr_at_20 value: 50.25050235209662 - type: mrr_at_3 value: 46.98427975617576 - type: mrr_at_5 value: 48.54347128649335 - type: nauc_map_at_1000_diff1 value: 45.408085068920215 - type: nauc_map_at_1000_max value: 34.81448683377689 - type: nauc_map_at_1000_std value: -9.031735114424992 - type: nauc_map_at_100_diff1 value: 45.39924061967858 - type: nauc_map_at_100_max value: 34.80839440059379 - type: nauc_map_at_100_std value: -9.085641846751734 - type: nauc_map_at_10_diff1 value: 45.33627511201888 - type: nauc_map_at_10_max value: 34.4388392225938 - type: nauc_map_at_10_std value: -9.70629057556224 - type: nauc_map_at_1_diff1 value: 52.682893277244716 - type: nauc_map_at_1_max value: 32.044557907599085 - type: nauc_map_at_1_std value: -12.239875326347203 - type: nauc_map_at_20_diff1 value: 45.39627205891398 - type: nauc_map_at_20_max value: 34.61700403661764 - type: nauc_map_at_20_std value: -9.427569608672833 - type: nauc_map_at_3_diff1 value: 46.487014365288324 - type: nauc_map_at_3_max value: 33.71243831775303 - type: nauc_map_at_3_std value: -11.065305302387815 - type: nauc_map_at_5_diff1 value: 45.98978894568299 - type: nauc_map_at_5_max value: 33.808112350208674 - type: nauc_map_at_5_std value: -10.88139872151709 - type: nauc_mrr_at_1000_diff1 value: 45.26198173932022 - type: nauc_mrr_at_1000_max value: 35.932478080635484 - type: nauc_mrr_at_1000_std value: -7.902048906790103 - type: nauc_mrr_at_100_diff1 value: 45.254393472843375 - type: nauc_mrr_at_100_max value: 35.9431209230091 - type: nauc_mrr_at_100_std value: -7.8981938920645005 - type: nauc_mrr_at_10_diff1 value: 45.08123828809612 - type: nauc_mrr_at_10_max value: 35.931229790288434 - type: nauc_mrr_at_10_std value: -7.936309463023789 - type: nauc_mrr_at_1_diff1 value: 49.22977022573317 - type: nauc_mrr_at_1_max value: 34.23781082286927 - type: nauc_mrr_at_1_std value: -9.287665167939494 - type: nauc_mrr_at_20_diff1 value: 45.252641516190934 - type: nauc_mrr_at_20_max value: 35.89539484155243 - type: nauc_mrr_at_20_std value: -7.972825938602268 - type: nauc_mrr_at_3_diff1 value: 45.171645262995845 - type: nauc_mrr_at_3_max value: 35.92717246816748 - type: nauc_mrr_at_3_std value: -8.902287553992233 - type: nauc_mrr_at_5_diff1 value: 45.19792559490479 - type: nauc_mrr_at_5_max value: 35.82925903242625 - type: nauc_mrr_at_5_std value: -8.849826039196444 - type: nauc_ndcg_at_1000_diff1 value: 44.10471154096236 - type: nauc_ndcg_at_1000_max value: 36.054463074424085 - type: nauc_ndcg_at_1000_std value: -6.413244683136655 - type: nauc_ndcg_at_100_diff1 value: 43.777951617371045 - type: nauc_ndcg_at_100_max value: 36.32691909636164 - type: nauc_ndcg_at_100_std value: -6.5182900903128 - type: nauc_ndcg_at_10_diff1 value: 43.40267963145658 - type: nauc_ndcg_at_10_max value: 35.2051607477023 - type: nauc_ndcg_at_10_std value: -8.007589526999686 - type: nauc_ndcg_at_1_diff1 value: 49.22977022573317 - type: nauc_ndcg_at_1_max value: 34.23781082286927 - type: nauc_ndcg_at_1_std value: -9.287665167939494 - type: nauc_ndcg_at_20_diff1 value: 43.84773553741647 - type: nauc_ndcg_at_20_max value: 35.43036436550042 - type: nauc_ndcg_at_20_std value: -7.804185207595764 - type: nauc_ndcg_at_3_diff1 value: 44.52230934112428 - type: nauc_ndcg_at_3_max value: 34.7781704927549 - type: nauc_ndcg_at_3_std value: -10.213716472980002 - type: nauc_ndcg_at_5_diff1 value: 44.42885848694502 - type: nauc_ndcg_at_5_max value: 34.5484713761122 - type: nauc_ndcg_at_5_std value: -10.379761727670532 - type: nauc_precision_at_1000_diff1 value: -17.115906857147447 - type: nauc_precision_at_1000_max value: -2.1987651081477533 - type: nauc_precision_at_1000_std value: 17.97179450110356 - type: nauc_precision_at_100_diff1 value: -10.925154095184467 - type: nauc_precision_at_100_max value: 9.022276663553845 - type: nauc_precision_at_100_std value: 19.049550737373757 - type: nauc_precision_at_10_diff1 value: 5.155657934683893 - type: nauc_precision_at_10_max value: 22.278949321494142 - type: nauc_precision_at_10_std value: 8.912133685935467 - type: nauc_precision_at_1_diff1 value: 49.22977022573317 - type: nauc_precision_at_1_max value: 34.23781082286927 - type: nauc_precision_at_1_std value: -9.287665167939494 - type: nauc_precision_at_20_diff1 value: 0.08671028194998398 - type: nauc_precision_at_20_max value: 17.69661489592792 - type: nauc_precision_at_20_std value: 11.784850126329411 - type: nauc_precision_at_3_diff1 value: 23.427889174415377 - type: nauc_precision_at_3_max value: 30.15686792703775 - type: nauc_precision_at_3_std value: -1.655045161470365 - type: nauc_precision_at_5_diff1 value: 16.81335324026242 - type: nauc_precision_at_5_max value: 26.762178572537245 - type: nauc_precision_at_5_std value: 0.9528620079402708 - type: nauc_recall_at_1000_diff1 value: 31.76437533883028 - type: nauc_recall_at_1000_max value: 60.981540164283665 - type: nauc_recall_at_1000_std value: 47.60668419126673 - type: nauc_recall_at_100_diff1 value: 30.481511839602387 - type: nauc_recall_at_100_max value: 42.8478963024075 - type: nauc_recall_at_100_std value: 8.840736718195856 - type: nauc_recall_at_10_diff1 value: 35.334955867865595 - type: nauc_recall_at_10_max value: 33.28096014996419 - type: nauc_recall_at_10_std value: -3.552696847109997 - type: nauc_recall_at_1_diff1 value: 52.682893277244716 - type: nauc_recall_at_1_max value: 32.044557907599085 - type: nauc_recall_at_1_std value: -12.239875326347203 - type: nauc_recall_at_20_diff1 value: 37.0961275784984 - type: nauc_recall_at_20_max value: 33.872582440669305 - type: nauc_recall_at_20_std value: -3.5901847360735726 - type: nauc_recall_at_3_diff1 value: 40.64045772324257 - type: nauc_recall_at_3_max value: 31.714945130932794 - type: nauc_recall_at_3_std value: -12.421740996971153 - type: nauc_recall_at_5_diff1 value: 38.912432084860036 - type: nauc_recall_at_5_max value: 31.220715412191886 - type: nauc_recall_at_5_std value: -12.330003159503198 - type: ndcg_at_1 value: 38.401999999999994 - type: ndcg_at_10 value: 51.841 - type: ndcg_at_100 value: 57.121 - type: ndcg_at_1000 value: 58.582 - type: ndcg_at_20 value: 53.947 - type: ndcg_at_3 value: 45.78 - type: ndcg_at_5 value: 48.352000000000004 - type: precision_at_1 value: 38.401999999999994 - type: precision_at_10 value: 9.692 - type: precision_at_100 value: 1.435 - type: precision_at_1000 value: 0.173 - type: precision_at_20 value: 5.611 - type: precision_at_3 value: 22.264999999999997 - type: precision_at_5 value: 15.669 - type: recall_at_1 value: 31.546000000000003 - type: recall_at_10 value: 66.961 - type: recall_at_100 value: 88.71 - type: recall_at_1000 value: 97.821 - type: recall_at_20 value: 74.033 - type: recall_at_3 value: 50.307 - type: recall_at_5 value: 56.825 task: type: Retrieval - dataset: config: default name: MTEB CQADupstackProgrammersRetrieval (default) revision: 6184bc1440d2dbc7612be22b50686b8826d22b32 split: test type: mteb/cqadupstack-programmers metrics: - type: main_score value: 45.814 - type: map_at_1 value: 26.784000000000002 - type: map_at_10 value: 38.958999999999996 - type: map_at_100 value: 40.441 - type: map_at_1000 value: 40.535 - type: map_at_20 value: 39.843 - type: map_at_3 value: 34.735 - type: map_at_5 value: 37.287 - type: mrr_at_1 value: 33.44748858447489 - type: mrr_at_10 value: 44.28281510473287 - type: mrr_at_100 value: 45.25904373827841 - type: mrr_at_1000 value: 45.29554751347883 - type: mrr_at_20 value: 44.92953754847277 - type: mrr_at_3 value: 41.305175038051736 - type: mrr_at_5 value: 43.20585996955856 - type: nauc_map_at_1000_diff1 value: 38.11736750763297 - type: nauc_map_at_1000_max value: 24.000294382878423 - type: nauc_map_at_1000_std value: -7.861373369299095 - type: nauc_map_at_100_diff1 value: 38.11654284813446 - type: nauc_map_at_100_max value: 23.98087270285323 - type: nauc_map_at_100_std value: -7.877385888095451 - type: nauc_map_at_10_diff1 value: 38.08070344774997 - type: nauc_map_at_10_max value: 23.543520971497905 - type: nauc_map_at_10_std value: -8.383371378491509 - type: nauc_map_at_1_diff1 value: 42.18319791470325 - type: nauc_map_at_1_max value: 23.34898555091723 - type: nauc_map_at_1_std value: -12.806880008265043 - type: nauc_map_at_20_diff1 value: 38.114999842598564 - type: nauc_map_at_20_max value: 23.90994956189344 - type: nauc_map_at_20_std value: -8.106492347533809 - type: nauc_map_at_3_diff1 value: 38.575286871887805 - type: nauc_map_at_3_max value: 23.058717018246153 - type: nauc_map_at_3_std value: -11.898614097081257 - type: nauc_map_at_5_diff1 value: 38.34033174186913 - type: nauc_map_at_5_max value: 23.477702791252955 - type: nauc_map_at_5_std value: -9.287267607162825 - type: nauc_mrr_at_1000_diff1 value: 37.32748841102093 - type: nauc_mrr_at_1000_max value: 26.193818618155245 - type: nauc_mrr_at_1000_std value: -5.306318224163809 - type: nauc_mrr_at_100_diff1 value: 37.33545309261833 - type: nauc_mrr_at_100_max value: 26.208771456341218 - type: nauc_mrr_at_100_std value: -5.289635468739874 - type: nauc_mrr_at_10_diff1 value: 36.99565346932208 - type: nauc_mrr_at_10_max value: 25.82424995530185 - type: nauc_mrr_at_10_std value: -5.4973436862637985 - type: nauc_mrr_at_1_diff1 value: 41.86501079427528 - type: nauc_mrr_at_1_max value: 27.346120407617946 - type: nauc_mrr_at_1_std value: -8.384870760349795 - type: nauc_mrr_at_20_diff1 value: 37.21763039260485 - type: nauc_mrr_at_20_max value: 26.140668660687904 - type: nauc_mrr_at_20_std value: -5.309579071500031 - type: nauc_mrr_at_3_diff1 value: 37.29447740662917 - type: nauc_mrr_at_3_max value: 25.72389331364019 - type: nauc_mrr_at_3_std value: -7.528056434106499 - type: nauc_mrr_at_5_diff1 value: 37.03096493380401 - type: nauc_mrr_at_5_max value: 26.20679567894994 - type: nauc_mrr_at_5_std value: -5.422404696106759 - type: nauc_ndcg_at_1000_diff1 value: 37.55262612632063 - type: nauc_ndcg_at_1000_max value: 25.03492533529183 - type: nauc_ndcg_at_1000_std value: -4.620158835806051 - type: nauc_ndcg_at_100_diff1 value: 37.727723031031324 - type: nauc_ndcg_at_100_max value: 25.29454657173904 - type: nauc_ndcg_at_100_std value: -3.947304020047461 - type: nauc_ndcg_at_10_diff1 value: 36.29215161134701 - type: nauc_ndcg_at_10_max value: 23.71291239328531 - type: nauc_ndcg_at_10_std value: -5.168801521084202 - type: nauc_ndcg_at_1_diff1 value: 41.86501079427528 - type: nauc_ndcg_at_1_max value: 27.346120407617946 - type: nauc_ndcg_at_1_std value: -8.384870760349795 - type: nauc_ndcg_at_20_diff1 value: 36.741456788442925 - type: nauc_ndcg_at_20_max value: 24.749531082944497 - type: nauc_ndcg_at_20_std value: -4.513522914617928 - type: nauc_ndcg_at_3_diff1 value: 36.337800581556415 - type: nauc_ndcg_at_3_max value: 23.79612744646427 - type: nauc_ndcg_at_3_std value: -9.814488529010495 - type: nauc_ndcg_at_5_diff1 value: 36.373783732275996 - type: nauc_ndcg_at_5_max value: 24.337705152303517 - type: nauc_ndcg_at_5_std value: -5.9433338033709076 - type: nauc_precision_at_1000_diff1 value: -11.166516654998434 - type: nauc_precision_at_1000_max value: 11.702984079447216 - type: nauc_precision_at_1000_std value: 18.158097934252027 - type: nauc_precision_at_100_diff1 value: -2.4183353698467203 - type: nauc_precision_at_100_max value: 12.354030267351504 - type: nauc_precision_at_100_std value: 17.66939702398267 - type: nauc_precision_at_10_diff1 value: 12.40698391491755 - type: nauc_precision_at_10_max value: 18.40154119372082 - type: nauc_precision_at_10_std value: 11.579162385831191 - type: nauc_precision_at_1_diff1 value: 41.86501079427528 - type: nauc_precision_at_1_max value: 27.346120407617946 - type: nauc_precision_at_1_std value: -8.384870760349795 - type: nauc_precision_at_20_diff1 value: 7.559066186043976 - type: nauc_precision_at_20_max value: 17.961972984402735 - type: nauc_precision_at_20_std value: 14.166666190732109 - type: nauc_precision_at_3_diff1 value: 24.26336438498472 - type: nauc_precision_at_3_max value: 23.115335702834876 - type: nauc_precision_at_3_std value: -2.8638426974295745 - type: nauc_precision_at_5_diff1 value: 18.086439894 - type: nauc_precision_at_5_max value: 21.902852913780674 - type: nauc_precision_at_5_std value: 7.913364070883889 - type: nauc_recall_at_1000_diff1 value: 37.997735697889524 - type: nauc_recall_at_1000_max value: 28.756920000718416 - type: nauc_recall_at_1000_std value: 45.88659450166037 - type: nauc_recall_at_100_diff1 value: 39.654529273336955 - type: nauc_recall_at_100_max value: 30.60687045709423 - type: nauc_recall_at_100_std value: 17.953087240972074 - type: nauc_recall_at_10_diff1 value: 30.14293305093271 - type: nauc_recall_at_10_max value: 19.316085797569126 - type: nauc_recall_at_10_std value: 1.775418885436544 - type: nauc_recall_at_1_diff1 value: 42.18319791470325 - type: nauc_recall_at_1_max value: 23.34898555091723 - type: nauc_recall_at_1_std value: -12.806880008265043 - type: nauc_recall_at_20_diff1 value: 31.113782770327525 - type: nauc_recall_at_20_max value: 23.48656801606143 - type: nauc_recall_at_20_std value: 6.214679015830062 - type: nauc_recall_at_3_diff1 value: 32.6018886813766 - type: nauc_recall_at_3_max value: 20.102015905459307 - type: nauc_recall_at_3_std value: -10.308174390589436 - type: nauc_recall_at_5_diff1 value: 31.44511441051162 - type: nauc_recall_at_5_max value: 20.931266508274724 - type: nauc_recall_at_5_std value: -1.6540639373360793 - type: ndcg_at_1 value: 33.446999999999996 - type: ndcg_at_10 value: 45.814 - type: ndcg_at_100 value: 51.834 - type: ndcg_at_1000 value: 53.553 - type: ndcg_at_20 value: 48.518 - type: ndcg_at_3 value: 39.588 - type: ndcg_at_5 value: 42.849 - type: precision_at_1 value: 33.446999999999996 - type: precision_at_10 value: 8.801 - type: precision_at_100 value: 1.381 - type: precision_at_1000 value: 0.17099999999999999 - type: precision_at_20 value: 5.228 - type: precision_at_3 value: 19.444 - type: precision_at_5 value: 14.521 - type: recall_at_1 value: 26.784000000000002 - type: recall_at_10 value: 60.63499999999999 - type: recall_at_100 value: 86.035 - type: recall_at_1000 value: 97.404 - type: recall_at_20 value: 70.473 - type: recall_at_3 value: 43.205 - type: recall_at_5 value: 51.800999999999995 task: type: Retrieval - dataset: config: default name: MTEB CQADupstackRetrieval (default) revision: CQADupstackRetrieval_is_a_combined_dataset split: test type: CQADupstackRetrieval_is_a_combined_dataset metrics: - type: main_score value: 43.98325 - type: ndcg_at_10 value: 43.98325 task: type: Retrieval - dataset: config: default name: MTEB CQADupstackStatsRetrieval (default) revision: 65ac3a16b8e91f9cee4c9828cc7c335575432a2a split: test type: mteb/cqadupstack-stats metrics: - type: main_score value: 36.291000000000004 - type: map_at_1 value: 23.199 - type: map_at_10 value: 31.516 - type: map_at_100 value: 32.649 - type: map_at_1000 value: 32.742 - type: map_at_20 value: 32.172 - type: map_at_3 value: 29.101 - type: map_at_5 value: 30.487 - type: mrr_at_1 value: 26.226993865030675 - type: mrr_at_10 value: 33.94816924724899 - type: mrr_at_100 value: 34.87358677823681 - type: mrr_at_1000 value: 34.94261020890034 - type: mrr_at_20 value: 34.506501776185196 - type: mrr_at_3 value: 31.518404907975466 - type: mrr_at_5 value: 32.94478527607363 - type: nauc_map_at_1000_diff1 value: 51.18542911225432 - type: nauc_map_at_1000_max value: 40.43058085111423 - type: nauc_map_at_1000_std value: 5.3974448907725066 - type: nauc_map_at_100_diff1 value: 51.15865749726925 - type: nauc_map_at_100_max value: 40.39622362575108 - type: nauc_map_at_100_std value: 5.387804213546535 - type: nauc_map_at_10_diff1 value: 51.26667063819357 - type: nauc_map_at_10_max value: 40.28890459349535 - type: nauc_map_at_10_std value: 4.769853969588081 - type: nauc_map_at_1_diff1 value: 57.84622181579997 - type: nauc_map_at_1_max value: 42.99833198892393 - type: nauc_map_at_1_std value: 2.623995688031952 - type: nauc_map_at_20_diff1 value: 51.20424873230057 - type: nauc_map_at_20_max value: 40.14426501047391 - type: nauc_map_at_20_std value: 5.172256348095352 - type: nauc_map_at_3_diff1 value: 52.62709469297852 - type: nauc_map_at_3_max value: 41.15448127921986 - type: nauc_map_at_3_std value: 3.4581704117373566 - type: nauc_map_at_5_diff1 value: 52.11442057512152 - type: nauc_map_at_5_max value: 40.771412481175986 - type: nauc_map_at_5_std value: 3.9467241008870335 - type: nauc_mrr_at_1000_diff1 value: 50.62386253616975 - type: nauc_mrr_at_1000_max value: 39.326081970356334 - type: nauc_mrr_at_1000_std value: 6.783860502612024 - type: nauc_mrr_at_100_diff1 value: 50.57526957456895 - type: nauc_mrr_at_100_max value: 39.30980897829472 - type: nauc_mrr_at_100_std value: 6.80161963789857 - type: nauc_mrr_at_10_diff1 value: 50.71426300210595 - type: nauc_mrr_at_10_max value: 39.39423685918339 - type: nauc_mrr_at_10_std value: 6.4784274139524545 - type: nauc_mrr_at_1_diff1 value: 56.83151355893988 - type: nauc_mrr_at_1_max value: 41.332173947300824 - type: nauc_mrr_at_1_std value: 5.584550639495193 - type: nauc_mrr_at_20_diff1 value: 50.51574800561875 - type: nauc_mrr_at_20_max value: 39.2435306936296 - type: nauc_mrr_at_20_std value: 6.636135681381268 - type: nauc_mrr_at_3_diff1 value: 51.968318561093206 - type: nauc_mrr_at_3_max value: 39.536950393663254 - type: nauc_mrr_at_3_std value: 5.336239991399375 - type: nauc_mrr_at_5_diff1 value: 51.23639004175655 - type: nauc_mrr_at_5_max value: 39.526442012927895 - type: nauc_mrr_at_5_std value: 5.85626988225536 - type: nauc_ndcg_at_1000_diff1 value: 47.94002347290064 - type: nauc_ndcg_at_1000_max value: 39.32366180439056 - type: nauc_ndcg_at_1000_std value: 8.84579590488393 - type: nauc_ndcg_at_100_diff1 value: 47.14572826825714 - type: nauc_ndcg_at_100_max value: 39.00544081440317 - type: nauc_ndcg_at_100_std value: 9.1174179575023 - type: nauc_ndcg_at_10_diff1 value: 47.846426124821676 - type: nauc_ndcg_at_10_max value: 38.82254197821222 - type: nauc_ndcg_at_10_std value: 6.173511994822973 - type: nauc_ndcg_at_1_diff1 value: 56.83151355893988 - type: nauc_ndcg_at_1_max value: 41.332173947300824 - type: nauc_ndcg_at_1_std value: 5.584550639495193 - type: nauc_ndcg_at_20_diff1 value: 47.499476839800174 - type: nauc_ndcg_at_20_max value: 38.176949417621366 - type: nauc_ndcg_at_20_std value: 7.211539332197563 - type: nauc_ndcg_at_3_diff1 value: 50.41865913024451 - type: nauc_ndcg_at_3_max value: 40.42211341834284 - type: nauc_ndcg_at_3_std value: 4.0996783989115855 - type: nauc_ndcg_at_5_diff1 value: 49.54432423009622 - type: nauc_ndcg_at_5_max value: 39.90824982557047 - type: nauc_ndcg_at_5_std value: 4.659386746150992 - type: nauc_precision_at_1000_diff1 value: -6.669236529596843 - type: nauc_precision_at_1000_max value: 9.948992721182313 - type: nauc_precision_at_1000_std value: 18.850247446285344 - type: nauc_precision_at_100_diff1 value: 5.0040204495885945 - type: nauc_precision_at_100_max value: 20.00879367393483 - type: nauc_precision_at_100_std value: 22.966181182852935 - type: nauc_precision_at_10_diff1 value: 27.68101776997308 - type: nauc_precision_at_10_max value: 27.712070876848816 - type: nauc_precision_at_10_std value: 14.766963486302046 - type: nauc_precision_at_1_diff1 value: 56.83151355893988 - type: nauc_precision_at_1_max value: 41.332173947300824 - type: nauc_precision_at_1_std value: 5.584550639495193 - type: nauc_precision_at_20_diff1 value: 20.370480241261426 - type: nauc_precision_at_20_max value: 23.236345054048897 - type: nauc_precision_at_20_std value: 18.705167446849206 - type: nauc_precision_at_3_diff1 value: 41.064327248640566 - type: nauc_precision_at_3_max value: 35.18704515627873 - type: nauc_precision_at_3_std value: 6.416516457891254 - type: nauc_precision_at_5_diff1 value: 36.89222110213938 - type: nauc_precision_at_5_max value: 32.76854314518032 - type: nauc_precision_at_5_std value: 9.578741823255536 - type: nauc_recall_at_1000_diff1 value: 21.174293136272045 - type: nauc_recall_at_1000_max value: 28.53784351553076 - type: nauc_recall_at_1000_std value: 41.3776314807875 - type: nauc_recall_at_100_diff1 value: 26.596788926351543 - type: nauc_recall_at_100_max value: 30.86754454927091 - type: nauc_recall_at_100_std value: 26.94531386198568 - type: nauc_recall_at_10_diff1 value: 37.24888866240404 - type: nauc_recall_at_10_max value: 33.69371948433766 - type: nauc_recall_at_10_std value: 8.502079126424375 - type: nauc_recall_at_1_diff1 value: 57.84622181579997 - type: nauc_recall_at_1_max value: 42.99833198892393 - type: nauc_recall_at_1_std value: 2.623995688031952 - type: nauc_recall_at_20_diff1 value: 34.84302275962853 - type: nauc_recall_at_20_max value: 30.698743812804143 - type: nauc_recall_at_20_std value: 11.559269171945918 - type: nauc_recall_at_3_diff1 value: 45.955643862037974 - type: nauc_recall_at_3_max value: 37.8434472742434 - type: nauc_recall_at_3_std value: 2.7907114019730006 - type: nauc_recall_at_5_diff1 value: 42.86077909818521 - type: nauc_recall_at_5_max value: 36.34586700724802 - type: nauc_recall_at_5_std value: 4.685965692823914 - type: ndcg_at_1 value: 26.227 - type: ndcg_at_10 value: 36.291000000000004 - type: ndcg_at_100 value: 41.684 - type: ndcg_at_1000 value: 43.949 - type: ndcg_at_20 value: 38.405 - type: ndcg_at_3 value: 31.568 - type: ndcg_at_5 value: 33.891 - type: precision_at_1 value: 26.227 - type: precision_at_10 value: 5.89 - type: precision_at_100 value: 0.9390000000000001 - type: precision_at_1000 value: 0.12 - type: precision_at_20 value: 3.528 - type: precision_at_3 value: 13.700999999999999 - type: precision_at_5 value: 9.724 - type: recall_at_1 value: 23.199 - type: recall_at_10 value: 48.59 - type: recall_at_100 value: 73.332 - type: recall_at_1000 value: 89.825 - type: recall_at_20 value: 56.264 - type: recall_at_3 value: 35.441 - type: recall_at_5 value: 41.284 task: type: Retrieval - dataset: config: default name: MTEB CQADupstackTexRetrieval (default) revision: 46989137a86843e03a6195de44b09deda022eec7 split: test type: mteb/cqadupstack-tex metrics: - type: main_score value: 30.003999999999998 - type: map_at_1 value: 16.203 - type: map_at_10 value: 24.504 - type: map_at_100 value: 25.77 - type: map_at_1000 value: 25.898 - type: map_at_20 value: 25.158 - type: map_at_3 value: 21.538 - type: map_at_5 value: 23.085 - type: mrr_at_1 value: 19.752236751548523 - type: mrr_at_10 value: 28.208304657031487 - type: mrr_at_100 value: 29.194168639099477 - type: mrr_at_1000 value: 29.264587079393394 - type: mrr_at_20 value: 28.733378414151307 - type: mrr_at_3 value: 25.424409268180785 - type: mrr_at_5 value: 26.907547602661225 - type: nauc_map_at_1000_diff1 value: 34.77946144979848 - type: nauc_map_at_1000_max value: 26.51333935423754 - type: nauc_map_at_1000_std value: -2.3836765711100454 - type: nauc_map_at_100_diff1 value: 34.726425095836575 - type: nauc_map_at_100_max value: 26.49453750185533 - type: nauc_map_at_100_std value: -2.3946501662202304 - type: nauc_map_at_10_diff1 value: 35.07483858053342 - type: nauc_map_at_10_max value: 26.38891175643252 - type: nauc_map_at_10_std value: -3.0820847495518584 - type: nauc_map_at_1_diff1 value: 42.277954296027964 - type: nauc_map_at_1_max value: 26.885584027024063 - type: nauc_map_at_1_std value: -5.352914133949852 - type: nauc_map_at_20_diff1 value: 34.8403655164 - type: nauc_map_at_20_max value: 26.458158875988115 - type: nauc_map_at_20_std value: -2.7335504835344664 - type: nauc_map_at_3_diff1 value: 36.881394727999556 - type: nauc_map_at_3_max value: 27.160969666974815 - type: nauc_map_at_3_std value: -4.396911947394683 - type: nauc_map_at_5_diff1 value: 35.639562247169096 - type: nauc_map_at_5_max value: 26.639954780470486 - type: nauc_map_at_5_std value: -3.7965397318705496 - type: nauc_mrr_at_1000_diff1 value: 33.20383220051444 - type: nauc_mrr_at_1000_max value: 25.4316499858392 - type: nauc_mrr_at_1000_std value: -2.045842535875331 - type: nauc_mrr_at_100_diff1 value: 33.16439008042712 - type: nauc_mrr_at_100_max value: 25.416257721385787 - type: nauc_mrr_at_100_std value: -2.0317917154425387 - type: nauc_mrr_at_10_diff1 value: 33.252126790566514 - type: nauc_mrr_at_10_max value: 25.377758108751387 - type: nauc_mrr_at_10_std value: -2.4264189386999044 - type: nauc_mrr_at_1_diff1 value: 39.834719885760556 - type: nauc_mrr_at_1_max value: 25.990117931311673 - type: nauc_mrr_at_1_std value: -4.765973133436674 - type: nauc_mrr_at_20_diff1 value: 33.16383536033632 - type: nauc_mrr_at_20_max value: 25.401527212984014 - type: nauc_mrr_at_20_std value: -2.2220606670426224 - type: nauc_mrr_at_3_diff1 value: 34.74059019329093 - type: nauc_mrr_at_3_max value: 26.307028811743553 - type: nauc_mrr_at_3_std value: -3.5724671708574434 - type: nauc_mrr_at_5_diff1 value: 33.59586580993283 - type: nauc_mrr_at_5_max value: 25.627333570990807 - type: nauc_mrr_at_5_std value: -3.1680352161214715 - type: nauc_ndcg_at_1000_diff1 value: 31.63614044978374 - type: nauc_ndcg_at_1000_max value: 26.07412869941323 - type: nauc_ndcg_at_1000_std value: 1.408963414287443 - type: nauc_ndcg_at_100_diff1 value: 30.611526137380334 - type: nauc_ndcg_at_100_max value: 25.8378308813979 - type: nauc_ndcg_at_100_std value: 1.8510734940750204 - type: nauc_ndcg_at_10_diff1 value: 31.77931205418245 - type: nauc_ndcg_at_10_max value: 25.647247948016282 - type: nauc_ndcg_at_10_std value: -1.1995099974005068 - type: nauc_ndcg_at_1_diff1 value: 39.834719885760556 - type: nauc_ndcg_at_1_max value: 25.990117931311673 - type: nauc_ndcg_at_1_std value: -4.765973133436674 - type: nauc_ndcg_at_20_diff1 value: 31.271749257906933 - type: nauc_ndcg_at_20_max value: 25.77643934349027 - type: nauc_ndcg_at_20_std value: -0.13617627006329658 - type: nauc_ndcg_at_3_diff1 value: 34.78789636847941 - type: nauc_ndcg_at_3_max value: 27.06853334540071 - type: nauc_ndcg_at_3_std value: -3.814111143031556 - type: nauc_ndcg_at_5_diff1 value: 32.80385670003178 - type: nauc_ndcg_at_5_max value: 26.142597326795915 - type: nauc_ndcg_at_5_std value: -2.826446486369442 - type: nauc_precision_at_1000_diff1 value: 5.337214061164788 - type: nauc_precision_at_1000_max value: 5.6432279110132395 - type: nauc_precision_at_1000_std value: 9.71828147452406 - type: nauc_precision_at_100_diff1 value: 6.098940283868962 - type: nauc_precision_at_100_max value: 11.523600551252407 - type: nauc_precision_at_100_std value: 13.53009320116428 - type: nauc_precision_at_10_diff1 value: 17.468734946980348 - type: nauc_precision_at_10_max value: 19.178439669583845 - type: nauc_precision_at_10_std value: 4.204201525745574 - type: nauc_precision_at_1_diff1 value: 39.834719885760556 - type: nauc_precision_at_1_max value: 25.990117931311673 - type: nauc_precision_at_1_std value: -4.765973133436674 - type: nauc_precision_at_20_diff1 value: 13.589754876845628 - type: nauc_precision_at_20_max value: 17.604665523039866 - type: nauc_precision_at_20_std value: 7.374810190128375 - type: nauc_precision_at_3_diff1 value: 28.36601705288169 - type: nauc_precision_at_3_max value: 25.80624066634199 - type: nauc_precision_at_3_std value: -2.8326342170744914 - type: nauc_precision_at_5_diff1 value: 22.742427315942095 - type: nauc_precision_at_5_max value: 22.267969242946513 - type: nauc_precision_at_5_std value: -0.04906212482543379 - type: nauc_recall_at_1000_diff1 value: 16.458371781103924 - type: nauc_recall_at_1000_max value: 25.194190498183794 - type: nauc_recall_at_1000_std value: 30.522986810850743 - type: nauc_recall_at_100_diff1 value: 14.359765512865144 - type: nauc_recall_at_100_max value: 21.553553609929946 - type: nauc_recall_at_100_std value: 19.114947206238213 - type: nauc_recall_at_10_diff1 value: 22.805221872665296 - type: nauc_recall_at_10_max value: 22.62916966651195 - type: nauc_recall_at_10_std value: 2.850986270075809 - type: nauc_recall_at_1_diff1 value: 42.277954296027964 - type: nauc_recall_at_1_max value: 26.885584027024063 - type: nauc_recall_at_1_std value: -5.352914133949852 - type: nauc_recall_at_20_diff1 value: 20.64279001295643 - type: nauc_recall_at_20_max value: 22.388150824277083 - type: nauc_recall_at_20_std value: 6.511559521468454 - type: nauc_recall_at_3_diff1 value: 30.501288503697793 - type: nauc_recall_at_3_max value: 26.87612437428851 - type: nauc_recall_at_3_std value: -2.6083207155232486 - type: nauc_recall_at_5_diff1 value: 25.722785433400364 - type: nauc_recall_at_5_max value: 24.179717355049384 - type: nauc_recall_at_5_std value: -0.9851754722969593 - type: ndcg_at_1 value: 19.752 - type: ndcg_at_10 value: 30.003999999999998 - type: ndcg_at_100 value: 35.992000000000004 - type: ndcg_at_1000 value: 38.706 - type: ndcg_at_20 value: 32.104 - type: ndcg_at_3 value: 24.549000000000003 - type: ndcg_at_5 value: 26.915 - type: precision_at_1 value: 19.752 - type: precision_at_10 value: 5.826 - type: precision_at_100 value: 1.039 - type: precision_at_1000 value: 0.145 - type: precision_at_20 value: 3.543 - type: precision_at_3 value: 11.906 - type: precision_at_5 value: 8.913 - type: recall_at_1 value: 16.203 - type: recall_at_10 value: 42.96 - type: recall_at_100 value: 69.896 - type: recall_at_1000 value: 88.763 - type: recall_at_20 value: 50.690000000000005 - type: recall_at_3 value: 27.706999999999997 - type: recall_at_5 value: 33.732 task: type: Retrieval - dataset: config: default name: MTEB CQADupstackUnixRetrieval (default) revision: 6c6430d3a6d36f8d2a829195bc5dc94d7e063e53 split: test type: mteb/cqadupstack-unix metrics: - type: main_score value: 44.214 - type: map_at_1 value: 26.362000000000002 - type: map_at_10 value: 37.903 - type: map_at_100 value: 39.225 - type: map_at_1000 value: 39.318 - type: map_at_20 value: 38.675 - type: map_at_3 value: 34.339 - type: map_at_5 value: 36.41 - type: mrr_at_1 value: 30.597014925373134 - type: mrr_at_10 value: 41.56501717602458 - type: mrr_at_100 value: 42.44037104600305 - type: mrr_at_1000 value: 42.49164682359615 - type: mrr_at_20 value: 42.065919605875514 - type: mrr_at_3 value: 38.54166666666664 - type: mrr_at_5 value: 40.28606965174122 - type: nauc_map_at_1000_diff1 value: 47.022193015822545 - type: nauc_map_at_1000_max value: 37.510069283996 - type: nauc_map_at_1000_std value: -2.619223749210864 - type: nauc_map_at_100_diff1 value: 47.0110128474831 - type: nauc_map_at_100_max value: 37.49734217286656 - type: nauc_map_at_100_std value: -2.623168195285326 - type: nauc_map_at_10_diff1 value: 47.333691992315586 - type: nauc_map_at_10_max value: 37.42544432726044 - type: nauc_map_at_10_std value: -3.032361221814562 - type: nauc_map_at_1_diff1 value: 55.56766500340758 - type: nauc_map_at_1_max value: 37.55816037621747 - type: nauc_map_at_1_std value: -3.4756245106235073 - type: nauc_map_at_20_diff1 value: 47.043553806230264 - type: nauc_map_at_20_max value: 37.46764790200979 - type: nauc_map_at_20_std value: -2.902106622032998 - type: nauc_map_at_3_diff1 value: 48.32329624357104 - type: nauc_map_at_3_max value: 37.069989364749 - type: nauc_map_at_3_std value: -3.8963394773157063 - type: nauc_map_at_5_diff1 value: 47.45984724665631 - type: nauc_map_at_5_max value: 37.49419828702461 - type: nauc_map_at_5_std value: -2.7001983351853083 - type: nauc_mrr_at_1000_diff1 value: 44.040549449179785 - type: nauc_mrr_at_1000_max value: 37.635183657218285 - type: nauc_mrr_at_1000_std value: -2.2998001164068165 - type: nauc_mrr_at_100_diff1 value: 44.01767577712088 - type: nauc_mrr_at_100_max value: 37.61732411034475 - type: nauc_mrr_at_100_std value: -2.298113876953238 - type: nauc_mrr_at_10_diff1 value: 43.99377722723508 - type: nauc_mrr_at_10_max value: 37.63920703610793 - type: nauc_mrr_at_10_std value: -2.46107841302828 - type: nauc_mrr_at_1_diff1 value: 51.58851879203434 - type: nauc_mrr_at_1_max value: 39.18276895998245 - type: nauc_mrr_at_1_std value: -3.7547202990719293 - type: nauc_mrr_at_20_diff1 value: 43.933755385151294 - type: nauc_mrr_at_20_max value: 37.63291344921131 - type: nauc_mrr_at_20_std value: -2.4461841577607557 - type: nauc_mrr_at_3_diff1 value: 44.23763203861928 - type: nauc_mrr_at_3_max value: 37.61510167030856 - type: nauc_mrr_at_3_std value: -2.9981794837896873 - type: nauc_mrr_at_5_diff1 value: 43.70678854004207 - type: nauc_mrr_at_5_max value: 37.958066094208924 - type: nauc_mrr_at_5_std value: -2.2069511290655743 - type: nauc_ndcg_at_1000_diff1 value: 43.89102270297004 - type: nauc_ndcg_at_1000_max value: 37.373942962075965 - type: nauc_ndcg_at_1000_std value: -0.79381131699489 - type: nauc_ndcg_at_100_diff1 value: 43.480624887214276 - type: nauc_ndcg_at_100_max value: 36.779323191929564 - type: nauc_ndcg_at_100_std value: -0.5142340495653143 - type: nauc_ndcg_at_10_diff1 value: 44.1480629394505 - type: nauc_ndcg_at_10_max value: 36.79038602100573 - type: nauc_ndcg_at_10_std value: -2.5113316190423336 - type: nauc_ndcg_at_1_diff1 value: 51.58851879203434 - type: nauc_ndcg_at_1_max value: 39.18276895998245 - type: nauc_ndcg_at_1_std value: -3.7547202990719293 - type: nauc_ndcg_at_20_diff1 value: 43.44148588178158 - type: nauc_ndcg_at_20_max value: 36.78079904803215 - type: nauc_ndcg_at_20_std value: -2.2475970493788338 - type: nauc_ndcg_at_3_diff1 value: 44.786947971061814 - type: nauc_ndcg_at_3_max value: 36.95060577653726 - type: nauc_ndcg_at_3_std value: -3.7387486325957204 - type: nauc_ndcg_at_5_diff1 value: 43.96479829748338 - type: nauc_ndcg_at_5_max value: 37.29927097229379 - type: nauc_ndcg_at_5_std value: -1.8612882187654674 - type: nauc_precision_at_1000_diff1 value: -17.776876205133522 - type: nauc_precision_at_1000_max value: -2.541489259766695 - type: nauc_precision_at_1000_std value: 2.131464419490259 - type: nauc_precision_at_100_diff1 value: -8.766821764176708 - type: nauc_precision_at_100_max value: 7.997796011277149 - type: nauc_precision_at_100_std value: 7.755850154638668 - type: nauc_precision_at_10_diff1 value: 13.884358851496742 - type: nauc_precision_at_10_max value: 26.053763030345863 - type: nauc_precision_at_10_std value: -0.7621762947543562 - type: nauc_precision_at_1_diff1 value: 51.58851879203434 - type: nauc_precision_at_1_max value: 39.18276895998245 - type: nauc_precision_at_1_std value: -3.7547202990719293 - type: nauc_precision_at_20_diff1 value: 5.289872641356584 - type: nauc_precision_at_20_max value: 20.607130764764804 - type: nauc_precision_at_20_std value: -0.19263962373893834 - type: nauc_precision_at_3_diff1 value: 28.611068231485216 - type: nauc_precision_at_3_max value: 33.906669095382576 - type: nauc_precision_at_3_std value: -3.528371042478906 - type: nauc_precision_at_5_diff1 value: 21.242353796078778 - type: nauc_precision_at_5_max value: 31.511434658857922 - type: nauc_precision_at_5_std value: 1.657940004239003 - type: nauc_recall_at_1000_diff1 value: 21.80735958513851 - type: nauc_recall_at_1000_max value: 46.49625873610379 - type: nauc_recall_at_1000_std value: 43.46830209225304 - type: nauc_recall_at_100_diff1 value: 28.962521400789143 - type: nauc_recall_at_100_max value: 28.645469512854305 - type: nauc_recall_at_100_std value: 14.568180770525782 - type: nauc_recall_at_10_diff1 value: 35.9753633121628 - type: nauc_recall_at_10_max value: 32.09412739670866 - type: nauc_recall_at_10_std value: -1.4303897530560201 - type: nauc_recall_at_1_diff1 value: 55.56766500340758 - type: nauc_recall_at_1_max value: 37.55816037621747 - type: nauc_recall_at_1_std value: -3.4756245106235073 - type: nauc_recall_at_20_diff1 value: 32.57151610283604 - type: nauc_recall_at_20_max value: 31.185132882427464 - type: nauc_recall_at_20_std value: -0.5972674819999952 - type: nauc_recall_at_3_diff1 value: 39.55078433962911 - type: nauc_recall_at_3_max value: 33.34789015929553 - type: nauc_recall_at_3_std value: -4.062145558641841 - type: nauc_recall_at_5_diff1 value: 36.224639758887754 - type: nauc_recall_at_5_max value: 34.233241128748105 - type: nauc_recall_at_5_std value: 0.10907791862296992 - type: ndcg_at_1 value: 30.597 - type: ndcg_at_10 value: 44.214 - type: ndcg_at_100 value: 49.834 - type: ndcg_at_1000 value: 51.696 - type: ndcg_at_20 value: 46.541 - type: ndcg_at_3 value: 38.086 - type: ndcg_at_5 value: 41.093 - type: precision_at_1 value: 30.597 - type: precision_at_10 value: 7.845000000000001 - type: precision_at_100 value: 1.201 - type: precision_at_1000 value: 0.147 - type: precision_at_20 value: 4.618 - type: precision_at_3 value: 17.785999999999998 - type: precision_at_5 value: 12.799 - type: recall_at_1 value: 26.362000000000002 - type: recall_at_10 value: 59.484 - type: recall_at_100 value: 83.353 - type: recall_at_1000 value: 95.719 - type: recall_at_20 value: 67.74900000000001 - type: recall_at_3 value: 42.83 - type: recall_at_5 value: 50.454 task: type: Retrieval - dataset: config: default name: MTEB CQADupstackWebmastersRetrieval (default) revision: 160c094312a0e1facb97e55eeddb698c0abe3571 split: test type: mteb/cqadupstack-webmasters metrics: - type: main_score value: 41.826 - type: map_at_1 value: 24.39 - type: map_at_10 value: 35.479 - type: map_at_100 value: 37.407000000000004 - type: map_at_1000 value: 37.632 - type: map_at_20 value: 36.399 - type: map_at_3 value: 32.33 - type: map_at_5 value: 34.123 - type: mrr_at_1 value: 29.446640316205535 - type: mrr_at_10 value: 39.91953698475437 - type: mrr_at_100 value: 40.974311857221245 - type: mrr_at_1000 value: 41.02039114169333 - type: mrr_at_20 value: 40.46987332341717 - type: mrr_at_3 value: 37.154150197628475 - type: mrr_at_5 value: 38.863636363636374 - type: nauc_map_at_1000_diff1 value: 42.006980615840476 - type: nauc_map_at_1000_max value: 19.588898254504482 - type: nauc_map_at_1000_std value: -0.5711501793009518 - type: nauc_map_at_100_diff1 value: 41.89311474305503 - type: nauc_map_at_100_max value: 19.66343179157545 - type: nauc_map_at_100_std value: -0.9493523888880031 - type: nauc_map_at_10_diff1 value: 42.218327545966794 - type: nauc_map_at_10_max value: 19.093480513385746 - type: nauc_map_at_10_std value: -2.575230373968492 - type: nauc_map_at_1_diff1 value: 49.55673669630866 - type: nauc_map_at_1_max value: 21.657821076583392 - type: nauc_map_at_1_std value: -2.822086497620212 - type: nauc_map_at_20_diff1 value: 42.06716413657905 - type: nauc_map_at_20_max value: 19.443018878171156 - type: nauc_map_at_20_std value: -2.0472340866881598 - type: nauc_map_at_3_diff1 value: 43.358256588830976 - type: nauc_map_at_3_max value: 20.29951410862834 - type: nauc_map_at_3_std value: -3.122031268596122 - type: nauc_map_at_5_diff1 value: 42.79238194223064 - type: nauc_map_at_5_max value: 19.15815022850817 - type: nauc_map_at_5_std value: -3.4023876930915935 - type: nauc_mrr_at_1000_diff1 value: 38.89485952557572 - type: nauc_mrr_at_1000_max value: 18.00288526280676 - type: nauc_mrr_at_1000_std value: 0.7637893917916386 - type: nauc_mrr_at_100_diff1 value: 38.86513791743708 - type: nauc_mrr_at_100_max value: 17.984440283930496 - type: nauc_mrr_at_100_std value: 0.7693475120218602 - type: nauc_mrr_at_10_diff1 value: 38.882346535299604 - type: nauc_mrr_at_10_max value: 18.018489876209962 - type: nauc_mrr_at_10_std value: 0.6894393091820157 - type: nauc_mrr_at_1_diff1 value: 43.23691098503311 - type: nauc_mrr_at_1_max value: 19.31011799251083 - type: nauc_mrr_at_1_std value: 0.5817043396319133 - type: nauc_mrr_at_20_diff1 value: 39.01036846117744 - type: nauc_mrr_at_20_max value: 18.083878464549187 - type: nauc_mrr_at_20_std value: 0.6036676825077607 - type: nauc_mrr_at_3_diff1 value: 39.025378279654234 - type: nauc_mrr_at_3_max value: 19.13536212310877 - type: nauc_mrr_at_3_std value: 0.6868166618588564 - type: nauc_mrr_at_5_diff1 value: 38.935577962802384 - type: nauc_mrr_at_5_max value: 17.828195279090846 - type: nauc_mrr_at_5_std value: 0.023977688818205518 - type: nauc_ndcg_at_1000_diff1 value: 39.793531662914624 - type: nauc_ndcg_at_1000_max value: 18.847958004720937 - type: nauc_ndcg_at_1000_std value: 1.804038871743905 - type: nauc_ndcg_at_100_diff1 value: 38.82881796342845 - type: nauc_ndcg_at_100_max value: 18.487091746204694 - type: nauc_ndcg_at_100_std value: 2.151729614942825 - type: nauc_ndcg_at_10_diff1 value: 39.84007249501964 - type: nauc_ndcg_at_10_max value: 17.939005773004503 - type: nauc_ndcg_at_10_std value: -0.08117427983218028 - type: nauc_ndcg_at_1_diff1 value: 43.23691098503311 - type: nauc_ndcg_at_1_max value: 19.31011799251083 - type: nauc_ndcg_at_1_std value: 0.5817043396319133 - type: nauc_ndcg_at_20_diff1 value: 39.8037093598719 - type: nauc_ndcg_at_20_max value: 18.48853530550249 - type: nauc_ndcg_at_20_std value: -0.10328037808023001 - type: nauc_ndcg_at_3_diff1 value: 40.446978150459806 - type: nauc_ndcg_at_3_max value: 19.84200589719808 - type: nauc_ndcg_at_3_std value: -0.42962792649394055 - type: nauc_ndcg_at_5_diff1 value: 40.28504143456421 - type: nauc_ndcg_at_5_max value: 17.817674524132045 - type: nauc_ndcg_at_5_std value: -1.1454777994636318 - type: nauc_precision_at_1000_diff1 value: 3.6440544638350914 - type: nauc_precision_at_1000_max value: -5.322395298494433 - type: nauc_precision_at_1000_std value: 34.045681102264446 - type: nauc_precision_at_100_diff1 value: 0.39376090580783235 - type: nauc_precision_at_100_max value: 1.521619479821471 - type: nauc_precision_at_100_std value: 31.437894340720597 - type: nauc_precision_at_10_diff1 value: 11.917497057371422 - type: nauc_precision_at_10_max value: 9.428271545180175 - type: nauc_precision_at_10_std value: 12.786501469546883 - type: nauc_precision_at_1_diff1 value: 43.23691098503311 - type: nauc_precision_at_1_max value: 19.31011799251083 - type: nauc_precision_at_1_std value: 0.5817043396319133 - type: nauc_precision_at_20_diff1 value: 6.023124970004902 - type: nauc_precision_at_20_max value: 8.805642599987998 - type: nauc_precision_at_20_std value: 18.846678438758165 - type: nauc_precision_at_3_diff1 value: 24.65558274053028 - type: nauc_precision_at_3_max value: 16.61089322421877 - type: nauc_precision_at_3_std value: 5.403032728665573 - type: nauc_precision_at_5_diff1 value: 18.18179370215765 - type: nauc_precision_at_5_max value: 10.59580743049896 - type: nauc_precision_at_5_std value: 6.8603748742003186 - type: nauc_recall_at_1000_diff1 value: 22.552468938352412 - type: nauc_recall_at_1000_max value: 0.5070109561586749 - type: nauc_recall_at_1000_std value: 36.4227590800954 - type: nauc_recall_at_100_diff1 value: 20.081881489107552 - type: nauc_recall_at_100_max value: 9.134216705040693 - type: nauc_recall_at_100_std value: 18.911359987932503 - type: nauc_recall_at_10_diff1 value: 33.5396704389877 - type: nauc_recall_at_10_max value: 13.961578985947318 - type: nauc_recall_at_10_std value: 0.3177885671992674 - type: nauc_recall_at_1_diff1 value: 49.55673669630866 - type: nauc_recall_at_1_max value: 21.657821076583392 - type: nauc_recall_at_1_std value: -2.822086497620212 - type: nauc_recall_at_20_diff1 value: 32.289790950429804 - type: nauc_recall_at_20_max value: 15.71770065051139 - type: nauc_recall_at_20_std value: 0.46912194119269873 - type: nauc_recall_at_3_diff1 value: 38.16094047537937 - type: nauc_recall_at_3_max value: 18.615919857237486 - type: nauc_recall_at_3_std value: -3.1020886129527763 - type: nauc_recall_at_5_diff1 value: 36.476258679863136 - type: nauc_recall_at_5_max value: 14.03780564222494 - type: nauc_recall_at_5_std value: -4.610491792353064 - type: ndcg_at_1 value: 29.447000000000003 - type: ndcg_at_10 value: 41.826 - type: ndcg_at_100 value: 48.559999999999995 - type: ndcg_at_1000 value: 50.678 - type: ndcg_at_20 value: 44.204 - type: ndcg_at_3 value: 36.687 - type: ndcg_at_5 value: 39.345 - type: precision_at_1 value: 29.447000000000003 - type: precision_at_10 value: 8.202 - type: precision_at_100 value: 1.735 - type: precision_at_1000 value: 0.253 - type: precision_at_20 value: 5.257 - type: precision_at_3 value: 17.523 - type: precision_at_5 value: 12.925 - type: recall_at_1 value: 24.39 - type: recall_at_10 value: 54.173 - type: recall_at_100 value: 83.648 - type: recall_at_1000 value: 96.819 - type: recall_at_20 value: 63.09 - type: recall_at_3 value: 40.146 - type: recall_at_5 value: 46.705999999999996 task: type: Retrieval - dataset: config: default name: MTEB CQADupstackWordpressRetrieval (default) revision: 4ffe81d471b1924886b33c7567bfb200e9eec5c4 split: test type: mteb/cqadupstack-wordpress metrics: - type: main_score value: 35.32 - type: map_at_1 value: 21.679000000000002 - type: map_at_10 value: 29.976000000000003 - type: map_at_100 value: 31.127 - type: map_at_1000 value: 31.207 - type: map_at_20 value: 30.628 - type: map_at_3 value: 26.884999999999998 - type: map_at_5 value: 28.698 - type: mrr_at_1 value: 23.10536044362292 - type: mrr_at_10 value: 31.68955197605845 - type: mrr_at_100 value: 32.696453094313846 - type: mrr_at_1000 value: 32.74644026497888 - type: mrr_at_20 value: 32.270216614045836 - type: mrr_at_3 value: 28.89710412815773 - type: mrr_at_5 value: 30.58841651263093 - type: nauc_map_at_1000_diff1 value: 36.527663233423816 - type: nauc_map_at_1000_max value: 25.70281352614131 - type: nauc_map_at_1000_std value: -10.47544573010382 - type: nauc_map_at_100_diff1 value: 36.49901520674112 - type: nauc_map_at_100_max value: 25.720411208615296 - type: nauc_map_at_100_std value: -10.4498182981879 - type: nauc_map_at_10_diff1 value: 36.56160743360435 - type: nauc_map_at_10_max value: 25.481122372570773 - type: nauc_map_at_10_std value: -11.386413278364452 - type: nauc_map_at_1_diff1 value: 41.96740973142382 - type: nauc_map_at_1_max value: 26.447948671103067 - type: nauc_map_at_1_std value: -13.055537837295866 - type: nauc_map_at_20_diff1 value: 36.53096807557235 - type: nauc_map_at_20_max value: 25.65603650587719 - type: nauc_map_at_20_std value: -10.662281391136927 - type: nauc_map_at_3_diff1 value: 37.49452415422011 - type: nauc_map_at_3_max value: 25.73144884866817 - type: nauc_map_at_3_std value: -13.338023323977556 - type: nauc_map_at_5_diff1 value: 36.606269278969975 - type: nauc_map_at_5_max value: 25.41542359776377 - type: nauc_map_at_5_std value: -11.855800226714972 - type: nauc_mrr_at_1000_diff1 value: 37.23554980964842 - type: nauc_mrr_at_1000_max value: 26.21821820816298 - type: nauc_mrr_at_1000_std value: -8.828254845751971 - type: nauc_mrr_at_100_diff1 value: 37.212657398099694 - type: nauc_mrr_at_100_max value: 26.22027175755234 - type: nauc_mrr_at_100_std value: -8.796357557047992 - type: nauc_mrr_at_10_diff1 value: 37.2057222353167 - type: nauc_mrr_at_10_max value: 26.066257028633437 - type: nauc_mrr_at_10_std value: -9.40582019511086 - type: nauc_mrr_at_1_diff1 value: 42.88369139125157 - type: nauc_mrr_at_1_max value: 28.11145662254025 - type: nauc_mrr_at_1_std value: -11.874419938670789 - type: nauc_mrr_at_20_diff1 value: 37.15164357320353 - type: nauc_mrr_at_20_max value: 26.179170369717674 - type: nauc_mrr_at_20_std value: -8.850966109061742 - type: nauc_mrr_at_3_diff1 value: 37.65133484078781 - type: nauc_mrr_at_3_max value: 27.013109167125126 - type: nauc_mrr_at_3_std value: -10.62551433790744 - type: nauc_mrr_at_5_diff1 value: 37.01454233799136 - type: nauc_mrr_at_5_max value: 26.487303256496585 - type: nauc_mrr_at_5_std value: -9.414516112557687 - type: nauc_ndcg_at_1000_diff1 value: 34.6398016933486 - type: nauc_ndcg_at_1000_max value: 25.383536282414426 - type: nauc_ndcg_at_1000_std value: -6.736596281730975 - type: nauc_ndcg_at_100_diff1 value: 33.96995602141814 - type: nauc_ndcg_at_100_max value: 25.233190880846283 - type: nauc_ndcg_at_100_std value: -6.232028137803393 - type: nauc_ndcg_at_10_diff1 value: 34.36578627190237 - type: nauc_ndcg_at_10_max value: 24.497955345913873 - type: nauc_ndcg_at_10_std value: -9.586241421890247 - type: nauc_ndcg_at_1_diff1 value: 42.88369139125157 - type: nauc_ndcg_at_1_max value: 28.11145662254025 - type: nauc_ndcg_at_1_std value: -11.874419938670789 - type: nauc_ndcg_at_20_diff1 value: 34.029206058207315 - type: nauc_ndcg_at_20_max value: 24.66376498000024 - type: nauc_ndcg_at_20_std value: -7.237996394617084 - type: nauc_ndcg_at_3_diff1 value: 35.42536049883026 - type: nauc_ndcg_at_3_max value: 25.919093993978805 - type: nauc_ndcg_at_3_std value: -12.333032157833742 - type: nauc_ndcg_at_5_diff1 value: 34.21670718788367 - type: nauc_ndcg_at_5_max value: 24.883264080042085 - type: nauc_ndcg_at_5_std value: -10.228412772316155 - type: nauc_precision_at_1000_diff1 value: -5.762103084787246 - type: nauc_precision_at_1000_max value: -7.5833194267759865 - type: nauc_precision_at_1000_std value: 10.675418300666859 - type: nauc_precision_at_100_diff1 value: 2.548944950287374 - type: nauc_precision_at_100_max value: 16.50696190698422 - type: nauc_precision_at_100_std value: 20.430914163842253 - type: nauc_precision_at_10_diff1 value: 18.385077734207343 - type: nauc_precision_at_10_max value: 22.121729211787244 - type: nauc_precision_at_10_std value: 2.4230944307235625 - type: nauc_precision_at_1_diff1 value: 42.88369139125157 - type: nauc_precision_at_1_max value: 28.11145662254025 - type: nauc_precision_at_1_std value: -11.874419938670789 - type: nauc_precision_at_20_diff1 value: 14.947847678278489 - type: nauc_precision_at_20_max value: 21.704383740839535 - type: nauc_precision_at_20_std value: 10.626943172005621 - type: nauc_precision_at_3_diff1 value: 29.387921830287894 - type: nauc_precision_at_3_max value: 26.314386080831557 - type: nauc_precision_at_3_std value: -8.577508887319222 - type: nauc_precision_at_5_diff1 value: 23.965022746288973 - type: nauc_precision_at_5_max value: 24.929773207772136 - type: nauc_precision_at_5_std value: -2.4211370627395503 - type: nauc_recall_at_1000_diff1 value: 14.67658540132588 - type: nauc_recall_at_1000_max value: 25.448574144193696 - type: nauc_recall_at_1000_std value: 36.56427335045912 - type: nauc_recall_at_100_diff1 value: 20.467829912574835 - type: nauc_recall_at_100_max value: 20.79654725970721 - type: nauc_recall_at_100_std value: 13.227837192809094 - type: nauc_recall_at_10_diff1 value: 27.310476416825875 - type: nauc_recall_at_10_max value: 20.224074911583465 - type: nauc_recall_at_10_std value: -6.358258513190565 - type: nauc_recall_at_1_diff1 value: 41.96740973142382 - type: nauc_recall_at_1_max value: 26.447948671103067 - type: nauc_recall_at_1_std value: -13.055537837295866 - type: nauc_recall_at_20_diff1 value: 24.956767447964573 - type: nauc_recall_at_20_max value: 19.201058474561965 - type: nauc_recall_at_20_std value: 2.8393009259024424 - type: nauc_recall_at_3_diff1 value: 30.029057692380977 - type: nauc_recall_at_3_max value: 24.62072994737222 - type: nauc_recall_at_3_std value: -12.694054132443867 - type: nauc_recall_at_5_diff1 value: 26.697765884037405 - type: nauc_recall_at_5_max value: 21.820140093186758 - type: nauc_recall_at_5_std value: -7.5604509717503845 - type: ndcg_at_1 value: 23.105 - type: ndcg_at_10 value: 35.32 - type: ndcg_at_100 value: 41.136 - type: ndcg_at_1000 value: 43.228 - type: ndcg_at_20 value: 37.57 - type: ndcg_at_3 value: 29.396 - type: ndcg_at_5 value: 32.494 - type: precision_at_1 value: 23.105 - type: precision_at_10 value: 5.878 - type: precision_at_100 value: 0.9520000000000001 - type: precision_at_1000 value: 0.125 - type: precision_at_20 value: 3.466 - type: precision_at_3 value: 12.692999999999998 - type: precision_at_5 value: 9.39 - type: recall_at_1 value: 21.679000000000002 - type: recall_at_10 value: 49.925999999999995 - type: recall_at_100 value: 77.274 - type: recall_at_1000 value: 92.907 - type: recall_at_20 value: 58.650000000000006 - type: recall_at_3 value: 34.143 - type: recall_at_5 value: 41.802 task: type: Retrieval - dataset: config: default name: MTEB ClimateFEVER (default) revision: 47f2ac6acb640fc46020b02a5b59fdda04d39380 split: test type: mteb/climate-fever metrics: - type: main_score value: 36.964000000000006 - type: map_at_1 value: 16.888 - type: map_at_10 value: 27.276 - type: map_at_100 value: 29.357 - type: map_at_1000 value: 29.511 - type: map_at_20 value: 28.492 - type: map_at_3 value: 23.075000000000003 - type: map_at_5 value: 25.312 - type: mrr_at_1 value: 38.957654723127035 - type: mrr_at_10 value: 50.258001137480015 - type: mrr_at_100 value: 50.985884572131546 - type: mrr_at_1000 value: 51.00775273659549 - type: mrr_at_20 value: 50.75537860384226 - type: mrr_at_3 value: 47.285559174810025 - type: mrr_at_5 value: 49.103148751357296 - type: nauc_map_at_1000_diff1 value: 25.747911729046617 - type: nauc_map_at_1000_max value: 31.255398545903596 - type: nauc_map_at_1000_std value: 24.872486591713496 - type: nauc_map_at_100_diff1 value: 25.740790002104784 - type: nauc_map_at_100_max value: 31.266176115777732 - type: nauc_map_at_100_std value: 24.84459109976713 - type: nauc_map_at_10_diff1 value: 26.02628794450362 - type: nauc_map_at_10_max value: 31.374003980672775 - type: nauc_map_at_10_std value: 23.587823327864708 - type: nauc_map_at_1_diff1 value: 34.73062154387364 - type: nauc_map_at_1_max value: 30.574707039457493 - type: nauc_map_at_1_std value: 19.436488371927435 - type: nauc_map_at_20_diff1 value: 25.895513631254737 - type: nauc_map_at_20_max value: 31.283103126369085 - type: nauc_map_at_20_std value: 24.530799671005095 - type: nauc_map_at_3_diff1 value: 27.24864828341282 - type: nauc_map_at_3_max value: 30.612466018878496 - type: nauc_map_at_3_std value: 21.28508055272999 - type: nauc_map_at_5_diff1 value: 26.896012525725325 - type: nauc_map_at_5_max value: 31.048230953772975 - type: nauc_map_at_5_std value: 22.36923819012987 - type: nauc_mrr_at_1000_diff1 value: 27.03321771604688 - type: nauc_mrr_at_1000_max value: 29.318016872314356 - type: nauc_mrr_at_1000_std value: 27.255662227981702 - type: nauc_mrr_at_100_diff1 value: 27.01645813593333 - type: nauc_mrr_at_100_max value: 29.30840557667501 - type: nauc_mrr_at_100_std value: 27.260764047829472 - type: nauc_mrr_at_10_diff1 value: 27.009721260288906 - type: nauc_mrr_at_10_max value: 29.467580933154487 - type: nauc_mrr_at_10_std value: 27.03289830021139 - type: nauc_mrr_at_1_diff1 value: 31.155140837576678 - type: nauc_mrr_at_1_max value: 28.451108545079713 - type: nauc_mrr_at_1_std value: 25.763588681961014 - type: nauc_mrr_at_20_diff1 value: 26.99961056015129 - type: nauc_mrr_at_20_max value: 29.361859812964674 - type: nauc_mrr_at_20_std value: 27.258149714708598 - type: nauc_mrr_at_3_diff1 value: 26.418761490514214 - type: nauc_mrr_at_3_max value: 29.04645399007843 - type: nauc_mrr_at_3_std value: 26.57078938243618 - type: nauc_mrr_at_5_diff1 value: 26.828957822719413 - type: nauc_mrr_at_5_max value: 29.183119511596573 - type: nauc_mrr_at_5_std value: 26.632517222850595 - type: nauc_ndcg_at_1000_diff1 value: 23.763944477348627 - type: nauc_ndcg_at_1000_max value: 30.4849287035792 - type: nauc_ndcg_at_1000_std value: 28.607171984717837 - type: nauc_ndcg_at_100_diff1 value: 23.771597354347985 - type: nauc_ndcg_at_100_max value: 30.840324918263008 - type: nauc_ndcg_at_100_std value: 28.353814075332277 - type: nauc_ndcg_at_10_diff1 value: 24.635557021112522 - type: nauc_ndcg_at_10_max value: 31.45726246077928 - type: nauc_ndcg_at_10_std value: 25.66603542922466 - type: nauc_ndcg_at_1_diff1 value: 31.155140837576678 - type: nauc_ndcg_at_1_max value: 28.451108545079713 - type: nauc_ndcg_at_1_std value: 25.763588681961014 - type: nauc_ndcg_at_20_diff1 value: 24.374570394904733 - type: nauc_ndcg_at_20_max value: 31.21182863225155 - type: nauc_ndcg_at_20_std value: 27.516107222806703 - type: nauc_ndcg_at_3_diff1 value: 25.204639941707107 - type: nauc_ndcg_at_3_max value: 29.821897512710354 - type: nauc_ndcg_at_3_std value: 22.955368886721388 - type: nauc_ndcg_at_5_diff1 value: 25.628786786945394 - type: nauc_ndcg_at_5_max value: 30.75374910801621 - type: nauc_ndcg_at_5_std value: 23.77602081355407 - type: nauc_precision_at_1000_diff1 value: -8.296864442882004 - type: nauc_precision_at_1000_max value: -1.0863639124110083 - type: nauc_precision_at_1000_std value: 13.942768215751009 - type: nauc_precision_at_100_diff1 value: -1.3641094219864045 - type: nauc_precision_at_100_max value: 9.389565817464774 - type: nauc_precision_at_100_std value: 20.969714828784273 - type: nauc_precision_at_10_diff1 value: 7.699985574715798 - type: nauc_precision_at_10_max value: 21.22938970938099 - type: nauc_precision_at_10_std value: 24.244673965667534 - type: nauc_precision_at_1_diff1 value: 31.155140837576678 - type: nauc_precision_at_1_max value: 28.451108545079713 - type: nauc_precision_at_1_std value: 25.763588681961014 - type: nauc_precision_at_20_diff1 value: 4.833145200843495 - type: nauc_precision_at_20_max value: 16.887995253179778 - type: nauc_precision_at_20_std value: 25.795764951079676 - type: nauc_precision_at_3_diff1 value: 14.206196520456535 - type: nauc_precision_at_3_max value: 25.787350079153164 - type: nauc_precision_at_3_std value: 23.690498209580376 - type: nauc_precision_at_5_diff1 value: 12.32708818053103 - type: nauc_precision_at_5_max value: 24.032115473559603 - type: nauc_precision_at_5_std value: 23.484892204331782 - type: nauc_recall_at_1000_diff1 value: 8.22006170896915 - type: nauc_recall_at_1000_max value: 18.81264480377967 - type: nauc_recall_at_1000_std value: 32.283741333393124 - type: nauc_recall_at_100_diff1 value: 12.18567832527598 - type: nauc_recall_at_100_max value: 23.099403731211336 - type: nauc_recall_at_100_std value: 28.36911088287291 - type: nauc_recall_at_10_diff1 value: 17.700146749912946 - type: nauc_recall_at_10_max value: 28.284625319657376 - type: nauc_recall_at_10_std value: 23.48564062964447 - type: nauc_recall_at_1_diff1 value: 34.73062154387364 - type: nauc_recall_at_1_max value: 30.574707039457493 - type: nauc_recall_at_1_std value: 19.436488371927435 - type: nauc_recall_at_20_diff1 value: 16.156165277785433 - type: nauc_recall_at_20_max value: 26.281037479122688 - type: nauc_recall_at_20_std value: 26.74972643606532 - type: nauc_recall_at_3_diff1 value: 21.750067698676038 - type: nauc_recall_at_3_max value: 28.754878789472237 - type: nauc_recall_at_3_std value: 20.01663586142874 - type: nauc_recall_at_5_diff1 value: 20.892218178864958 - type: nauc_recall_at_5_max value: 28.124407488104154 - type: nauc_recall_at_5_std value: 20.61434989716216 - type: ndcg_at_1 value: 38.958 - type: ndcg_at_10 value: 36.964000000000006 - type: ndcg_at_100 value: 44.115 - type: ndcg_at_1000 value: 46.796 - type: ndcg_at_20 value: 40.062 - type: ndcg_at_3 value: 31.316 - type: ndcg_at_5 value: 33.211 - type: precision_at_1 value: 38.958 - type: precision_at_10 value: 11.153 - type: precision_at_100 value: 1.8769999999999998 - type: precision_at_1000 value: 0.23900000000000002 - type: precision_at_20 value: 6.898999999999999 - type: precision_at_3 value: 22.91 - type: precision_at_5 value: 17.316000000000003 - type: recall_at_1 value: 16.888 - type: recall_at_10 value: 42.161 - type: recall_at_100 value: 66.102 - type: recall_at_1000 value: 81.026 - type: recall_at_20 value: 50.86000000000001 - type: recall_at_3 value: 27.598 - type: recall_at_5 value: 33.814 task: type: Retrieval - dataset: config: default name: MTEB DBPedia (default) revision: c0f706b76e590d620bd6618b3ca8efdd34e2d659 split: test type: mteb/dbpedia metrics: - type: main_score value: 47.156 - type: map_at_1 value: 10.148 - type: map_at_10 value: 22.569 - type: map_at_100 value: 31.448999999999998 - type: map_at_1000 value: 33.44 - type: map_at_20 value: 26.069 - type: map_at_3 value: 16.014999999999997 - type: map_at_5 value: 18.901 - type: mrr_at_1 value: 76.5 - type: mrr_at_10 value: 82.2888888888889 - type: mrr_at_100 value: 82.49538251419122 - type: mrr_at_1000 value: 82.5021397953138 - type: mrr_at_20 value: 82.4252913752914 - type: mrr_at_3 value: 81.16666666666667 - type: mrr_at_5 value: 81.74166666666667 - type: nauc_map_at_1000_diff1 value: 23.301407358655897 - type: nauc_map_at_1000_max value: 31.424515733703277 - type: nauc_map_at_1000_std value: 12.046348501470613 - type: nauc_map_at_100_diff1 value: 24.334834238228208 - type: nauc_map_at_100_max value: 32.08530087581391 - type: nauc_map_at_100_std value: 9.739338172156248 - type: nauc_map_at_10_diff1 value: 24.73875979445744 - type: nauc_map_at_10_max value: 33.276940524471236 - type: nauc_map_at_10_std value: -9.164629695967152 - type: nauc_map_at_1_diff1 value: 32.00102576762782 - type: nauc_map_at_1_max value: 33.51205113985759 - type: nauc_map_at_1_std value: -23.44857278246293 - type: nauc_map_at_20_diff1 value: 24.778913289961395 - type: nauc_map_at_20_max value: 33.448516812943105 - type: nauc_map_at_20_std value: -2.6683319171528352 - type: nauc_map_at_3_diff1 value: 25.23076431439413 - type: nauc_map_at_3_max value: 31.154454837782296 - type: nauc_map_at_3_std value: -19.385936810006037 - type: nauc_map_at_5_diff1 value: 24.259635425617436 - type: nauc_map_at_5_max value: 31.729395753450955 - type: nauc_map_at_5_std value: -16.127594389625504 - type: nauc_mrr_at_1000_diff1 value: 55.03942750204178 - type: nauc_mrr_at_1000_max value: 50.96490431671482 - type: nauc_mrr_at_1000_std value: 25.836343538312683 - type: nauc_mrr_at_100_diff1 value: 55.04552440551358 - type: nauc_mrr_at_100_max value: 50.963427582835784 - type: nauc_mrr_at_100_std value: 25.831153320454614 - type: nauc_mrr_at_10_diff1 value: 55.229048800522484 - type: nauc_mrr_at_10_max value: 51.3390557615654 - type: nauc_mrr_at_10_std value: 25.922071812454657 - type: nauc_mrr_at_1_diff1 value: 55.37295537295538 - type: nauc_mrr_at_1_max value: 50.218350218350224 - type: nauc_mrr_at_1_std value: 21.025971025971035 - type: nauc_mrr_at_20_diff1 value: 55.00593357106184 - type: nauc_mrr_at_20_max value: 51.05472083367735 - type: nauc_mrr_at_20_std value: 25.909336886328717 - type: nauc_mrr_at_3_diff1 value: 54.446322775722045 - type: nauc_mrr_at_3_max value: 50.46360292721918 - type: nauc_mrr_at_3_std value: 26.329787827966765 - type: nauc_mrr_at_5_diff1 value: 55.12344889931978 - type: nauc_mrr_at_5_max value: 50.56903196971039 - type: nauc_mrr_at_5_std value: 26.55026150180298 - type: nauc_ndcg_at_1000_diff1 value: 30.228357984090167 - type: nauc_ndcg_at_1000_max value: 36.2083465618033 - type: nauc_ndcg_at_1000_std value: 27.053206106506984 - type: nauc_ndcg_at_100_diff1 value: 31.703353354291863 - type: nauc_ndcg_at_100_max value: 36.373859031948 - type: nauc_ndcg_at_100_std value: 20.068846970727698 - type: nauc_ndcg_at_10_diff1 value: 32.256110136975494 - type: nauc_ndcg_at_10_max value: 38.99767005264608 - type: nauc_ndcg_at_10_std value: 17.039892454085155 - type: nauc_ndcg_at_1_diff1 value: 47.26276005028265 - type: nauc_ndcg_at_1_max value: 41.38306374405809 - type: nauc_ndcg_at_1_std value: 14.684984307489964 - type: nauc_ndcg_at_20_diff1 value: 31.76825501599332 - type: nauc_ndcg_at_20_max value: 38.20899873887362 - type: nauc_ndcg_at_20_std value: 13.097118845724665 - type: nauc_ndcg_at_3_diff1 value: 31.321786216514703 - type: nauc_ndcg_at_3_max value: 35.981146886743026 - type: nauc_ndcg_at_3_std value: 18.307636914108354 - type: nauc_ndcg_at_5_diff1 value: 29.66664100781043 - type: nauc_ndcg_at_5_max value: 37.59729053135147 - type: nauc_ndcg_at_5_std value: 17.373905471488126 - type: nauc_precision_at_1000_diff1 value: -18.87091581184578 - type: nauc_precision_at_1000_max value: -18.696227759596617 - type: nauc_precision_at_1000_std value: 4.226333183482085 - type: nauc_precision_at_100_diff1 value: -5.287216009706794 - type: nauc_precision_at_100_max value: -4.887951805802842 - type: nauc_precision_at_100_std value: 35.6235756181467 - type: nauc_precision_at_10_diff1 value: 5.858724870109219 - type: nauc_precision_at_10_max value: 12.913194030354791 - type: nauc_precision_at_10_std value: 38.69077081708489 - type: nauc_precision_at_1_diff1 value: 55.37295537295538 - type: nauc_precision_at_1_max value: 50.218350218350224 - type: nauc_precision_at_1_std value: 21.025971025971035 - type: nauc_precision_at_20_diff1 value: 2.1852822564080414 - type: nauc_precision_at_20_max value: 7.877715960706996 - type: nauc_precision_at_20_std value: 39.87418052705391 - type: nauc_precision_at_3_diff1 value: 15.964890926148051 - type: nauc_precision_at_3_max value: 20.914035316890114 - type: nauc_precision_at_3_std value: 26.831455070123145 - type: nauc_precision_at_5_diff1 value: 8.337345796875258 - type: nauc_precision_at_5_max value: 17.75805675007055 - type: nauc_precision_at_5_std value: 32.10856342736335 - type: nauc_recall_at_1000_diff1 value: 17.846516157012843 - type: nauc_recall_at_1000_max value: 17.900524965136565 - type: nauc_recall_at_1000_std value: 33.98517407615005 - type: nauc_recall_at_100_diff1 value: 21.018315900271688 - type: nauc_recall_at_100_max value: 22.94790843604604 - type: nauc_recall_at_100_std value: 18.03552806307113 - type: nauc_recall_at_10_diff1 value: 22.18291118606378 - type: nauc_recall_at_10_max value: 30.252113761422056 - type: nauc_recall_at_10_std value: -10.179895110312067 - type: nauc_recall_at_1_diff1 value: 32.00102576762782 - type: nauc_recall_at_1_max value: 33.51205113985759 - type: nauc_recall_at_1_std value: -23.44857278246293 - type: nauc_recall_at_20_diff1 value: 21.6294261882514 - type: nauc_recall_at_20_max value: 29.28972334436445 - type: nauc_recall_at_20_std value: -3.252679363030184 - type: nauc_recall_at_3_diff1 value: 22.94021002974748 - type: nauc_recall_at_3_max value: 29.08903130551997 - type: nauc_recall_at_3_std value: -19.552669466489 - type: nauc_recall_at_5_diff1 value: 21.74684484136416 - type: nauc_recall_at_5_max value: 28.266794468484495 - type: nauc_recall_at_5_std value: -17.0802549689716 - type: ndcg_at_1 value: 63.5 - type: ndcg_at_10 value: 47.156 - type: ndcg_at_100 value: 51.564 - type: ndcg_at_1000 value: 59.386 - type: ndcg_at_20 value: 46.233999999999995 - type: ndcg_at_3 value: 52.90899999999999 - type: ndcg_at_5 value: 49.482 - type: precision_at_1 value: 76.5 - type: precision_at_10 value: 36.875 - type: precision_at_100 value: 11.591999999999999 - type: precision_at_1000 value: 2.346 - type: precision_at_20 value: 27.612 - type: precision_at_3 value: 56.083000000000006 - type: precision_at_5 value: 46.949999999999996 - type: recall_at_1 value: 10.148 - type: recall_at_10 value: 28.183999999999997 - type: recall_at_100 value: 57.187 - type: recall_at_1000 value: 82.069 - type: recall_at_20 value: 36.02 - type: recall_at_3 value: 17.31 - type: recall_at_5 value: 21.711 task: type: Retrieval - dataset: config: default name: MTEB EmotionClassification (default) revision: 4f58c6b202a23cf9a4da393831edf4f9183cad37 split: test type: mteb/emotion metrics: - type: accuracy value: 53.449999999999996 - type: f1 value: 47.565489310302986 - type: f1_weighted value: 55.143079860495234 - type: main_score value: 53.449999999999996 task: type: Classification - dataset: config: default name: MTEB FEVER (default) revision: bea83ef9e8fb933d90a2f1d5515737465d613e12 split: test type: mteb/fever metrics: - type: main_score value: 92.201 - type: map_at_1 value: 84.672 - type: map_at_10 value: 89.755 - type: map_at_100 value: 89.90899999999999 - type: map_at_1000 value: 89.92099999999999 - type: map_at_20 value: 89.84100000000001 - type: map_at_3 value: 89.226 - type: map_at_5 value: 89.527 - type: mrr_at_1 value: 91.1941194119412 - type: mrr_at_10 value: 94.87830925949736 - type: mrr_at_100 value: 94.90049591414446 - type: mrr_at_1000 value: 94.9007044904392 - type: mrr_at_20 value: 94.89512092080201 - type: mrr_at_3 value: 94.67196719671963 - type: mrr_at_5 value: 94.79947994799474 - type: nauc_map_at_1000_diff1 value: 35.639838917786946 - type: nauc_map_at_1000_max value: 13.71831703137999 - type: nauc_map_at_1000_std value: -16.244788042624585 - type: nauc_map_at_100_diff1 value: 35.597291964012435 - type: nauc_map_at_100_max value: 13.707968885019728 - type: nauc_map_at_100_std value: -16.234475561207322 - type: nauc_map_at_10_diff1 value: 35.456175745479825 - type: nauc_map_at_10_max value: 13.886262397124549 - type: nauc_map_at_10_std value: -16.267873880349104 - type: nauc_map_at_1_diff1 value: 46.05069433503191 - type: nauc_map_at_1_max value: 14.89528908470166 - type: nauc_map_at_1_std value: -18.236070781621756 - type: nauc_map_at_20_diff1 value: 35.451928451516864 - type: nauc_map_at_20_max value: 13.6810017695262 - type: nauc_map_at_20_std value: -16.207941145958856 - type: nauc_map_at_3_diff1 value: 35.63821451113733 - type: nauc_map_at_3_max value: 14.50663022315242 - type: nauc_map_at_3_std value: -17.576242065815734 - type: nauc_map_at_5_diff1 value: 35.39067191316313 - type: nauc_map_at_5_max value: 13.95304590199052 - type: nauc_map_at_5_std value: -16.951256853731795 - type: nauc_mrr_at_1000_diff1 value: 75.84258563766511 - type: nauc_mrr_at_1000_max value: 21.62062081403913 - type: nauc_mrr_at_1000_std value: -41.83920347093032 - type: nauc_mrr_at_100_diff1 value: 75.8435737070719 - type: nauc_mrr_at_100_max value: 21.623097020990052 - type: nauc_mrr_at_100_std value: -41.83508809623058 - type: nauc_mrr_at_10_diff1 value: 75.81801484107942 - type: nauc_mrr_at_10_max value: 21.704185197428245 - type: nauc_mrr_at_10_std value: -42.03425200091995 - type: nauc_mrr_at_1_diff1 value: 77.04724957066034 - type: nauc_mrr_at_1_max value: 22.024144657121404 - type: nauc_mrr_at_1_std value: -35.10421745431855 - type: nauc_mrr_at_20_diff1 value: 75.83479830947383 - type: nauc_mrr_at_20_max value: 21.58622779021691 - type: nauc_mrr_at_20_std value: -41.86125228142864 - type: nauc_mrr_at_3_diff1 value: 75.48936568920693 - type: nauc_mrr_at_3_max value: 22.342677145925645 - type: nauc_mrr_at_3_std value: -44.18742305905927 - type: nauc_mrr_at_5_diff1 value: 75.65824242070917 - type: nauc_mrr_at_5_max value: 21.663850758591447 - type: nauc_mrr_at_5_std value: -43.1894070995383 - type: nauc_ndcg_at_1000_diff1 value: 37.14673606903847 - type: nauc_ndcg_at_1000_max value: 13.880930479763675 - type: nauc_ndcg_at_1000_std value: -16.65051678433904 - type: nauc_ndcg_at_100_diff1 value: 36.05555838296192 - type: nauc_ndcg_at_100_max value: 13.635131503635712 - type: nauc_ndcg_at_100_std value: -16.229637436428554 - type: nauc_ndcg_at_10_diff1 value: 35.339471530051725 - type: nauc_ndcg_at_10_max value: 14.068641881473336 - type: nauc_ndcg_at_10_std value: -16.57935046771473 - type: nauc_ndcg_at_1_diff1 value: 77.04724957066034 - type: nauc_ndcg_at_1_max value: 22.024144657121404 - type: nauc_ndcg_at_1_std value: -35.10421745431855 - type: nauc_ndcg_at_20_diff1 value: 35.24453482200621 - type: nauc_ndcg_at_20_max value: 13.358263743060084 - type: nauc_ndcg_at_20_std value: -16.142097913894858 - type: nauc_ndcg_at_3_diff1 value: 37.293476898887505 - type: nauc_ndcg_at_3_max value: 15.133932930960345 - type: nauc_ndcg_at_3_std value: -21.422125651374348 - type: nauc_ndcg_at_5_diff1 value: 35.7967558978059 - type: nauc_ndcg_at_5_max value: 14.153790043028987 - type: nauc_ndcg_at_5_std value: -19.007127645863324 - type: nauc_precision_at_1000_diff1 value: -6.961103885855311 - type: nauc_precision_at_1000_max value: -7.54526392119496 - type: nauc_precision_at_1000_std value: 2.4581095173539143 - type: nauc_precision_at_100_diff1 value: -10.61256188833412 - type: nauc_precision_at_100_max value: -8.41405913180417 - type: nauc_precision_at_100_std value: 4.237708359203656 - type: nauc_precision_at_10_diff1 value: -10.614577160630317 - type: nauc_precision_at_10_max value: -5.167597481869389 - type: nauc_precision_at_10_std value: 1.140168377848993 - type: nauc_precision_at_1_diff1 value: 77.04724957066034 - type: nauc_precision_at_1_max value: 22.024144657121404 - type: nauc_precision_at_1_std value: -35.10421745431855 - type: nauc_precision_at_20_diff1 value: -12.461125809680928 - type: nauc_precision_at_20_max value: -8.635206490614799 - type: nauc_precision_at_20_std value: 3.514319507686466 - type: nauc_precision_at_3_diff1 value: 0.904547422970022 - type: nauc_precision_at_3_max value: 4.762429279213669 - type: nauc_precision_at_3_std value: -17.03444137257749 - type: nauc_precision_at_5_diff1 value: -6.19509635703273 - type: nauc_precision_at_5_max value: -1.748040103739541 - type: nauc_precision_at_5_std value: -8.398471580131982 - type: nauc_recall_at_1000_diff1 value: -24.618899403404722 - type: nauc_recall_at_1000_max value: 2.2601153418804993 - type: nauc_recall_at_1000_std value: 37.967286945129594 - type: nauc_recall_at_100_diff1 value: -16.707602974172563 - type: nauc_recall_at_100_max value: 2.5960324943305695 - type: nauc_recall_at_100_std value: 23.36149042284208 - type: nauc_recall_at_10_diff1 value: -0.9129283126796908 - type: nauc_recall_at_10_max value: 7.721727797820066 - type: nauc_recall_at_10_std value: 5.965839799258808 - type: nauc_recall_at_1_diff1 value: 46.05069433503191 - type: nauc_recall_at_1_max value: 14.89528908470166 - type: nauc_recall_at_1_std value: -18.236070781621756 - type: nauc_recall_at_20_diff1 value: -6.870488981132769 - type: nauc_recall_at_20_max value: 3.020122380835828 - type: nauc_recall_at_20_std value: 12.269552351901797 - type: nauc_recall_at_3_diff1 value: 11.772652803808322 - type: nauc_recall_at_3_max value: 12.467973905249256 - type: nauc_recall_at_3_std value: -12.480260715760009 - type: nauc_recall_at_5_diff1 value: 6.0800516850026405 - type: nauc_recall_at_5_max value: 9.078645485005183 - type: nauc_recall_at_5_std value: -6.26939010430003 - type: ndcg_at_1 value: 91.194 - type: ndcg_at_10 value: 92.201 - type: ndcg_at_100 value: 92.714 - type: ndcg_at_1000 value: 92.91199999999999 - type: ndcg_at_20 value: 92.407 - type: ndcg_at_3 value: 91.56800000000001 - type: ndcg_at_5 value: 91.838 - type: precision_at_1 value: 91.194 - type: precision_at_10 value: 10.485999999999999 - type: precision_at_100 value: 1.093 - type: precision_at_1000 value: 0.11299999999999999 - type: precision_at_20 value: 5.3100000000000005 - type: precision_at_3 value: 33.833 - type: precision_at_5 value: 20.585 - type: recall_at_1 value: 84.672 - type: recall_at_10 value: 94.878 - type: recall_at_100 value: 96.851 - type: recall_at_1000 value: 98.00999999999999 - type: recall_at_20 value: 95.55900000000001 - type: recall_at_3 value: 92.925 - type: recall_at_5 value: 93.811 task: type: Retrieval - dataset: config: default name: MTEB FiQA2018 (default) revision: 27a168819829fe9bcd655c2df245fb19452e8e06 split: test type: mteb/fiqa metrics: - type: main_score value: 55.846 - type: map_at_1 value: 28.559 - type: map_at_10 value: 47.788000000000004 - type: map_at_100 value: 50.005 - type: map_at_1000 value: 50.135 - type: map_at_20 value: 49.136 - type: map_at_3 value: 42.069 - type: map_at_5 value: 45.359 - type: mrr_at_1 value: 54.166666666666664 - type: mrr_at_10 value: 63.058801195375246 - type: mrr_at_100 value: 63.736243940040104 - type: mrr_at_1000 value: 63.76207658887102 - type: mrr_at_20 value: 63.5120384043063 - type: mrr_at_3 value: 61.419753086419746 - type: mrr_at_5 value: 62.569444444444414 - type: nauc_map_at_1000_diff1 value: 44.69490283021471 - type: nauc_map_at_1000_max value: 30.92551782663574 - type: nauc_map_at_1000_std value: -14.430206995603301 - type: nauc_map_at_100_diff1 value: 44.69185357435577 - type: nauc_map_at_100_max value: 30.862237715726366 - type: nauc_map_at_100_std value: -14.372571056295163 - type: nauc_map_at_10_diff1 value: 45.132555549285684 - type: nauc_map_at_10_max value: 29.69986917324885 - type: nauc_map_at_10_std value: -15.667505260541766 - type: nauc_map_at_1_diff1 value: 54.024618817771106 - type: nauc_map_at_1_max value: 16.708530893815503 - type: nauc_map_at_1_std value: -16.498600116938213 - type: nauc_map_at_20_diff1 value: 44.77799946034473 - type: nauc_map_at_20_max value: 30.299416183664622 - type: nauc_map_at_20_std value: -15.294677224633338 - type: nauc_map_at_3_diff1 value: 47.7216165906905 - type: nauc_map_at_3_max value: 24.80525467300038 - type: nauc_map_at_3_std value: -16.167977354068125 - type: nauc_map_at_5_diff1 value: 46.02953711254381 - type: nauc_map_at_5_max value: 27.79201453498519 - type: nauc_map_at_5_std value: -15.862235643724656 - type: nauc_mrr_at_1000_diff1 value: 52.604857791228795 - type: nauc_mrr_at_1000_max value: 41.90760486398485 - type: nauc_mrr_at_1000_std value: -11.645858368861806 - type: nauc_mrr_at_100_diff1 value: 52.592257747310256 - type: nauc_mrr_at_100_max value: 41.90215764910407 - type: nauc_mrr_at_100_std value: -11.629701977702393 - type: nauc_mrr_at_10_diff1 value: 52.41749778055747 - type: nauc_mrr_at_10_max value: 41.848571254096726 - type: nauc_mrr_at_10_std value: -11.666002358568836 - type: nauc_mrr_at_1_diff1 value: 56.38878064687782 - type: nauc_mrr_at_1_max value: 41.723130889319506 - type: nauc_mrr_at_1_std value: -14.860577950564089 - type: nauc_mrr_at_20_diff1 value: 52.46272520475529 - type: nauc_mrr_at_20_max value: 41.8213025754279 - type: nauc_mrr_at_20_std value: -11.733990850254274 - type: nauc_mrr_at_3_diff1 value: 52.907019830141664 - type: nauc_mrr_at_3_max value: 42.091778179711596 - type: nauc_mrr_at_3_std value: -12.083789626498938 - type: nauc_mrr_at_5_diff1 value: 52.504019426379934 - type: nauc_mrr_at_5_max value: 42.18926010896201 - type: nauc_mrr_at_5_std value: -11.681910264950588 - type: nauc_ndcg_at_1000_diff1 value: 45.106796590190186 - type: nauc_ndcg_at_1000_max value: 35.1491443230931 - type: nauc_ndcg_at_1000_std value: -10.564391346254856 - type: nauc_ndcg_at_100_diff1 value: 45.0013474629376 - type: nauc_ndcg_at_100_max value: 34.41256574321341 - type: nauc_ndcg_at_100_std value: -9.633883528747441 - type: nauc_ndcg_at_10_diff1 value: 45.237721582816334 - type: nauc_ndcg_at_10_max value: 32.26898539095568 - type: nauc_ndcg_at_10_std value: -13.9237235021985 - type: nauc_ndcg_at_1_diff1 value: 56.38878064687782 - type: nauc_ndcg_at_1_max value: 41.723130889319506 - type: nauc_ndcg_at_1_std value: -14.860577950564089 - type: nauc_ndcg_at_20_diff1 value: 44.79397541599229 - type: nauc_ndcg_at_20_max value: 32.63346133703614 - type: nauc_ndcg_at_20_std value: -13.134715591222845 - type: nauc_ndcg_at_3_diff1 value: 44.2180519401759 - type: nauc_ndcg_at_3_max value: 34.12433670269642 - type: nauc_ndcg_at_3_std value: -14.270883602867066 - type: nauc_ndcg_at_5_diff1 value: 44.7842185764849 - type: nauc_ndcg_at_5_max value: 33.07177981305266 - type: nauc_ndcg_at_5_std value: -14.727492973620834 - type: nauc_precision_at_1000_diff1 value: -20.913994966271765 - type: nauc_precision_at_1000_max value: 23.20465985071573 - type: nauc_precision_at_1000_std value: 12.925926506702536 - type: nauc_precision_at_100_diff1 value: -15.403005704192823 - type: nauc_precision_at_100_max value: 26.893804588333875 - type: nauc_precision_at_100_std value: 15.444177311713084 - type: nauc_precision_at_10_diff1 value: -0.3346158191962367 - type: nauc_precision_at_10_max value: 34.35784817973049 - type: nauc_precision_at_10_std value: 1.564875689584266 - type: nauc_precision_at_1_diff1 value: 56.38878064687782 - type: nauc_precision_at_1_max value: 41.723130889319506 - type: nauc_precision_at_1_std value: -14.860577950564089 - type: nauc_precision_at_20_diff1 value: -7.911475407210952 - type: nauc_precision_at_20_max value: 30.98272615969314 - type: nauc_precision_at_20_std value: 5.867214521505607 - type: nauc_precision_at_3_diff1 value: 16.75568275861896 - type: nauc_precision_at_3_max value: 37.24231639527263 - type: nauc_precision_at_3_std value: -3.9715436715209314 - type: nauc_precision_at_5_diff1 value: 6.630969568351265 - type: nauc_precision_at_5_max value: 37.17946067183075 - type: nauc_precision_at_5_std value: -1.043726756895724 - type: nauc_recall_at_1000_diff1 value: 13.830553206891198 - type: nauc_recall_at_1000_max value: 27.133157795437697 - type: nauc_recall_at_1000_std value: 48.60055204311751 - type: nauc_recall_at_100_diff1 value: 31.861798106199263 - type: nauc_recall_at_100_max value: 23.85176506448154 - type: nauc_recall_at_100_std value: 13.429501664456197 - type: nauc_recall_at_10_diff1 value: 35.84630942635496 - type: nauc_recall_at_10_max value: 22.13537652643054 - type: nauc_recall_at_10_std value: -12.889731755986558 - type: nauc_recall_at_1_diff1 value: 54.024618817771106 - type: nauc_recall_at_1_max value: 16.708530893815503 - type: nauc_recall_at_1_std value: -16.498600116938213 - type: nauc_recall_at_20_diff1 value: 32.12586341954912 - type: nauc_recall_at_20_max value: 20.180665442485186 - type: nauc_recall_at_20_std value: -10.417213522381246 - type: nauc_recall_at_3_diff1 value: 42.32867586873781 - type: nauc_recall_at_3_max value: 20.365513905388248 - type: nauc_recall_at_3_std value: -15.533455093420095 - type: nauc_recall_at_5_diff1 value: 38.37329209164264 - type: nauc_recall_at_5_max value: 22.003563642599406 - type: nauc_recall_at_5_std value: -14.300434277371249 - type: ndcg_at_1 value: 54.167 - type: ndcg_at_10 value: 55.846 - type: ndcg_at_100 value: 62.427 - type: ndcg_at_1000 value: 64.301 - type: ndcg_at_20 value: 58.858 - type: ndcg_at_3 value: 52.29899999999999 - type: ndcg_at_5 value: 53.535 - type: precision_at_1 value: 54.167 - type: precision_at_10 value: 15.386 - type: precision_at_100 value: 2.244 - type: precision_at_1000 value: 0.256 - type: precision_at_20 value: 9.043 - type: precision_at_3 value: 35.391 - type: precision_at_5 value: 25.679000000000002 - type: recall_at_1 value: 28.559 - type: recall_at_10 value: 62.746 - type: recall_at_100 value: 85.943 - type: recall_at_1000 value: 97.111 - type: recall_at_20 value: 71.857 - type: recall_at_3 value: 47.905 - type: recall_at_5 value: 55.083000000000006 task: type: Retrieval - dataset: config: default name: MTEB HotpotQA (default) revision: ab518f4d6fcca38d87c25209f94beba119d02014 split: test type: mteb/hotpotqa metrics: - type: main_score value: 77.125 - type: map_at_1 value: 41.519 - type: map_at_10 value: 70.167 - type: map_at_100 value: 70.953 - type: map_at_1000 value: 71.001 - type: map_at_20 value: 70.649 - type: map_at_3 value: 66.981 - type: map_at_5 value: 69.015 - type: mrr_at_1 value: 83.03848750844024 - type: mrr_at_10 value: 87.86450596443821 - type: mrr_at_100 value: 87.99248423178447 - type: mrr_at_1000 value: 87.99763635592048 - type: mrr_at_20 value: 87.94595084887314 - type: mrr_at_3 value: 87.16857978843102 - type: mrr_at_5 value: 87.64528471753292 - type: nauc_map_at_1000_diff1 value: 13.006772427678182 - type: nauc_map_at_1000_max value: 19.27772881602346 - type: nauc_map_at_1000_std value: -1.6365617870914253 - type: nauc_map_at_100_diff1 value: 12.989359854738897 - type: nauc_map_at_100_max value: 19.26281165622185 - type: nauc_map_at_100_std value: -1.593792957025615 - type: nauc_map_at_10_diff1 value: 12.534914210271793 - type: nauc_map_at_10_max value: 19.10074296644962 - type: nauc_map_at_10_std value: -2.08708846366053 - type: nauc_map_at_1_diff1 value: 65.02315378915155 - type: nauc_map_at_1_max value: 51.17274340822432 - type: nauc_map_at_1_std value: -13.958643399099477 - type: nauc_map_at_20_diff1 value: 12.789367750628891 - type: nauc_map_at_20_max value: 19.196329125088766 - type: nauc_map_at_20_std value: -1.6469320227872484 - type: nauc_map_at_3_diff1 value: 11.729016028480842 - type: nauc_map_at_3_max value: 18.934857415045407 - type: nauc_map_at_3_std value: -5.093979543429253 - type: nauc_map_at_5_diff1 value: 12.276578116231432 - type: nauc_map_at_5_max value: 19.136131837434224 - type: nauc_map_at_5_std value: -3.132205288648261 - type: nauc_mrr_at_1000_diff1 value: 64.62877037309876 - type: nauc_mrr_at_1000_max value: 53.98693739367828 - type: nauc_mrr_at_1000_std value: -9.126474060405949 - type: nauc_mrr_at_100_diff1 value: 64.62922052398424 - type: nauc_mrr_at_100_max value: 53.988818580376716 - type: nauc_mrr_at_100_std value: -9.112000794260997 - type: nauc_mrr_at_10_diff1 value: 64.67968982658613 - type: nauc_mrr_at_10_max value: 54.09549018870873 - type: nauc_mrr_at_10_std value: -9.085391215905886 - type: nauc_mrr_at_1_diff1 value: 65.02315378915155 - type: nauc_mrr_at_1_max value: 51.17274340822432 - type: nauc_mrr_at_1_std value: -13.958643399099477 - type: nauc_mrr_at_20_diff1 value: 64.64602458319244 - type: nauc_mrr_at_20_max value: 54.04291820548144 - type: nauc_mrr_at_20_std value: -9.029239521036427 - type: nauc_mrr_at_3_diff1 value: 64.12171234144306 - type: nauc_mrr_at_3_max value: 54.20212686844387 - type: nauc_mrr_at_3_std value: -9.46453489594666 - type: nauc_mrr_at_5_diff1 value: 64.5365566477813 - type: nauc_mrr_at_5_max value: 54.192604824973614 - type: nauc_mrr_at_5_std value: -8.94640972357742 - type: nauc_ndcg_at_1000_diff1 value: 20.6624137699915 - type: nauc_ndcg_at_1000_max value: 24.209126006971047 - type: nauc_ndcg_at_1000_std value: 1.1807098633960282 - type: nauc_ndcg_at_100_diff1 value: 20.013258997064227 - type: nauc_ndcg_at_100_max value: 23.720082477112257 - type: nauc_ndcg_at_100_std value: 2.2242860498709613 - type: nauc_ndcg_at_10_diff1 value: 18.09695752050781 - type: nauc_ndcg_at_10_max value: 22.98525969470443 - type: nauc_ndcg_at_10_std value: 0.2786454782720934 - type: nauc_ndcg_at_1_diff1 value: 65.02315378915155 - type: nauc_ndcg_at_1_max value: 51.17274340822432 - type: nauc_ndcg_at_1_std value: -13.958643399099477 - type: nauc_ndcg_at_20_diff1 value: 18.753834550561898 - type: nauc_ndcg_at_20_max value: 23.203931192654537 - type: nauc_ndcg_at_20_std value: 1.682937143064539 - type: nauc_ndcg_at_3_diff1 value: 17.167189283772277 - type: nauc_ndcg_at_3_max value: 23.118180393133674 - type: nauc_ndcg_at_3_std value: -4.535717324455032 - type: nauc_ndcg_at_5_diff1 value: 17.699969883446347 - type: nauc_ndcg_at_5_max value: 23.182339354183792 - type: nauc_ndcg_at_5_std value: -1.6781755997000052 - type: nauc_precision_at_1000_diff1 value: 7.5788444847542245 - type: nauc_precision_at_1000_max value: 9.433104189270141 - type: nauc_precision_at_1000_std value: 44.591190042314686 - type: nauc_precision_at_100_diff1 value: 6.692654365804252 - type: nauc_precision_at_100_max value: 11.25176923344072 - type: nauc_precision_at_100_std value: 31.693966326856067 - type: nauc_precision_at_10_diff1 value: 4.635248064775999 - type: nauc_precision_at_10_max value: 13.589851125207542 - type: nauc_precision_at_10_std value: 10.894403687664266 - type: nauc_precision_at_1_diff1 value: 65.02315378915155 - type: nauc_precision_at_1_max value: 51.17274340822432 - type: nauc_precision_at_1_std value: -13.958643399099477 - type: nauc_precision_at_20_diff1 value: 4.525274589346107 - type: nauc_precision_at_20_max value: 12.438591940128317 - type: nauc_precision_at_20_std value: 18.15861880489477 - type: nauc_precision_at_3_diff1 value: 6.184298058843799 - type: nauc_precision_at_3_max value: 16.54134519266211 - type: nauc_precision_at_3_std value: -1.4484102465558355 - type: nauc_precision_at_5_diff1 value: 5.657946113014627 - type: nauc_precision_at_5_max value: 15.559958816376259 - type: nauc_precision_at_5_std value: 4.5741247848934 - type: nauc_recall_at_1000_diff1 value: 7.57884448475466 - type: nauc_recall_at_1000_max value: 9.433104189270699 - type: nauc_recall_at_1000_std value: 44.59119004231575 - type: nauc_recall_at_100_diff1 value: 6.692654365804161 - type: nauc_recall_at_100_max value: 11.251769233440859 - type: nauc_recall_at_100_std value: 31.69396632685608 - type: nauc_recall_at_10_diff1 value: 4.6352480647758165 - type: nauc_recall_at_10_max value: 13.589851125207373 - type: nauc_recall_at_10_std value: 10.89440368766406 - type: nauc_recall_at_1_diff1 value: 65.02315378915155 - type: nauc_recall_at_1_max value: 51.17274340822432 - type: nauc_recall_at_1_std value: -13.958643399099477 - type: nauc_recall_at_20_diff1 value: 4.525274589346308 - type: nauc_recall_at_20_max value: 12.43859194012829 - type: nauc_recall_at_20_std value: 18.158618804894918 - type: nauc_recall_at_3_diff1 value: 6.18429805884378 - type: nauc_recall_at_3_max value: 16.541345192662053 - type: nauc_recall_at_3_std value: -1.448410246555865 - type: nauc_recall_at_5_diff1 value: 5.657946113014677 - type: nauc_recall_at_5_max value: 15.559958816376168 - type: nauc_recall_at_5_std value: 4.5741247848933995 - type: ndcg_at_1 value: 83.038 - type: ndcg_at_10 value: 77.125 - type: ndcg_at_100 value: 79.714 - type: ndcg_at_1000 value: 80.589 - type: ndcg_at_20 value: 78.277 - type: ndcg_at_3 value: 72.78099999999999 - type: ndcg_at_5 value: 75.274 - type: precision_at_1 value: 83.038 - type: precision_at_10 value: 16.104 - type: precision_at_100 value: 1.81 - type: precision_at_1000 value: 0.192 - type: precision_at_20 value: 8.421 - type: precision_at_3 value: 47.327999999999996 - type: precision_at_5 value: 30.358 - type: recall_at_1 value: 41.519 - type: recall_at_10 value: 80.52 - type: recall_at_100 value: 90.506 - type: recall_at_1000 value: 96.219 - type: recall_at_20 value: 84.21300000000001 - type: recall_at_3 value: 70.993 - type: recall_at_5 value: 75.895 task: type: Retrieval - dataset: config: default name: MTEB ImdbClassification (default) revision: 3d86128a09e091d6018b6d26cad27f2739fc2db7 split: test type: mteb/imdb metrics: - type: accuracy value: 87.502 - type: ap value: 82.95784261019377 - type: ap_weighted value: 82.95784261019377 - type: f1 value: 87.46268628092736 - type: f1_weighted value: 87.46268628092734 - type: main_score value: 87.502 task: type: Classification - dataset: config: default name: MTEB MSMARCO (default) revision: c5a29a104738b98a9e76336939199e264163d4a0 split: dev type: mteb/msmarco metrics: - type: main_score value: 42.828 - type: map_at_1 value: 22.622 - type: map_at_10 value: 35.551 - type: map_at_100 value: 36.720000000000006 - type: map_at_1000 value: 36.764 - type: map_at_20 value: 36.289 - type: map_at_3 value: 31.452 - type: map_at_5 value: 33.78 - type: mrr_at_1 value: 23.30945558739255 - type: mrr_at_10 value: 36.1412311820621 - type: mrr_at_100 value: 37.24636679692144 - type: mrr_at_1000 value: 37.28460384375999 - type: mrr_at_20 value: 36.846278028371025 - type: mrr_at_3 value: 32.13228271251181 - type: mrr_at_5 value: 34.41380133715374 - type: nauc_map_at_1000_diff1 value: 34.255139266678064 - type: nauc_map_at_1000_max value: 27.634109755448193 - type: nauc_map_at_1000_std value: -25.068179763907832 - type: nauc_map_at_100_diff1 value: 34.2417213419493 - type: nauc_map_at_100_max value: 27.65433689860232 - type: nauc_map_at_100_std value: -25.039117799889272 - type: nauc_map_at_10_diff1 value: 34.28723089283392 - type: nauc_map_at_10_max value: 27.860718359060744 - type: nauc_map_at_10_std value: -25.751446063547345 - type: nauc_map_at_1_diff1 value: 38.198938023289145 - type: nauc_map_at_1_max value: 23.254490119071715 - type: nauc_map_at_1_std value: -22.980905207183305 - type: nauc_map_at_20_diff1 value: 34.26024735371797 - type: nauc_map_at_20_max value: 27.75693143222282 - type: nauc_map_at_20_std value: -25.28503577794799 - type: nauc_map_at_3_diff1 value: 34.64447883282737 - type: nauc_map_at_3_max value: 26.378811567506215 - type: nauc_map_at_3_std value: -25.67491148988595 - type: nauc_map_at_5_diff1 value: 34.29213028848551 - type: nauc_map_at_5_max value: 27.34485307415968 - type: nauc_map_at_5_std value: -25.85010405437415 - type: nauc_mrr_at_1000_diff1 value: 33.86421338143246 - type: nauc_mrr_at_1000_max value: 27.250892398176134 - type: nauc_mrr_at_1000_std value: -24.929427212418865 - type: nauc_mrr_at_100_diff1 value: 33.85402401030439 - type: nauc_mrr_at_100_max value: 27.27120328136017 - type: nauc_mrr_at_100_std value: -24.902391477950804 - type: nauc_mrr_at_10_diff1 value: 33.880770277581384 - type: nauc_mrr_at_10_max value: 27.441949715579018 - type: nauc_mrr_at_10_std value: -25.580715458382443 - type: nauc_mrr_at_1_diff1 value: 37.6771342755869 - type: nauc_mrr_at_1_max value: 23.265052651660735 - type: nauc_mrr_at_1_std value: -23.205472826319582 - type: nauc_mrr_at_20_diff1 value: 33.85060604038063 - type: nauc_mrr_at_20_max value: 27.354473158418273 - type: nauc_mrr_at_20_std value: -25.12086278884105 - type: nauc_mrr_at_3_diff1 value: 34.06405277224216 - type: nauc_mrr_at_3_max value: 26.07372143403643 - type: nauc_mrr_at_3_std value: -25.630362103104886 - type: nauc_mrr_at_5_diff1 value: 33.87118592722288 - type: nauc_mrr_at_5_max value: 26.975769667003075 - type: nauc_mrr_at_5_std value: -25.71929161450833 - type: nauc_ndcg_at_1000_diff1 value: 33.22331426038353 - type: nauc_ndcg_at_1000_max value: 28.608268643287936 - type: nauc_ndcg_at_1000_std value: -24.044160312444436 - type: nauc_ndcg_at_100_diff1 value: 32.922164280838416 - type: nauc_ndcg_at_100_max value: 29.21462172377906 - type: nauc_ndcg_at_100_std value: -23.030348617039646 - type: nauc_ndcg_at_10_diff1 value: 33.09358216672861 - type: nauc_ndcg_at_10_max value: 30.195095713195773 - type: nauc_ndcg_at_10_std value: -26.59142800601194 - type: nauc_ndcg_at_1_diff1 value: 37.6771342755869 - type: nauc_ndcg_at_1_max value: 23.265052651660735 - type: nauc_ndcg_at_1_std value: -23.205472826319582 - type: nauc_ndcg_at_20_diff1 value: 32.94672668778977 - type: nauc_ndcg_at_20_max value: 29.953670242101982 - type: nauc_ndcg_at_20_std value: -24.813603915035287 - type: nauc_ndcg_at_3_diff1 value: 33.62562972482792 - type: nauc_ndcg_at_3_max value: 27.190259593091398 - type: nauc_ndcg_at_3_std value: -26.515769994261813 - type: nauc_ndcg_at_5_diff1 value: 33.10136789340702 - type: nauc_ndcg_at_5_max value: 28.837591134098755 - type: nauc_ndcg_at_5_std value: -26.780772701125983 - type: nauc_precision_at_1000_diff1 value: -9.899137556294614 - type: nauc_precision_at_1000_max value: -4.36970723719003 - type: nauc_precision_at_1000_std value: 8.958402933111124 - type: nauc_precision_at_100_diff1 value: 6.280164246108402 - type: nauc_precision_at_100_max value: 20.31023470088511 - type: nauc_precision_at_100_std value: 8.411373550208506 - type: nauc_precision_at_10_diff1 value: 25.404046800053337 - type: nauc_precision_at_10_max value: 35.785692630641734 - type: nauc_precision_at_10_std value: -27.68021725976897 - type: nauc_precision_at_1_diff1 value: 37.6771342755869 - type: nauc_precision_at_1_max value: 23.265052651660735 - type: nauc_precision_at_1_std value: -23.205472826319582 - type: nauc_precision_at_20_diff1 value: 21.065313055028774 - type: nauc_precision_at_20_max value: 33.42637158232517 - type: nauc_precision_at_20_std value: -17.587879787399963 - type: nauc_precision_at_3_diff1 value: 30.05809515417773 - type: nauc_precision_at_3_max value: 29.186749260489574 - type: nauc_precision_at_3_std value: -28.68718050967194 - type: nauc_precision_at_5_diff1 value: 27.643220081226616 - type: nauc_precision_at_5_max value: 32.24514921367616 - type: nauc_precision_at_5_std value: -28.86571961903398 - type: nauc_recall_at_1000_diff1 value: -4.845074018125273 - type: nauc_recall_at_1000_max value: 29.244169860264673 - type: nauc_recall_at_1000_std value: 66.46444727873201 - type: nauc_recall_at_100_diff1 value: 21.726863329795037 - type: nauc_recall_at_100_max value: 42.20994961358766 - type: nauc_recall_at_100_std value: 11.221444032745431 - type: nauc_recall_at_10_diff1 value: 29.421574107699655 - type: nauc_recall_at_10_max value: 38.66483553848921 - type: nauc_recall_at_10_std value: -29.12606967024905 - type: nauc_recall_at_1_diff1 value: 38.198938023289145 - type: nauc_recall_at_1_max value: 23.254490119071715 - type: nauc_recall_at_1_std value: -22.980905207183305 - type: nauc_recall_at_20_diff1 value: 27.651691823296915 - type: nauc_recall_at_20_max value: 40.03031738570312 - type: nauc_recall_at_20_std value: -20.543175657258416 - type: nauc_recall_at_3_diff1 value: 31.198692423845504 - type: nauc_recall_at_3_max value: 29.39596634981627 - type: nauc_recall_at_3_std value: -28.616355439292363 - type: nauc_recall_at_5_diff1 value: 29.866348488833548 - type: nauc_recall_at_5_max value: 33.13370189331543 - type: nauc_recall_at_5_std value: -29.203314508482975 - type: ndcg_at_1 value: 23.308999999999997 - type: ndcg_at_10 value: 42.828 - type: ndcg_at_100 value: 48.378 - type: ndcg_at_1000 value: 49.448 - type: ndcg_at_20 value: 45.454 - type: ndcg_at_3 value: 34.495 - type: ndcg_at_5 value: 38.631 - type: precision_at_1 value: 23.308999999999997 - type: precision_at_10 value: 6.834 - type: precision_at_100 value: 0.96 - type: precision_at_1000 value: 0.105 - type: precision_at_20 value: 3.966 - type: precision_at_3 value: 14.761 - type: precision_at_5 value: 10.968 - type: recall_at_1 value: 22.622 - type: recall_at_10 value: 65.281 - type: recall_at_100 value: 90.753 - type: recall_at_1000 value: 98.89099999999999 - type: recall_at_20 value: 75.497 - type: recall_at_3 value: 42.631 - type: recall_at_5 value: 52.537 task: type: Retrieval - dataset: config: en name: MTEB MTOPDomainClassification (en) revision: d80d48c1eb48d3562165c59d59d0034df9fff0bf split: test type: mteb/mtop_domain metrics: - type: accuracy value: 95.39899680802554 - type: f1 value: 95.07663901231138 - type: f1_weighted value: 95.41776447870696 - type: main_score value: 95.39899680802554 task: type: Classification - dataset: config: en name: MTEB MTOPIntentClassification (en) revision: ae001d0e6b1228650b7bd1c2c65fb50ad11a8aba split: test type: mteb/mtop_intent metrics: - type: accuracy value: 78.61833105335158 - type: f1 value: 59.28800280436582 - type: f1_weighted value: 80.57433767622484 - type: main_score value: 78.61833105335158 task: type: Classification - dataset: config: en name: MTEB MassiveIntentClassification (en) revision: 4672e20407010da34463acc759c162ca9734bca6 split: test type: mteb/amazon_massive_intent metrics: - type: accuracy value: 77.96906523201076 - type: f1 value: 77.08935874634084 - type: f1_weighted value: 77.21652401469349 - type: main_score value: 77.96906523201076 task: type: Classification - dataset: config: en name: MTEB MassiveScenarioClassification (en) revision: fad2c6e8459f9e1c45d9315f4953d921437d70f8 split: test type: mteb/amazon_massive_scenario metrics: - type: accuracy value: 80.13786146603901 - type: f1 value: 79.4209677252264 - type: f1_weighted value: 80.02557645073855 - type: main_score value: 80.13786146603901 task: type: Classification - dataset: config: default name: MTEB MedrxivClusteringP2P (default) revision: e7a26af6f3ae46b30dde8737f02c07b1505bcc73 split: test type: mteb/medrxiv-clustering-p2p metrics: - type: main_score value: 32.51760527055797 - type: v_measure value: 32.51760527055797 - type: v_measure_std value: 1.6034905723582067 task: type: Clustering - dataset: config: default name: MTEB MedrxivClusteringS2S (default) revision: 35191c8c0dca72d8ff3efcd72aa802307d469663 split: test type: mteb/medrxiv-clustering-s2s metrics: - type: main_score value: 32.82947207274322 - type: v_measure value: 32.82947207274322 - type: v_measure_std value: 1.4989355394854873 task: type: Clustering - dataset: config: default name: MTEB MindSmallReranking (default) revision: 59042f120c80e8afa9cdbb224f67076cec0fc9a7 split: test type: mteb/mind_small metrics: - type: main_score value: 32.31611300891167 - type: map value: 32.31611300891167 - type: mrr value: 33.511223415752724 - type: nAUC_map_diff1 value: 11.450478664025022 - type: nAUC_map_max value: -20.133408006057575 - type: nAUC_map_std value: 1.4219207720032518 - type: nAUC_mrr_diff1 value: 10.75655549847738 - type: nAUC_mrr_max value: -15.057198169120886 - type: nAUC_mrr_std value: 3.135802935705323 task: type: Reranking - dataset: config: default name: MTEB NFCorpus (default) revision: ec0fa4fe99da2ff19ca1214b7966684033a58814 split: test type: mteb/nfcorpus metrics: - type: main_score value: 40.897 - type: map_at_1 value: 6.502 - type: map_at_10 value: 15.42 - type: map_at_100 value: 20.230999999999998 - type: map_at_1000 value: 21.97 - type: map_at_20 value: 17.288999999999998 - type: map_at_3 value: 11.023 - type: map_at_5 value: 13.043 - type: mrr_at_1 value: 51.39318885448917 - type: mrr_at_10 value: 60.90458990613791 - type: mrr_at_100 value: 61.30822147543883 - type: mrr_at_1000 value: 61.348380045065255 - type: mrr_at_20 value: 61.17699278343732 - type: mrr_at_3 value: 58.926728586171336 - type: mrr_at_5 value: 60.18059855521158 - type: nauc_map_at_1000_diff1 value: 22.314584774002412 - type: nauc_map_at_1000_max value: 25.641377755476196 - type: nauc_map_at_1000_std value: 11.600298129762008 - type: nauc_map_at_100_diff1 value: 24.373091310053177 - type: nauc_map_at_100_max value: 25.210868210148224 - type: nauc_map_at_100_std value: 8.080464969348098 - type: nauc_map_at_10_diff1 value: 27.966938127237427 - type: nauc_map_at_10_max value: 19.50242948342481 - type: nauc_map_at_10_std value: -3.2987968208791165 - type: nauc_map_at_1_diff1 value: 46.77689461811175 - type: nauc_map_at_1_max value: 5.992563122661309 - type: nauc_map_at_1_std value: -19.87851166453543 - type: nauc_map_at_20_diff1 value: 26.613502043659892 - type: nauc_map_at_20_max value: 22.01981060253049 - type: nauc_map_at_20_std value: 0.35709748629600185 - type: nauc_map_at_3_diff1 value: 35.07956999136986 - type: nauc_map_at_3_max value: 10.09189113221904 - type: nauc_map_at_3_std value: -12.806383375188329 - type: nauc_map_at_5_diff1 value: 31.171056747033298 - type: nauc_map_at_5_max value: 13.891688163973285 - type: nauc_map_at_5_std value: -9.749824390421193 - type: nauc_mrr_at_1000_diff1 value: 29.800909959137623 - type: nauc_mrr_at_1000_max value: 33.341220744025435 - type: nauc_mrr_at_1000_std value: 21.75001312294901 - type: nauc_mrr_at_100_diff1 value: 29.800367929516835 - type: nauc_mrr_at_100_max value: 33.38136058844821 - type: nauc_mrr_at_100_std value: 21.789223788566236 - type: nauc_mrr_at_10_diff1 value: 29.974925130970114 - type: nauc_mrr_at_10_max value: 33.432216655498976 - type: nauc_mrr_at_10_std value: 21.74795266714447 - type: nauc_mrr_at_1_diff1 value: 32.8838135925804 - type: nauc_mrr_at_1_max value: 24.529371483084574 - type: nauc_mrr_at_1_std value: 9.58748035277067 - type: nauc_mrr_at_20_diff1 value: 29.91438913779757 - type: nauc_mrr_at_20_max value: 33.28744302059532 - type: nauc_mrr_at_20_std value: 21.724201609379104 - type: nauc_mrr_at_3_diff1 value: 30.166535157699588 - type: nauc_mrr_at_3_max value: 31.75339502993889 - type: nauc_mrr_at_3_std value: 20.75073010210299 - type: nauc_mrr_at_5_diff1 value: 30.11312282641972 - type: nauc_mrr_at_5_max value: 33.67582756088135 - type: nauc_mrr_at_5_std value: 21.686983222728376 - type: nauc_ndcg_at_1000_diff1 value: 20.035313336303506 - type: nauc_ndcg_at_1000_max value: 39.86024861036285 - type: nauc_ndcg_at_1000_std value: 30.06202908899349 - type: nauc_ndcg_at_100_diff1 value: 17.713593456491267 - type: nauc_ndcg_at_100_max value: 33.98291104786461 - type: nauc_ndcg_at_100_std value: 24.365237125638483 - type: nauc_ndcg_at_10_diff1 value: 13.96135837037495 - type: nauc_ndcg_at_10_max value: 31.474413824866204 - type: nauc_ndcg_at_10_std value: 22.86874340960288 - type: nauc_ndcg_at_1_diff1 value: 32.68811889497608 - type: nauc_ndcg_at_1_max value: 23.17817936378513 - type: nauc_ndcg_at_1_std value: 10.111894875706865 - type: nauc_ndcg_at_20_diff1 value: 13.69750453207742 - type: nauc_ndcg_at_20_max value: 29.238632311676156 - type: nauc_ndcg_at_20_std value: 21.51101902939643 - type: nauc_ndcg_at_3_diff1 value: 18.142662080015292 - type: nauc_ndcg_at_3_max value: 28.79319631593914 - type: nauc_ndcg_at_3_std value: 20.37950822552368 - type: nauc_ndcg_at_5_diff1 value: 15.4836510230026 - type: nauc_ndcg_at_5_max value: 29.524697678622612 - type: nauc_ndcg_at_5_std value: 20.31186496372464 - type: nauc_precision_at_1000_diff1 value: -22.60022337362534 - type: nauc_precision_at_1000_max value: 0.5297084926025946 - type: nauc_precision_at_1000_std value: 31.86960110378292 - type: nauc_precision_at_100_diff1 value: -17.109296942324217 - type: nauc_precision_at_100_max value: 13.114543385872182 - type: nauc_precision_at_100_std value: 40.546888665997436 - type: nauc_precision_at_10_diff1 value: -5.751184011272755 - type: nauc_precision_at_10_max value: 30.397788247842115 - type: nauc_precision_at_10_std value: 36.40734041302498 - type: nauc_precision_at_1_diff1 value: 32.09040033058441 - type: nauc_precision_at_1_max value: 25.115800505270425 - type: nauc_precision_at_1_std value: 11.185512689606364 - type: nauc_precision_at_20_diff1 value: -8.865872005542506 - type: nauc_precision_at_20_max value: 24.858906642791396 - type: nauc_precision_at_20_std value: 37.31331026229662 - type: nauc_precision_at_3_diff1 value: 6.279790630752277 - type: nauc_precision_at_3_max value: 31.18333274155458 - type: nauc_precision_at_3_std value: 28.5156938516134 - type: nauc_precision_at_5_diff1 value: -0.5685414700385947 - type: nauc_precision_at_5_max value: 30.032601466710123 - type: nauc_precision_at_5_std value: 29.72096748351155 - type: nauc_recall_at_1000_diff1 value: 8.024431478610873 - type: nauc_recall_at_1000_max value: 26.25396667158672 - type: nauc_recall_at_1000_std value: 17.773467989292417 - type: nauc_recall_at_100_diff1 value: 9.825317515542215 - type: nauc_recall_at_100_max value: 25.786552000740272 - type: nauc_recall_at_100_std value: 14.229170280235248 - type: nauc_recall_at_10_diff1 value: 18.41325848063197 - type: nauc_recall_at_10_max value: 20.97636763770323 - type: nauc_recall_at_10_std value: -1.3953906116846648 - type: nauc_recall_at_1_diff1 value: 46.77689461811175 - type: nauc_recall_at_1_max value: 5.992563122661309 - type: nauc_recall_at_1_std value: -19.87851166453543 - type: nauc_recall_at_20_diff1 value: 14.796541894273506 - type: nauc_recall_at_20_max value: 17.963697314215956 - type: nauc_recall_at_20_std value: -1.713455868691847 - type: nauc_recall_at_3_diff1 value: 30.30471830588464 - type: nauc_recall_at_3_max value: 9.339098307835556 - type: nauc_recall_at_3_std value: -11.089655201238893 - type: nauc_recall_at_5_diff1 value: 24.446476829886976 - type: nauc_recall_at_5_max value: 15.769417011561993 - type: nauc_recall_at_5_std value: -7.8192403091680145 - type: ndcg_at_1 value: 50.0 - type: ndcg_at_10 value: 40.897 - type: ndcg_at_100 value: 37.849 - type: ndcg_at_1000 value: 46.666999999999994 - type: ndcg_at_20 value: 38.224000000000004 - type: ndcg_at_3 value: 45.964 - type: ndcg_at_5 value: 44.157999999999994 - type: precision_at_1 value: 51.702999999999996 - type: precision_at_10 value: 31.084 - type: precision_at_100 value: 9.913 - type: precision_at_1000 value: 2.316 - type: precision_at_20 value: 22.833000000000002 - type: precision_at_3 value: 43.24 - type: precision_at_5 value: 39.133 - type: recall_at_1 value: 6.502 - type: recall_at_10 value: 20.165 - type: recall_at_100 value: 38.957 - type: recall_at_1000 value: 71.273 - type: recall_at_20 value: 25.21 - type: recall_at_3 value: 12.458 - type: recall_at_5 value: 15.519 task: type: Retrieval - dataset: config: default name: MTEB NQ (default) revision: b774495ed302d8c44a3a7ea25c90dbce03968f31 split: test type: mteb/nq metrics: - type: main_score value: 65.44 - type: map_at_1 value: 40.571 - type: map_at_10 value: 57.969 - type: map_at_100 value: 58.728 - type: map_at_1000 value: 58.739 - type: map_at_20 value: 58.51 - type: map_at_3 value: 53.748 - type: map_at_5 value: 56.54900000000001 - type: mrr_at_1 value: 45.74159907300116 - type: mrr_at_10 value: 60.22695607791187 - type: mrr_at_100 value: 60.74267440494396 - type: mrr_at_1000 value: 60.749624106896846 - type: mrr_at_20 value: 60.59501256435345 - type: mrr_at_3 value: 57.08285052143659 - type: mrr_at_5 value: 59.252607184240844 - type: nauc_map_at_1000_diff1 value: 37.171845352505656 - type: nauc_map_at_1000_max value: 15.432607015597297 - type: nauc_map_at_1000_std value: -11.97932899920431 - type: nauc_map_at_100_diff1 value: 37.16682506561907 - type: nauc_map_at_100_max value: 15.437190638823806 - type: nauc_map_at_100_std value: -11.975557698168156 - type: nauc_map_at_10_diff1 value: 37.192551151144194 - type: nauc_map_at_10_max value: 15.753997478818714 - type: nauc_map_at_10_std value: -12.40974603357028 - type: nauc_map_at_1_diff1 value: 40.34581435107217 - type: nauc_map_at_1_max value: 11.236223915633456 - type: nauc_map_at_1_std value: -11.972657299364364 - type: nauc_map_at_20_diff1 value: 37.161885832329325 - type: nauc_map_at_20_max value: 15.470469711929697 - type: nauc_map_at_20_std value: -12.060519221962725 - type: nauc_map_at_3_diff1 value: 37.065913320170644 - type: nauc_map_at_3_max value: 15.314893136101853 - type: nauc_map_at_3_std value: -12.937730813736737 - type: nauc_map_at_5_diff1 value: 36.65247215835913 - type: nauc_map_at_5_max value: 15.63968636667063 - type: nauc_map_at_5_std value: -12.833328148854456 - type: nauc_mrr_at_1000_diff1 value: 37.79278360150881 - type: nauc_mrr_at_1000_max value: 15.572097376565196 - type: nauc_mrr_at_1000_std value: -10.348084506439804 - type: nauc_mrr_at_100_diff1 value: 37.786865896948825 - type: nauc_mrr_at_100_max value: 15.575893210789642 - type: nauc_mrr_at_100_std value: -10.341461472904756 - type: nauc_mrr_at_10_diff1 value: 37.70769184810465 - type: nauc_mrr_at_10_max value: 15.848696020672007 - type: nauc_mrr_at_10_std value: -10.478264744120263 - type: nauc_mrr_at_1_diff1 value: 40.680357536723974 - type: nauc_mrr_at_1_max value: 12.418751441192686 - type: nauc_mrr_at_1_std value: -9.868157052983221 - type: nauc_mrr_at_20_diff1 value: 37.77469856387031 - type: nauc_mrr_at_20_max value: 15.613476149089827 - type: nauc_mrr_at_20_std value: -10.350727767432192 - type: nauc_mrr_at_3_diff1 value: 37.67813738445281 - type: nauc_mrr_at_3_max value: 15.752643632881952 - type: nauc_mrr_at_3_std value: -10.709782839058049 - type: nauc_mrr_at_5_diff1 value: 37.41714382663168 - type: nauc_mrr_at_5_max value: 15.910297855025743 - type: nauc_mrr_at_5_std value: -10.780431722579811 - type: nauc_ndcg_at_1000_diff1 value: 36.84355056294766 - type: nauc_ndcg_at_1000_max value: 16.16466414816963 - type: nauc_ndcg_at_1000_std value: -10.754459767111936 - type: nauc_ndcg_at_100_diff1 value: 36.658503024942654 - type: nauc_ndcg_at_100_max value: 16.278958071287068 - type: nauc_ndcg_at_100_std value: -10.430455391889696 - type: nauc_ndcg_at_10_diff1 value: 36.52384188024424 - type: nauc_ndcg_at_10_max value: 17.61214827030915 - type: nauc_ndcg_at_10_std value: -11.870026397604043 - type: nauc_ndcg_at_1_diff1 value: 40.680357536723974 - type: nauc_ndcg_at_1_max value: 12.418751441192686 - type: nauc_ndcg_at_1_std value: -9.868157052983221 - type: nauc_ndcg_at_20_diff1 value: 36.57525846877024 - type: nauc_ndcg_at_20_max value: 16.570059599819693 - type: nauc_ndcg_at_20_std value: -10.740934396121888 - type: nauc_ndcg_at_3_diff1 value: 36.270540924932114 - type: nauc_ndcg_at_3_max value: 16.702218429064946 - type: nauc_ndcg_at_3_std value: -12.749903314668357 - type: nauc_ndcg_at_5_diff1 value: 35.41435543582453 - type: nauc_ndcg_at_5_max value: 17.330255799030986 - type: nauc_ndcg_at_5_std value: -12.761837100653455 - type: nauc_precision_at_1000_diff1 value: -9.434969469177616 - type: nauc_precision_at_1000_max value: 0.8279994242627504 - type: nauc_precision_at_1000_std value: 16.293409606648844 - type: nauc_precision_at_100_diff1 value: -7.884252124549447 - type: nauc_precision_at_100_max value: 2.645686939190529 - type: nauc_precision_at_100_std value: 17.366169984302203 - type: nauc_precision_at_10_diff1 value: 7.290688773146785 - type: nauc_precision_at_10_max value: 13.268047958073753 - type: nauc_precision_at_10_std value: 4.450363229854657 - type: nauc_precision_at_1_diff1 value: 40.680357536723974 - type: nauc_precision_at_1_max value: 12.418751441192686 - type: nauc_precision_at_1_std value: -9.868157052983221 - type: nauc_precision_at_20_diff1 value: 0.23713971775484072 - type: nauc_precision_at_20_max value: 7.536520203232905 - type: nauc_precision_at_20_std value: 12.182206974457724 - type: nauc_precision_at_3_diff1 value: 21.039494672443197 - type: nauc_precision_at_3_max value: 17.553973358369372 - type: nauc_precision_at_3_std value: -5.012633324245794 - type: nauc_precision_at_5_diff1 value: 12.102914882846234 - type: nauc_precision_at_5_max value: 15.856278497759096 - type: nauc_precision_at_5_std value: -1.5482564850321778 - type: nauc_recall_at_1000_diff1 value: 20.012897379043007 - type: nauc_recall_at_1000_max value: 56.73005884879772 - type: nauc_recall_at_1000_std value: 59.152380526066885 - type: nauc_recall_at_100_diff1 value: 18.446836069732065 - type: nauc_recall_at_100_max value: 29.993570692813982 - type: nauc_recall_at_100_std value: 28.41587000672941 - type: nauc_recall_at_10_diff1 value: 30.605039392458387 - type: nauc_recall_at_10_max value: 27.738218801407278 - type: nauc_recall_at_10_std value: -12.508590794323988 - type: nauc_recall_at_1_diff1 value: 40.34581435107217 - type: nauc_recall_at_1_max value: 11.236223915633456 - type: nauc_recall_at_1_std value: -11.972657299364364 - type: nauc_recall_at_20_diff1 value: 29.244236606603284 - type: nauc_recall_at_20_max value: 23.284403303254493 - type: nauc_recall_at_20_std value: -2.8338618437665617 - type: nauc_recall_at_3_diff1 value: 32.129243232419086 - type: nauc_recall_at_3_max value: 20.119033875551796 - type: nauc_recall_at_3_std value: -14.240312656172122 - type: nauc_recall_at_5_diff1 value: 27.763518445624864 - type: nauc_recall_at_5_max value: 23.172013642098666 - type: nauc_recall_at_5_std value: -15.04492351689741 - type: ndcg_at_1 value: 45.742 - type: ndcg_at_10 value: 65.44 - type: ndcg_at_100 value: 68.377 - type: ndcg_at_1000 value: 68.619 - type: ndcg_at_20 value: 67.093 - type: ndcg_at_3 value: 57.98799999999999 - type: ndcg_at_5 value: 62.49400000000001 - type: precision_at_1 value: 45.742 - type: precision_at_10 value: 10.278 - type: precision_at_100 value: 1.191 - type: precision_at_1000 value: 0.121 - type: precision_at_20 value: 5.542 - type: precision_at_3 value: 26.149 - type: precision_at_5 value: 18.308 - type: recall_at_1 value: 40.571 - type: recall_at_10 value: 85.6 - type: recall_at_100 value: 97.955 - type: recall_at_1000 value: 99.754 - type: recall_at_20 value: 91.594 - type: recall_at_3 value: 66.862 - type: recall_at_5 value: 77.16300000000001 task: type: Retrieval - dataset: config: default name: MTEB QuoraRetrieval (default) revision: e4e08e0b7dbe3c8700f0daef558ff32256715259 split: test type: mteb/quora metrics: - type: main_score value: 89.983 - type: map_at_1 value: 72.333 - type: map_at_10 value: 86.64500000000001 - type: map_at_100 value: 87.27000000000001 - type: map_at_1000 value: 87.28 - type: map_at_20 value: 87.07000000000001 - type: map_at_3 value: 83.69 - type: map_at_5 value: 85.616 - type: mrr_at_1 value: 83.17999999999999 - type: mrr_at_10 value: 88.92811111111087 - type: mrr_at_100 value: 89.01533250070398 - type: mrr_at_1000 value: 89.01591712027928 - type: mrr_at_20 value: 88.99762238367897 - type: mrr_at_3 value: 88.05499999999975 - type: mrr_at_5 value: 88.68599999999968 - type: nauc_map_at_1000_diff1 value: 77.41719971001054 - type: nauc_map_at_1000_max value: 50.82056919544904 - type: nauc_map_at_1000_std value: -66.72165494301285 - type: nauc_map_at_100_diff1 value: 77.42352497743428 - type: nauc_map_at_100_max value: 50.82093289011799 - type: nauc_map_at_100_std value: -66.78049718823219 - type: nauc_map_at_10_diff1 value: 77.74624557071816 - type: nauc_map_at_10_max value: 50.79536454457503 - type: nauc_map_at_10_std value: -69.71456042954112 - type: nauc_map_at_1_diff1 value: 82.68164323521295 - type: nauc_map_at_1_max value: 38.954818287207324 - type: nauc_map_at_1_std value: -56.68166218708859 - type: nauc_map_at_20_diff1 value: 77.55325844438951 - type: nauc_map_at_20_max value: 50.84356384001606 - type: nauc_map_at_20_std value: -67.98703548320766 - type: nauc_map_at_3_diff1 value: 78.85574278943115 - type: nauc_map_at_3_max value: 48.52471401555999 - type: nauc_map_at_3_std value: -71.30299825100148 - type: nauc_map_at_5_diff1 value: 78.16471107015477 - type: nauc_map_at_5_max value: 50.30985590553375 - type: nauc_map_at_5_std value: -71.33684372158167 - type: nauc_mrr_at_1000_diff1 value: 77.62295824280032 - type: nauc_mrr_at_1000_max value: 51.80058309016847 - type: nauc_mrr_at_1000_std value: -62.37197972047579 - type: nauc_mrr_at_100_diff1 value: 77.62234522673566 - type: nauc_mrr_at_100_max value: 51.801934646829274 - type: nauc_mrr_at_100_std value: -62.37322038621588 - type: nauc_mrr_at_10_diff1 value: 77.59747253748361 - type: nauc_mrr_at_10_max value: 51.87334226198358 - type: nauc_mrr_at_10_std value: -62.64200232227941 - type: nauc_mrr_at_1_diff1 value: 78.5768801049508 - type: nauc_mrr_at_1_max value: 50.903137386851384 - type: nauc_mrr_at_1_std value: -57.80272717360846 - type: nauc_mrr_at_20_diff1 value: 77.62208305068552 - type: nauc_mrr_at_20_max value: 51.828904905587606 - type: nauc_mrr_at_20_std value: -62.433948957948374 - type: nauc_mrr_at_3_diff1 value: 77.38155121033225 - type: nauc_mrr_at_3_max value: 51.68106631416263 - type: nauc_mrr_at_3_std value: -63.45491126025229 - type: nauc_mrr_at_5_diff1 value: 77.5183226181727 - type: nauc_mrr_at_5_max value: 51.97540888258937 - type: nauc_mrr_at_5_std value: -63.01365659630005 - type: nauc_ndcg_at_1000_diff1 value: 77.0968099860845 - type: nauc_ndcg_at_1000_max value: 51.35967273719281 - type: nauc_ndcg_at_1000_std value: -64.58895061628968 - type: nauc_ndcg_at_100_diff1 value: 77.1099003765838 - type: nauc_ndcg_at_100_max value: 51.418685751945894 - type: nauc_ndcg_at_100_std value: -64.82586438143728 - type: nauc_ndcg_at_10_diff1 value: 77.29986674759448 - type: nauc_ndcg_at_10_max value: 51.8605146908119 - type: nauc_ndcg_at_10_std value: -69.97528521090494 - type: nauc_ndcg_at_1_diff1 value: 78.49711376568688 - type: nauc_ndcg_at_1_max value: 51.0794152100377 - type: nauc_ndcg_at_1_std value: -57.536843107052185 - type: nauc_ndcg_at_20_diff1 value: 77.30637560599035 - type: nauc_ndcg_at_20_max value: 51.68220059769314 - type: nauc_ndcg_at_20_std value: -67.76406452812292 - type: nauc_ndcg_at_3_diff1 value: 76.86431160054454 - type: nauc_ndcg_at_3_max value: 50.749560195519614 - type: nauc_ndcg_at_3_std value: -69.45815827753151 - type: nauc_ndcg_at_5_diff1 value: 77.14024715468048 - type: nauc_ndcg_at_5_max value: 51.930214613990465 - type: nauc_ndcg_at_5_std value: -70.77018788982289 - type: nauc_precision_at_1000_diff1 value: -47.44532117687583 - type: nauc_precision_at_1000_max value: -17.299891917223214 - type: nauc_precision_at_1000_std value: 52.51591233187087 - type: nauc_precision_at_100_diff1 value: -47.265828632688155 - type: nauc_precision_at_100_max value: -16.71812733047803 - type: nauc_precision_at_100_std value: 51.51757632883793 - type: nauc_precision_at_10_diff1 value: -43.237865438735575 - type: nauc_precision_at_10_max value: -9.883793141400469 - type: nauc_precision_at_10_std value: 33.86730977694455 - type: nauc_precision_at_1_diff1 value: 78.49711376568688 - type: nauc_precision_at_1_max value: 51.0794152100377 - type: nauc_precision_at_1_std value: -57.536843107052185 - type: nauc_precision_at_20_diff1 value: -45.7649305042739 - type: nauc_precision_at_20_max value: -13.839318564567368 - type: nauc_precision_at_20_std value: 42.705695137308446 - type: nauc_precision_at_3_diff1 value: -25.060485489006908 - type: nauc_precision_at_3_max value: 4.882317523957804 - type: nauc_precision_at_3_std value: 5.301548406664098 - type: nauc_precision_at_5_diff1 value: -36.934659107383084 - type: nauc_precision_at_5_max value: -3.599741390594996 - type: nauc_precision_at_5_std value: 21.13020599488238 - type: nauc_recall_at_1000_diff1 value: .nan - type: nauc_recall_at_1000_max value: .nan - type: nauc_recall_at_1000_std value: .nan - type: nauc_recall_at_100_diff1 value: 74.89226971528723 - type: nauc_recall_at_100_max value: 68.38872094070474 - type: nauc_recall_at_100_std value: -88.3256053707361 - type: nauc_recall_at_10_diff1 value: 75.57948535605118 - type: nauc_recall_at_10_max value: 54.842401327152324 - type: nauc_recall_at_10_std value: -108.96477628690668 - type: nauc_recall_at_1_diff1 value: 82.68164323521295 - type: nauc_recall_at_1_max value: 38.954818287207324 - type: nauc_recall_at_1_std value: -56.68166218708859 - type: nauc_recall_at_20_diff1 value: 77.01723207559192 - type: nauc_recall_at_20_max value: 56.542295047119765 - type: nauc_recall_at_20_std value: -115.99929151139956 - type: nauc_recall_at_3_diff1 value: 75.75126841997931 - type: nauc_recall_at_3_max value: 46.87962275985744 - type: nauc_recall_at_3_std value: -84.35001944931008 - type: nauc_recall_at_5_diff1 value: 74.32806613815409 - type: nauc_recall_at_5_max value: 51.53090250807227 - type: nauc_recall_at_5_std value: -95.23567626837752 - type: ndcg_at_1 value: 83.22 - type: ndcg_at_10 value: 89.983 - type: ndcg_at_100 value: 91.01700000000001 - type: ndcg_at_1000 value: 91.065 - type: ndcg_at_20 value: 90.584 - type: ndcg_at_3 value: 87.351 - type: ndcg_at_5 value: 88.92 - type: precision_at_1 value: 83.22 - type: precision_at_10 value: 13.688 - type: precision_at_100 value: 1.548 - type: precision_at_1000 value: 0.157 - type: precision_at_20 value: 7.254 - type: precision_at_3 value: 38.327 - type: precision_at_5 value: 25.266 - type: recall_at_1 value: 72.333 - type: recall_at_10 value: 96.422 - type: recall_at_100 value: 99.801 - type: recall_at_1000 value: 100.0 - type: recall_at_20 value: 98.31400000000001 - type: recall_at_3 value: 88.897 - type: recall_at_5 value: 93.253 task: type: Retrieval - dataset: config: default name: MTEB RedditClustering (default) revision: 24640382cdbf8abc73003fb0fa6d111a705499eb split: test type: mteb/reddit-clustering metrics: - type: main_score value: 64.30844996368535 - type: v_measure value: 64.30844996368535 - type: v_measure_std value: 4.848791865396891 task: type: Clustering - dataset: config: default name: MTEB RedditClusteringP2P (default) revision: 385e3cb46b4cfa89021f56c4380204149d0efe33 split: test type: mteb/reddit-clustering-p2p metrics: - type: main_score value: 61.463095258322184 - type: v_measure value: 61.463095258322184 - type: v_measure_std value: 13.825524480187179 task: type: Clustering - dataset: config: default name: MTEB SCIDOCS (default) revision: f8c2fcf00f625baaa80f62ec5bd9e1fff3b8ae88 split: test type: mteb/scidocs metrics: - type: main_score value: 22.506 - type: map_at_1 value: 5.122999999999999 - type: map_at_10 value: 13.652000000000001 - type: map_at_100 value: 16.012 - type: map_at_1000 value: 16.357 - type: map_at_20 value: 14.832 - type: map_at_3 value: 9.458 - type: map_at_5 value: 11.526 - type: mrr_at_1 value: 25.3 - type: mrr_at_10 value: 37.14178571428571 - type: mrr_at_100 value: 38.2928507804565 - type: mrr_at_1000 value: 38.33371299085187 - type: mrr_at_20 value: 37.87531826532599 - type: mrr_at_3 value: 34.016666666666644 - type: mrr_at_5 value: 35.72666666666662 - type: nauc_map_at_1000_diff1 value: 19.181908239372856 - type: nauc_map_at_1000_max value: 34.45175558983018 - type: nauc_map_at_1000_std value: 18.189319929391996 - type: nauc_map_at_100_diff1 value: 19.19692982484398 - type: nauc_map_at_100_max value: 34.44154432269017 - type: nauc_map_at_100_std value: 17.94122358029364 - type: nauc_map_at_10_diff1 value: 19.291919391054222 - type: nauc_map_at_10_max value: 34.36232526749502 - type: nauc_map_at_10_std value: 14.816004018633237 - type: nauc_map_at_1_diff1 value: 24.558617169487032 - type: nauc_map_at_1_max value: 26.511296106317868 - type: nauc_map_at_1_std value: 7.82772589755214 - type: nauc_map_at_20_diff1 value: 19.348081062276208 - type: nauc_map_at_20_max value: 34.58455151721281 - type: nauc_map_at_20_std value: 16.151710432004197 - type: nauc_map_at_3_diff1 value: 21.5514840413981 - type: nauc_map_at_3_max value: 33.92222109817227 - type: nauc_map_at_3_std value: 10.502576651566999 - type: nauc_map_at_5_diff1 value: 19.967350394527863 - type: nauc_map_at_5_max value: 32.95165629885447 - type: nauc_map_at_5_std value: 11.122330247985717 - type: nauc_mrr_at_1000_diff1 value: 21.829308820999017 - type: nauc_mrr_at_1000_max value: 29.463800052705437 - type: nauc_mrr_at_1000_std value: 13.12696263120055 - type: nauc_mrr_at_100_diff1 value: 21.814298979701803 - type: nauc_mrr_at_100_max value: 29.46773136681717 - type: nauc_mrr_at_100_std value: 13.165157602965758 - type: nauc_mrr_at_10_diff1 value: 21.653878744765784 - type: nauc_mrr_at_10_max value: 29.58483868113361 - type: nauc_mrr_at_10_std value: 12.985515874358859 - type: nauc_mrr_at_1_diff1 value: 24.6715181065629 - type: nauc_mrr_at_1_max value: 26.81252204146797 - type: nauc_mrr_at_1_std value: 8.014680166410344 - type: nauc_mrr_at_20_diff1 value: 21.843640497550087 - type: nauc_mrr_at_20_max value: 29.593176268547204 - type: nauc_mrr_at_20_std value: 13.177395949373 - type: nauc_mrr_at_3_diff1 value: 22.29904422694039 - type: nauc_mrr_at_3_max value: 29.803363880621564 - type: nauc_mrr_at_3_std value: 12.08505345679438 - type: nauc_mrr_at_5_diff1 value: 21.71612843315846 - type: nauc_mrr_at_5_max value: 29.72242599153432 - type: nauc_mrr_at_5_std value: 12.514520503901528 - type: nauc_ndcg_at_1000_diff1 value: 17.812454067736155 - type: nauc_ndcg_at_1000_max value: 32.536786674780096 - type: nauc_ndcg_at_1000_std value: 25.854527875299326 - type: nauc_ndcg_at_100_diff1 value: 17.890769383828005 - type: nauc_ndcg_at_100_max value: 32.83258185170293 - type: nauc_ndcg_at_100_std value: 24.659819111590707 - type: nauc_ndcg_at_10_diff1 value: 18.26938087394905 - type: nauc_ndcg_at_10_max value: 33.78188684219704 - type: nauc_ndcg_at_10_std value: 17.21331892353095 - type: nauc_ndcg_at_1_diff1 value: 24.6715181065629 - type: nauc_ndcg_at_1_max value: 26.81252204146797 - type: nauc_ndcg_at_1_std value: 8.014680166410344 - type: nauc_ndcg_at_20_diff1 value: 18.9423678041254 - type: nauc_ndcg_at_20_max value: 34.22627638045056 - type: nauc_ndcg_at_20_std value: 19.457855372495466 - type: nauc_ndcg_at_3_diff1 value: 21.027029543031645 - type: nauc_ndcg_at_3_max value: 33.7897019283245 - type: nauc_ndcg_at_3_std value: 12.483167034609401 - type: nauc_ndcg_at_5_diff1 value: 18.892039685363475 - type: nauc_ndcg_at_5_max value: 32.45122363892051 - type: nauc_ndcg_at_5_std value: 13.41357877918938 - type: nauc_precision_at_1000_diff1 value: 5.157601529395117 - type: nauc_precision_at_1000_max value: 17.143073555203763 - type: nauc_precision_at_1000_std value: 42.98106178602979 - type: nauc_precision_at_100_diff1 value: 9.794992975326593 - type: nauc_precision_at_100_max value: 23.833302546894338 - type: nauc_precision_at_100_std value: 34.912511583237965 - type: nauc_precision_at_10_diff1 value: 13.798035613654628 - type: nauc_precision_at_10_max value: 33.262730178644546 - type: nauc_precision_at_10_std value: 21.062476706391156 - type: nauc_precision_at_1_diff1 value: 24.6715181065629 - type: nauc_precision_at_1_max value: 26.81252204146797 - type: nauc_precision_at_1_std value: 8.014680166410344 - type: nauc_precision_at_20_diff1 value: 14.663054363753272 - type: nauc_precision_at_20_max value: 32.12409736351085 - type: nauc_precision_at_20_std value: 24.341721044475626 - type: nauc_precision_at_3_diff1 value: 19.061499046645455 - type: nauc_precision_at_3_max value: 36.69387198980568 - type: nauc_precision_at_3_std value: 14.527611053399674 - type: nauc_precision_at_5_diff1 value: 15.122401114437281 - type: nauc_precision_at_5_max value: 32.37345191163627 - type: nauc_precision_at_5_std value: 15.159496915998183 - type: nauc_recall_at_1000_diff1 value: 4.90333908560398 - type: nauc_recall_at_1000_max value: 16.80152272446881 - type: nauc_recall_at_1000_std value: 44.34743343052233 - type: nauc_recall_at_100_diff1 value: 9.162169138584812 - type: nauc_recall_at_100_max value: 23.373784049981932 - type: nauc_recall_at_100_std value: 35.29658840515363 - type: nauc_recall_at_10_diff1 value: 13.255119541826458 - type: nauc_recall_at_10_max value: 32.79473451751747 - type: nauc_recall_at_10_std value: 21.126286808362067 - type: nauc_recall_at_1_diff1 value: 24.558617169487032 - type: nauc_recall_at_1_max value: 26.511296106317868 - type: nauc_recall_at_1_std value: 7.82772589755214 - type: nauc_recall_at_20_diff1 value: 14.336119216887313 - type: nauc_recall_at_20_max value: 31.763323157259993 - type: nauc_recall_at_20_std value: 24.41182092352001 - type: nauc_recall_at_3_diff1 value: 18.633092732140433 - type: nauc_recall_at_3_max value: 36.18581689585463 - type: nauc_recall_at_3_std value: 14.33948600334347 - type: nauc_recall_at_5_diff1 value: 14.679028389399267 - type: nauc_recall_at_5_max value: 31.897317036615412 - type: nauc_recall_at_5_std value: 15.037553354455593 - type: ndcg_at_1 value: 25.3 - type: ndcg_at_10 value: 22.506 - type: ndcg_at_100 value: 31.361 - type: ndcg_at_1000 value: 36.862 - type: ndcg_at_20 value: 25.717000000000002 - type: ndcg_at_3 value: 21.199 - type: ndcg_at_5 value: 18.562 - type: precision_at_1 value: 25.3 - type: precision_at_10 value: 11.81 - type: precision_at_100 value: 2.455 - type: precision_at_1000 value: 0.376 - type: precision_at_20 value: 7.775 - type: precision_at_3 value: 20.033 - type: precision_at_5 value: 16.46 - type: recall_at_1 value: 5.122999999999999 - type: recall_at_10 value: 23.953 - type: recall_at_100 value: 49.805 - type: recall_at_1000 value: 76.423 - type: recall_at_20 value: 31.493 - type: recall_at_3 value: 12.178 - type: recall_at_5 value: 16.682 task: type: Retrieval - dataset: config: default name: MTEB SICK-R (default) revision: 20a6d6f312dd54037fe07a32d58e5e168867909d split: test type: mteb/sickr-sts metrics: - type: cosine_pearson value: 86.24041625333884 - type: cosine_spearman value: 83.72893029404888 - type: euclidean_pearson value: 83.33176396347261 - type: euclidean_spearman value: 83.72893162160777 - type: main_score value: 83.72893029404888 - type: manhattan_pearson value: 83.2951639248276 - type: manhattan_spearman value: 83.70786795927772 - type: pearson value: 86.24041617887043 - type: spearman value: 83.72891772746166 task: type: STS - dataset: config: default name: MTEB STS12 (default) revision: a0d554a64d88156834ff5ae9920b964011b16384 split: test type: mteb/sts12-sts metrics: - type: cosine_pearson value: 87.17185113605036 - type: cosine_spearman value: 81.39213356797624 - type: euclidean_pearson value: 83.70587654694744 - type: euclidean_spearman value: 81.39213356797624 - type: main_score value: 81.39213356797624 - type: manhattan_pearson value: 83.63386349627461 - type: manhattan_spearman value: 81.35222067791558 - type: pearson value: 87.17185260321112 - type: spearman value: 81.38945351411505 task: type: STS - dataset: config: default name: MTEB STS13 (default) revision: 7e90230a92c190f1bf69ae9002b8cea547a64cca split: test type: mteb/sts13-sts metrics: - type: cosine_pearson value: 87.34454475793186 - type: cosine_spearman value: 87.96270687409215 - type: euclidean_pearson value: 87.90388791262633 - type: euclidean_spearman value: 87.96270687409215 - type: main_score value: 87.96270687409215 - type: manhattan_pearson value: 87.83677697801643 - type: manhattan_spearman value: 87.86991808368111 - type: pearson value: 87.34454465314778 - type: spearman value: 87.96270679590305 task: type: STS - dataset: config: default name: MTEB STS14 (default) revision: 6031580fec1f6af667f0bd2da0a551cf4f0b2375 split: test type: mteb/sts14-sts metrics: - type: cosine_pearson value: 86.3049595329839 - type: cosine_spearman value: 85.57773596099139 - type: euclidean_pearson value: 85.61735029771381 - type: euclidean_spearman value: 85.57774644488029 - type: main_score value: 85.57773596099139 - type: manhattan_pearson value: 85.58315505256886 - type: manhattan_spearman value: 85.55100867169023 - type: pearson value: 86.30495993546997 - type: spearman value: 85.57781195336361 task: type: STS - dataset: config: default name: MTEB STS15 (default) revision: ae752c7c21bf194d8b67fd573edf7ae58183cbe3 split: test type: mteb/sts15-sts metrics: - type: cosine_pearson value: 89.43687886823619 - type: cosine_spearman value: 89.77514474209266 - type: euclidean_pearson value: 89.16048792386724 - type: euclidean_spearman value: 89.77514474209266 - type: main_score value: 89.77514474209266 - type: manhattan_pearson value: 89.13664728081469 - type: manhattan_spearman value: 89.75080436431723 - type: pearson value: 89.43687930700762 - type: spearman value: 89.7750799990083 task: type: STS - dataset: config: default name: MTEB STS16 (default) revision: 4d8694f8f0e0100860b497b999b3dbed754a0513 split: test type: mteb/sts16-sts metrics: - type: cosine_pearson value: 86.6450386341272 - type: cosine_spearman value: 87.89409021728935 - type: euclidean_pearson value: 87.49933167247268 - type: euclidean_spearman value: 87.89409021728935 - type: main_score value: 87.89409021728935 - type: manhattan_pearson value: 87.50687956428204 - type: manhattan_spearman value: 87.9178498829234 - type: pearson value: 86.64503867578216 - type: spearman value: 87.8940895850418 task: type: STS - dataset: config: nl-en name: MTEB STS17 (nl-en) revision: faeb762787bd10488a50c8b5be4a3b82e411949c split: test type: mteb/sts17-crosslingual-sts metrics: - type: cosine_pearson value: 89.6338656277505 - type: cosine_spearman value: 89.75701751196556 - type: euclidean_pearson value: 90.00670260496013 - type: euclidean_spearman value: 89.75701751196556 - type: main_score value: 89.75701751196556 - type: manhattan_pearson value: 90.02629735900686 - type: manhattan_spearman value: 89.7213070723708 - type: pearson value: 89.63386499936783 - type: spearman value: 89.75701751196556 task: type: STS - dataset: config: en-tr name: MTEB STS17 (en-tr) revision: faeb762787bd10488a50c8b5be4a3b82e411949c split: test type: mteb/sts17-crosslingual-sts metrics: - type: cosine_pearson value: 83.93143564932134 - type: cosine_spearman value: 82.93577775011765 - type: euclidean_pearson value: 84.34621382409651 - type: euclidean_spearman value: 82.93577775011765 - type: main_score value: 82.93577775011765 - type: manhattan_pearson value: 84.31655977000447 - type: manhattan_spearman value: 82.76584164596149 - type: pearson value: 83.9314336093673 - type: spearman value: 82.93577775011765 task: type: STS - dataset: config: en-en name: MTEB STS17 (en-en) revision: faeb762787bd10488a50c8b5be4a3b82e411949c split: test type: mteb/sts17-crosslingual-sts metrics: - type: cosine_pearson value: 91.88424206333126 - type: cosine_spearman value: 92.02107475813581 - type: euclidean_pearson value: 91.9171107672214 - type: euclidean_spearman value: 92.02107475813581 - type: main_score value: 92.02107475813581 - type: manhattan_pearson value: 91.98552208613084 - type: manhattan_spearman value: 92.04844379257229 - type: pearson value: 91.88424111217977 - type: spearman value: 92.02107475813581 task: type: STS - dataset: config: en-de name: MTEB STS17 (en-de) revision: faeb762787bd10488a50c8b5be4a3b82e411949c split: test type: mteb/sts17-crosslingual-sts metrics: - type: cosine_pearson value: 90.04918981538256 - type: cosine_spearman value: 90.03247608046513 - type: euclidean_pearson value: 90.28411220954267 - type: euclidean_spearman value: 90.03247608046513 - type: main_score value: 90.03247608046513 - type: manhattan_pearson value: 90.3114074165839 - type: manhattan_spearman value: 90.09098123553856 - type: pearson value: 90.04919169584696 - type: spearman value: 90.03247608046513 task: type: STS - dataset: config: fr-en name: MTEB STS17 (fr-en) revision: faeb762787bd10488a50c8b5be4a3b82e411949c split: test type: mteb/sts17-crosslingual-sts metrics: - type: cosine_pearson value: 88.9930259888809 - type: cosine_spearman value: 88.83327416007984 - type: euclidean_pearson value: 89.39082582616547 - type: euclidean_spearman value: 88.83327416007984 - type: main_score value: 88.83327416007984 - type: manhattan_pearson value: 89.38701261060531 - type: manhattan_spearman value: 88.92998833233004 - type: pearson value: 88.99302543858553 - type: spearman value: 88.83327416007984 task: type: STS - dataset: config: es-en name: MTEB STS17 (es-en) revision: faeb762787bd10488a50c8b5be4a3b82e411949c split: test type: mteb/sts17-crosslingual-sts metrics: - type: cosine_pearson value: 89.95636875543408 - type: cosine_spearman value: 90.39244260353328 - type: euclidean_pearson value: 90.29925474076606 - type: euclidean_spearman value: 90.39244260353328 - type: main_score value: 90.39244260353328 - type: manhattan_pearson value: 90.37981122989076 - type: manhattan_spearman value: 90.41247149045391 - type: pearson value: 89.95636893306808 - type: spearman value: 90.39244260353328 task: type: STS - dataset: config: it-en name: MTEB STS17 (it-en) revision: faeb762787bd10488a50c8b5be4a3b82e411949c split: test type: mteb/sts17-crosslingual-sts metrics: - type: cosine_pearson value: 89.53145408873213 - type: cosine_spearman value: 89.64992463283636 - type: euclidean_pearson value: 89.92739726473282 - type: euclidean_spearman value: 89.64992463283636 - type: main_score value: 89.64992463283636 - type: manhattan_pearson value: 89.88973812881389 - type: manhattan_spearman value: 89.66533893453442 - type: pearson value: 89.53145070613068 - type: spearman value: 89.64992463283636 task: type: STS - dataset: config: en-ar name: MTEB STS17 (en-ar) revision: faeb762787bd10488a50c8b5be4a3b82e411949c split: test type: mteb/sts17-crosslingual-sts metrics: - type: cosine_pearson value: 83.73753567473895 - type: cosine_spearman value: 83.56786342290584 - type: euclidean_pearson value: 84.2185817227647 - type: euclidean_spearman value: 83.56786342290584 - type: main_score value: 83.56786342290584 - type: manhattan_pearson value: 84.138637673995 - type: manhattan_spearman value: 83.5447994878456 - type: pearson value: 83.73753620404003 - type: spearman value: 83.56786342290584 task: type: STS - dataset: config: en name: MTEB STS22 (en) revision: de9d86b3b84231dc21f76c7b7af1f28e2f57f6e3 split: test type: mteb/sts22-crosslingual-sts metrics: - type: cosine_pearson value: 70.0174654163193 - type: cosine_spearman value: 69.00945960078879 - type: euclidean_pearson value: 70.00006875963157 - type: euclidean_spearman value: 69.00945960078879 - type: main_score value: 69.00945960078879 - type: manhattan_pearson value: 70.003828333656 - type: manhattan_spearman value: 69.18289416785358 - type: pearson value: 70.01746869245112 - type: spearman value: 69.00945960078879 task: type: STS - dataset: config: pl-en name: MTEB STS22 (pl-en) revision: de9d86b3b84231dc21f76c7b7af1f28e2f57f6e3 split: test type: mteb/sts22-crosslingual-sts metrics: - type: cosine_pearson value: 74.4773241351154 - type: cosine_spearman value: 72.23771584903263 - type: euclidean_pearson value: 74.91922500307354 - type: euclidean_spearman value: 72.23771584903263 - type: main_score value: 72.23771584903263 - type: manhattan_pearson value: 75.40992669459347 - type: manhattan_spearman value: 72.89930017966125 - type: pearson value: 74.47733091997848 - type: spearman value: 72.23771584903263 task: type: STS - dataset: config: zh-en name: MTEB STS22 (zh-en) revision: de9d86b3b84231dc21f76c7b7af1f28e2f57f6e3 split: test type: mteb/sts22-crosslingual-sts metrics: - type: cosine_pearson value: 74.66163345044995 - type: cosine_spearman value: 73.412615202234 - type: euclidean_pearson value: 76.51572664173365 - type: euclidean_spearman value: 73.412615202234 - type: main_score value: 73.412615202234 - type: manhattan_pearson value: 76.44349976731687 - type: manhattan_spearman value: 73.40243152214946 - type: pearson value: 74.66163870997642 - type: spearman value: 73.412615202234 task: type: STS - dataset: config: de-en name: MTEB STS22 (de-en) revision: de9d86b3b84231dc21f76c7b7af1f28e2f57f6e3 split: test type: mteb/sts22-crosslingual-sts metrics: - type: cosine_pearson value: 65.14758907269797 - type: cosine_spearman value: 58.30708936630836 - type: euclidean_pearson value: 67.98705996212436 - type: euclidean_spearman value: 58.30708936630836 - type: main_score value: 58.30708936630836 - type: manhattan_pearson value: 68.41525035556984 - type: manhattan_spearman value: 58.879912875433405 - type: pearson value: 65.1475973244717 - type: spearman value: 58.30708936630836 task: type: STS - dataset: config: es-en name: MTEB STS22 (es-en) revision: de9d86b3b84231dc21f76c7b7af1f28e2f57f6e3 split: test type: mteb/sts22-crosslingual-sts metrics: - type: cosine_pearson value: 74.73278263018503 - type: cosine_spearman value: 77.18831783868316 - type: euclidean_pearson value: 76.28171718825621 - type: euclidean_spearman value: 77.18831783868316 - type: main_score value: 77.18831783868316 - type: manhattan_pearson value: 76.73656610143712 - type: manhattan_spearman value: 77.45086643213952 - type: pearson value: 74.73278783040479 - type: spearman value: 77.18831783868316 task: type: STS - dataset: config: default name: MTEB STSBenchmark (default) revision: b0fddb56ed78048fa8b90373c8a3cfc37b684831 split: test type: mteb/stsbenchmark-sts metrics: - type: cosine_pearson value: 88.25299312156041 - type: cosine_spearman value: 88.82703987306 - type: euclidean_pearson value: 88.42751133294018 - type: euclidean_spearman value: 88.82706405302517 - type: main_score value: 88.82703987306 - type: manhattan_pearson value: 88.41336953833218 - type: manhattan_spearman value: 88.81246784315815 - type: pearson value: 88.25299276543255 - type: spearman value: 88.82706405302517 task: type: STS - dataset: config: default name: MTEB SciDocsRR (default) revision: d3c5e1fc0b855ab6097bf1cda04dd73947d7caab split: test type: mteb/scidocs-reranking metrics: - type: main_score value: 86.15363437294592 - type: map value: 86.15363437294592 - type: mrr value: 96.44521056285762 - type: nAUC_map_diff1 value: -0.7840810005333841 - type: nAUC_map_max value: 54.6142751932947 - type: nAUC_map_std value: 70.13929703223857 - type: nAUC_mrr_diff1 value: 44.20312446414372 - type: nAUC_mrr_max value: 89.20896097421188 - type: nAUC_mrr_std value: 85.57161978228 task: type: Reranking - dataset: config: default name: MTEB SciFact (default) revision: 0228b52cf27578f30900b9e5271d331663a030d7 split: test type: mteb/scifact metrics: - type: main_score value: 77.962 - type: map_at_1 value: 60.260999999999996 - type: map_at_10 value: 72.719 - type: map_at_100 value: 73.071 - type: map_at_1000 value: 73.08 - type: map_at_20 value: 72.99 - type: map_at_3 value: 69.848 - type: map_at_5 value: 71.61999999999999 - type: mrr_at_1 value: 63.33333333333333 - type: mrr_at_10 value: 73.84933862433863 - type: mrr_at_100 value: 74.10371086565036 - type: mrr_at_1000 value: 74.1131185688027 - type: mrr_at_20 value: 74.05291111394054 - type: mrr_at_3 value: 71.88888888888891 - type: mrr_at_5 value: 73.0388888888889 - type: nauc_map_at_1000_diff1 value: 73.28126828990776 - type: nauc_map_at_1000_max value: 56.022306476443916 - type: nauc_map_at_1000_std value: -13.480460345909812 - type: nauc_map_at_100_diff1 value: 73.27839985062069 - type: nauc_map_at_100_max value: 56.03493565179317 - type: nauc_map_at_100_std value: -13.454356894686633 - type: nauc_map_at_10_diff1 value: 73.12140372590137 - type: nauc_map_at_10_max value: 56.152651928007735 - type: nauc_map_at_10_std value: -13.995855741014923 - type: nauc_map_at_1_diff1 value: 77.99227635266456 - type: nauc_map_at_1_max value: 52.094851365294005 - type: nauc_map_at_1_std value: -19.182145033673432 - type: nauc_map_at_20_diff1 value: 73.20306405935685 - type: nauc_map_at_20_max value: 56.081550664849445 - type: nauc_map_at_20_std value: -13.584356458468374 - type: nauc_map_at_3_diff1 value: 73.19141715738098 - type: nauc_map_at_3_max value: 53.23078378298865 - type: nauc_map_at_3_std value: -16.528167718115526 - type: nauc_map_at_5_diff1 value: 73.24024366833353 - type: nauc_map_at_5_max value: 55.180209378383424 - type: nauc_map_at_5_std value: -14.183250373332857 - type: nauc_mrr_at_1000_diff1 value: 73.71148233352608 - type: nauc_mrr_at_1000_max value: 56.5362956066201 - type: nauc_mrr_at_1000_std value: -11.469817601047366 - type: nauc_mrr_at_100_diff1 value: 73.70835498956635 - type: nauc_mrr_at_100_max value: 56.54902766570207 - type: nauc_mrr_at_100_std value: -11.443909853912619 - type: nauc_mrr_at_10_diff1 value: 73.57665142725747 - type: nauc_mrr_at_10_max value: 56.74885354320269 - type: nauc_mrr_at_10_std value: -11.545364799905967 - type: nauc_mrr_at_1_diff1 value: 77.9008777001104 - type: nauc_mrr_at_1_max value: 56.02354548799079 - type: nauc_mrr_at_1_std value: -12.985757082041294 - type: nauc_mrr_at_20_diff1 value: 73.63832217061234 - type: nauc_mrr_at_20_max value: 56.548555401068235 - type: nauc_mrr_at_20_std value: -11.560701355628849 - type: nauc_mrr_at_3_diff1 value: 72.90361783677665 - type: nauc_mrr_at_3_max value: 54.67757749945108 - type: nauc_mrr_at_3_std value: -11.094286378923906 - type: nauc_mrr_at_5_diff1 value: 73.54194799038713 - type: nauc_mrr_at_5_max value: 56.396527065053235 - type: nauc_mrr_at_5_std value: -10.973412029472279 - type: nauc_ndcg_at_1000_diff1 value: 72.71349972979385 - type: nauc_ndcg_at_1000_max value: 56.72661710430519 - type: nauc_ndcg_at_1000_std value: -12.077139347298028 - type: nauc_ndcg_at_100_diff1 value: 72.6223579041901 - type: nauc_ndcg_at_100_max value: 57.0943772697301 - type: nauc_ndcg_at_100_std value: -11.290467310203653 - type: nauc_ndcg_at_10_diff1 value: 71.58603155055886 - type: nauc_ndcg_at_10_max value: 57.57370636587106 - type: nauc_ndcg_at_10_std value: -13.64915129057672 - type: nauc_ndcg_at_1_diff1 value: 77.9008777001104 - type: nauc_ndcg_at_1_max value: 56.02354548799079 - type: nauc_ndcg_at_1_std value: -12.985757082041294 - type: nauc_ndcg_at_20_diff1 value: 71.862346973291 - type: nauc_ndcg_at_20_max value: 57.14084365257247 - type: nauc_ndcg_at_20_std value: -12.577106212178743 - type: nauc_ndcg_at_3_diff1 value: 70.96798592929984 - type: nauc_ndcg_at_3_max value: 53.31683131655929 - type: nauc_ndcg_at_3_std value: -13.884044044488068 - type: nauc_ndcg_at_5_diff1 value: 71.94439154102938 - type: nauc_ndcg_at_5_max value: 55.87371904515247 - type: nauc_ndcg_at_5_std value: -13.144606786194293 - type: nauc_precision_at_1000_diff1 value: -29.909943379221655 - type: nauc_precision_at_1000_max value: 3.445405323591539 - type: nauc_precision_at_1000_std value: 51.033002414810845 - type: nauc_precision_at_100_diff1 value: -23.37133080871195 - type: nauc_precision_at_100_max value: 9.078385011181364 - type: nauc_precision_at_100_std value: 50.96275222185906 - type: nauc_precision_at_10_diff1 value: -7.8465425644179465 - type: nauc_precision_at_10_max value: 23.93400510157945 - type: nauc_precision_at_10_std value: 33.12529020508839 - type: nauc_precision_at_1_diff1 value: 77.9008777001104 - type: nauc_precision_at_1_max value: 56.02354548799079 - type: nauc_precision_at_1_std value: -12.985757082041294 - type: nauc_precision_at_20_diff1 value: -17.41288362682839 - type: nauc_precision_at_20_max value: 15.35913155626735 - type: nauc_precision_at_20_std value: 41.51207496195062 - type: nauc_precision_at_3_diff1 value: 31.762128480906544 - type: nauc_precision_at_3_max value: 37.91372130404831 - type: nauc_precision_at_3_std value: 9.88524722427369 - type: nauc_precision_at_5_diff1 value: 13.038734315330009 - type: nauc_precision_at_5_max value: 30.94110952724539 - type: nauc_precision_at_5_std value: 24.082971097384295 - type: nauc_recall_at_1000_diff1 value: .nan - type: nauc_recall_at_1000_max value: .nan - type: nauc_recall_at_1000_std value: .nan - type: nauc_recall_at_100_diff1 value: 65.36881419234354 - type: nauc_recall_at_100_max value: 85.52754435107354 - type: nauc_recall_at_100_std value: 50.009337068160995 - type: nauc_recall_at_10_diff1 value: 55.74074309501903 - type: nauc_recall_at_10_max value: 66.4080962692691 - type: nauc_recall_at_10_std value: -23.91596073628651 - type: nauc_recall_at_1_diff1 value: 77.99227635266456 - type: nauc_recall_at_1_max value: 52.094851365294005 - type: nauc_recall_at_1_std value: -19.182145033673432 - type: nauc_recall_at_20_diff1 value: 50.05955137960694 - type: nauc_recall_at_20_max value: 66.30384437239482 - type: nauc_recall_at_20_std value: -17.94959453597701 - type: nauc_recall_at_3_diff1 value: 62.840959206923856 - type: nauc_recall_at_3_max value: 46.242626211644556 - type: nauc_recall_at_3_std value: -17.827394342135214 - type: nauc_recall_at_5_diff1 value: 63.99508486474444 - type: nauc_recall_at_5_max value: 56.50230441658991 - type: nauc_recall_at_5_std value: -15.143300506173162 - type: ndcg_at_1 value: 63.333 - type: ndcg_at_10 value: 77.962 - type: ndcg_at_100 value: 79.29899999999999 - type: ndcg_at_1000 value: 79.521 - type: ndcg_at_20 value: 78.81099999999999 - type: ndcg_at_3 value: 73.387 - type: ndcg_at_5 value: 75.76899999999999 - type: precision_at_1 value: 63.333 - type: precision_at_10 value: 10.433 - type: precision_at_100 value: 1.113 - type: precision_at_1000 value: 0.11299999999999999 - type: precision_at_20 value: 5.417000000000001 - type: precision_at_3 value: 29.221999999999998 - type: precision_at_5 value: 19.2 - type: recall_at_1 value: 60.260999999999996 - type: recall_at_10 value: 92.656 - type: recall_at_100 value: 98.333 - type: recall_at_1000 value: 100.0 - type: recall_at_20 value: 95.767 - type: recall_at_3 value: 80.561 - type: recall_at_5 value: 86.483 task: type: Retrieval - dataset: config: default name: MTEB SprintDuplicateQuestions (default) revision: d66bd1f72af766a5cc4b0ca5e00c162f89e8cc46 split: test type: mteb/sprintduplicatequestions-pairclassification metrics: - type: cosine_accuracy value: 99.83564356435643 - type: cosine_accuracy_threshold value: 82.17267990112305 - type: cosine_ap value: 96.54143761927851 - type: cosine_f1 value: 91.76011701608971 - type: cosine_f1_threshold value: 79.5323133468628 - type: cosine_precision value: 89.5337773549001 - type: cosine_recall value: 94.1 - type: dot_accuracy value: 99.83564356435643 - type: dot_accuracy_threshold value: 82.17268586158752 - type: dot_ap value: 96.54143761927853 - type: dot_f1 value: 91.76011701608971 - type: dot_f1_threshold value: 79.53230142593384 - type: dot_precision value: 89.5337773549001 - type: dot_recall value: 94.1 - type: euclidean_accuracy value: 99.83564356435643 - type: euclidean_accuracy_threshold value: 59.711503982543945 - type: euclidean_ap value: 96.54143761927851 - type: euclidean_f1 value: 91.76011701608971 - type: euclidean_f1_threshold value: 63.98076415061951 - type: euclidean_precision value: 89.5337773549001 - type: euclidean_recall value: 94.1 - type: main_score value: 96.54143761927853 - type: manhattan_accuracy value: 99.83564356435643 - type: manhattan_accuracy_threshold value: 3019.1539764404297 - type: manhattan_ap value: 96.52689339314482 - type: manhattan_f1 value: 91.88138065143411 - type: manhattan_f1_threshold value: 3262.8021240234375 - type: manhattan_precision value: 89.40397350993378 - type: manhattan_recall value: 94.5 - type: max_accuracy value: 99.83564356435643 - type: max_ap value: 96.54143761927853 - type: max_f1 value: 91.88138065143411 - type: max_precision value: 89.5337773549001 - type: max_recall value: 94.5 - type: similarity_accuracy value: 99.83564356435643 - type: similarity_accuracy_threshold value: 1865.63720703125 - type: similarity_ap value: 96.54143761927853 - type: similarity_f1 value: 91.76011701608971 - type: similarity_f1_threshold value: 1805.6900024414062 - type: similarity_precision value: 89.5337773549001 - type: similarity_recall value: 94.1 task: type: PairClassification - dataset: config: default name: MTEB StackExchangeClustering (default) revision: 6cbc1f7b2bc0622f2e39d2c77fa502909748c259 split: test type: mteb/stackexchange-clustering metrics: - type: main_score value: 70.93157767772517 - type: v_measure value: 70.93157767772517 - type: v_measure_std value: 4.307686954444112 task: type: Clustering - dataset: config: default name: MTEB StackExchangeClusteringP2P (default) revision: 815ca46b2622cec33ccafc3735d572c266efdb44 split: test type: mteb/stackexchange-clustering-p2p metrics: - type: main_score value: 33.95011259900505 - type: v_measure value: 33.95011259900505 - type: v_measure_std value: 1.626841664415958 task: type: Clustering - dataset: config: default name: MTEB StackOverflowDupQuestions (default) revision: e185fbe320c72810689fc5848eb6114e1ef5ec69 split: test type: mteb/stackoverflowdupquestions-reranking metrics: - type: main_score value: 53.57599561506372 - type: map value: 53.57599561506372 - type: mrr value: 54.616410958322724 - type: nAUC_map_diff1 value: 36.11395800900299 - type: nAUC_map_max value: 13.023479833306077 - type: nAUC_map_std value: 5.631816933793585 - type: nAUC_mrr_diff1 value: 35.769021835275794 - type: nAUC_mrr_max value: 13.870729016716313 - type: nAUC_mrr_std value: 5.924960324107413 task: type: Reranking - dataset: config: default name: MTEB SummEval (default) revision: cda12ad7615edc362dbf25a00fdd61d3b1eaf93c split: test type: mteb/summeval metrics: - type: cosine_pearson value: 30.73610071443582 - type: cosine_spearman value: 31.187718139843245 - type: dot_pearson value: 30.736100295017323 - type: dot_spearman value: 31.187718139843245 - type: main_score value: 31.187718139843245 - type: pearson value: 30.736095662320906 - type: spearman value: 31.187718139843245 task: type: Summarization - dataset: config: default name: MTEB TRECCOVID (default) revision: bb9466bac8153a0349341eb1b22e06409e78ef4e split: test type: mteb/trec-covid metrics: - type: main_score value: 82.472 - type: map_at_1 value: 0.244 - type: map_at_10 value: 2.046 - type: map_at_100 value: 13.645 - type: map_at_1000 value: 33.754 - type: map_at_20 value: 3.857 - type: map_at_3 value: 0.657 - type: map_at_5 value: 1.088 - type: mrr_at_1 value: 92.0 - type: mrr_at_10 value: 96.0 - type: mrr_at_100 value: 96.0 - type: mrr_at_1000 value: 96.0 - type: mrr_at_20 value: 96.0 - type: mrr_at_3 value: 96.0 - type: mrr_at_5 value: 96.0 - type: nauc_map_at_1000_diff1 value: 7.8754742601956265 - type: nauc_map_at_1000_max value: 34.597938302994095 - type: nauc_map_at_1000_std value: 69.85267631958843 - type: nauc_map_at_100_diff1 value: 34.92265650076894 - type: nauc_map_at_100_max value: 29.634429617158474 - type: nauc_map_at_100_std value: 51.34791604889786 - type: nauc_map_at_10_diff1 value: 28.079719799537035 - type: nauc_map_at_10_max value: 7.68543580943164 - type: nauc_map_at_10_std value: 9.800164176251364 - type: nauc_map_at_1_diff1 value: 26.03047942436184 - type: nauc_map_at_1_max value: -1.8634718917346798 - type: nauc_map_at_1_std value: -2.7881471230811767 - type: nauc_map_at_20_diff1 value: 30.323518515058662 - type: nauc_map_at_20_max value: 14.96919999636421 - type: nauc_map_at_20_std value: 19.064769986247782 - type: nauc_map_at_3_diff1 value: 26.295930355719438 - type: nauc_map_at_3_max value: 2.26895373159702 - type: nauc_map_at_3_std value: 5.6154000225639855 - type: nauc_map_at_5_diff1 value: 29.914803705791954 - type: nauc_map_at_5_max value: 2.3350193956307788 - type: nauc_map_at_5_std value: 7.787294789682958 - type: nauc_mrr_at_1000_diff1 value: 37.92016806722721 - type: nauc_mrr_at_1000_max value: 8.916900093370733 - type: nauc_mrr_at_1000_std value: 22.117180205415533 - type: nauc_mrr_at_100_diff1 value: 37.92016806722721 - type: nauc_mrr_at_100_max value: 8.916900093370733 - type: nauc_mrr_at_100_std value: 22.117180205415533 - type: nauc_mrr_at_10_diff1 value: 37.92016806722721 - type: nauc_mrr_at_10_max value: 8.916900093370733 - type: nauc_mrr_at_10_std value: 22.117180205415533 - type: nauc_mrr_at_1_diff1 value: 37.920168067226776 - type: nauc_mrr_at_1_max value: 8.916900093370584 - type: nauc_mrr_at_1_std value: 22.117180205415405 - type: nauc_mrr_at_20_diff1 value: 37.92016806722721 - type: nauc_mrr_at_20_max value: 8.916900093370733 - type: nauc_mrr_at_20_std value: 22.117180205415533 - type: nauc_mrr_at_3_diff1 value: 37.92016806722721 - type: nauc_mrr_at_3_max value: 8.916900093370733 - type: nauc_mrr_at_3_std value: 22.117180205415533 - type: nauc_mrr_at_5_diff1 value: 37.92016806722721 - type: nauc_mrr_at_5_max value: 8.916900093370733 - type: nauc_mrr_at_5_std value: 22.117180205415533 - type: nauc_ndcg_at_1000_diff1 value: 8.521491123342738 - type: nauc_ndcg_at_1000_max value: 33.00473635451372 - type: nauc_ndcg_at_1000_std value: 68.23856521354314 - type: nauc_ndcg_at_100_diff1 value: 7.579300791205225 - type: nauc_ndcg_at_100_max value: 37.710370018529176 - type: nauc_ndcg_at_100_std value: 64.88287259377455 - type: nauc_ndcg_at_10_diff1 value: 16.222263093560027 - type: nauc_ndcg_at_10_max value: 24.992121096921053 - type: nauc_ndcg_at_10_std value: 38.55100841743667 - type: nauc_ndcg_at_1_diff1 value: 34.26238738738736 - type: nauc_ndcg_at_1_max value: 2.0994208494208912 - type: nauc_ndcg_at_1_std value: 34.78523166023159 - type: nauc_ndcg_at_20_diff1 value: 14.66836044758468 - type: nauc_ndcg_at_20_max value: 37.980897102959254 - type: nauc_ndcg_at_20_std value: 55.088065073193384 - type: nauc_ndcg_at_3_diff1 value: 14.260719289394878 - type: nauc_ndcg_at_3_max value: 9.987815227921645 - type: nauc_ndcg_at_3_std value: 36.89480489474395 - type: nauc_ndcg_at_5_diff1 value: 21.44965758928399 - type: nauc_ndcg_at_5_max value: 16.800339135540813 - type: nauc_ndcg_at_5_std value: 40.970864822902655 - type: nauc_precision_at_1000_diff1 value: -24.790752711798277 - type: nauc_precision_at_1000_max value: 14.744105121143447 - type: nauc_precision_at_1000_std value: 27.330538098341282 - type: nauc_precision_at_100_diff1 value: 5.207401427044022 - type: nauc_precision_at_100_max value: 33.68994458941364 - type: nauc_precision_at_100_std value: 63.67064009046427 - type: nauc_precision_at_10_diff1 value: 9.202327262712192 - type: nauc_precision_at_10_max value: 33.09033635143045 - type: nauc_precision_at_10_std value: 30.090887182841573 - type: nauc_precision_at_1_diff1 value: 37.920168067226776 - type: nauc_precision_at_1_max value: 8.916900093370584 - type: nauc_precision_at_1_std value: 22.117180205415405 - type: nauc_precision_at_20_diff1 value: 9.241778934352723 - type: nauc_precision_at_20_max value: 46.3131480478757 - type: nauc_precision_at_20_std value: 51.48742898243859 - type: nauc_precision_at_3_diff1 value: 8.528129662677928 - type: nauc_precision_at_3_max value: 28.849360755975507 - type: nauc_precision_at_3_std value: 34.75029987419905 - type: nauc_precision_at_5_diff1 value: 19.869351197305594 - type: nauc_precision_at_5_max value: 29.914134497945955 - type: nauc_precision_at_5_std value: 38.28417039860837 - type: nauc_recall_at_1000_diff1 value: 1.4864052347959933 - type: nauc_recall_at_1000_max value: 26.677496064573784 - type: nauc_recall_at_1000_std value: 57.58908006108822 - type: nauc_recall_at_100_diff1 value: 33.936895170931095 - type: nauc_recall_at_100_max value: 21.088704619988643 - type: nauc_recall_at_100_std value: 37.01573558787262 - type: nauc_recall_at_10_diff1 value: 26.69952553674343 - type: nauc_recall_at_10_max value: 5.387440771414719 - type: nauc_recall_at_10_std value: 5.475592115850053 - type: nauc_recall_at_1_diff1 value: 26.03047942436184 - type: nauc_recall_at_1_max value: -1.8634718917346798 - type: nauc_recall_at_1_std value: -2.7881471230811767 - type: nauc_recall_at_20_diff1 value: 29.20290402741169 - type: nauc_recall_at_20_max value: 10.503476589449743 - type: nauc_recall_at_20_std value: 12.909057340580492 - type: nauc_recall_at_3_diff1 value: 26.1874599109351 - type: nauc_recall_at_3_max value: 3.2330495640914725 - type: nauc_recall_at_3_std value: 3.4613897806900646 - type: nauc_recall_at_5_diff1 value: 28.414989541462955 - type: nauc_recall_at_5_max value: 0.2662078485691208 - type: nauc_recall_at_5_std value: 4.924665776620815 - type: ndcg_at_1 value: 88.0 - type: ndcg_at_10 value: 82.472 - type: ndcg_at_100 value: 66.188 - type: ndcg_at_1000 value: 60.05200000000001 - type: ndcg_at_20 value: 79.782 - type: ndcg_at_3 value: 84.939 - type: ndcg_at_5 value: 84.54700000000001 - type: precision_at_1 value: 92.0 - type: precision_at_10 value: 85.8 - type: precision_at_100 value: 68.16 - type: precision_at_1000 value: 26.496 - type: precision_at_20 value: 83.7 - type: precision_at_3 value: 88.0 - type: precision_at_5 value: 88.4 - type: recall_at_1 value: 0.244 - type: recall_at_10 value: 2.22 - type: recall_at_100 value: 16.697 - type: recall_at_1000 value: 57.033 - type: recall_at_20 value: 4.301 - type: recall_at_3 value: 0.685 - type: recall_at_5 value: 1.159 task: type: Retrieval - dataset: config: default name: MTEB Touche2020 (default) revision: a34f9a33db75fa0cbb21bb5cfc3dae8dc8bec93f split: test type: mteb/touche2020 metrics: - type: main_score value: 24.7 - type: map_at_1 value: 1.8350000000000002 - type: map_at_10 value: 9.385 - type: map_at_100 value: 15.751999999999999 - type: map_at_1000 value: 17.357 - type: map_at_20 value: 11.93 - type: map_at_3 value: 4.455 - type: map_at_5 value: 6.188 - type: mrr_at_1 value: 28.57142857142857 - type: mrr_at_10 value: 43.27340459993521 - type: mrr_at_100 value: 44.588722072878205 - type: mrr_at_1000 value: 44.588722072878205 - type: mrr_at_20 value: 44.17137463324359 - type: mrr_at_3 value: 39.455782312925166 - type: mrr_at_5 value: 41.70068027210883 - type: nauc_map_at_1000_diff1 value: 3.8298195437770537 - type: nauc_map_at_1000_max value: 13.771844732194829 - type: nauc_map_at_1000_std value: 19.29786404290547 - type: nauc_map_at_100_diff1 value: 3.8927604843499735 - type: nauc_map_at_100_max value: 12.769257441869922 - type: nauc_map_at_100_std value: 15.788237027355665 - type: nauc_map_at_10_diff1 value: 5.931490598251997 - type: nauc_map_at_10_max value: 13.680069332373563 - type: nauc_map_at_10_std value: 1.1621360655557855 - type: nauc_map_at_1_diff1 value: 2.5191045547339246 - type: nauc_map_at_1_max value: 26.262684180827723 - type: nauc_map_at_1_std value: 11.673938727339559 - type: nauc_map_at_20_diff1 value: 3.006592455468332 - type: nauc_map_at_20_max value: 11.717797097718446 - type: nauc_map_at_20_std value: 2.2724143319154315 - type: nauc_map_at_3_diff1 value: 12.831129328925664 - type: nauc_map_at_3_max value: 12.405779573598991 - type: nauc_map_at_3_std value: -2.8124187492891264 - type: nauc_map_at_5_diff1 value: 15.98021350983196 - type: nauc_map_at_5_max value: 9.056183547839753 - type: nauc_map_at_5_std value: -3.2826962878341788 - type: nauc_mrr_at_1000_diff1 value: -5.794890738524456 - type: nauc_mrr_at_1000_max value: 13.697185831632897 - type: nauc_mrr_at_1000_std value: 24.631969103480568 - type: nauc_mrr_at_100_diff1 value: -5.794890738524456 - type: nauc_mrr_at_100_max value: 13.697185831632897 - type: nauc_mrr_at_100_std value: 24.631969103480568 - type: nauc_mrr_at_10_diff1 value: -6.386203318843087 - type: nauc_mrr_at_10_max value: 13.244363733609843 - type: nauc_mrr_at_10_std value: 23.85218563666624 - type: nauc_mrr_at_1_diff1 value: -10.071571494502948 - type: nauc_mrr_at_1_max value: 17.28112978017911 - type: nauc_mrr_at_1_std value: 21.458308992920365 - type: nauc_mrr_at_20_diff1 value: -6.003367578570198 - type: nauc_mrr_at_20_max value: 13.911873192715065 - type: nauc_mrr_at_20_std value: 25.477100749222902 - type: nauc_mrr_at_3_diff1 value: -6.867484873134922 - type: nauc_mrr_at_3_max value: 15.64058174587745 - type: nauc_mrr_at_3_std value: 22.712306358508364 - type: nauc_mrr_at_5_diff1 value: -4.809966508264632 - type: nauc_mrr_at_5_max value: 12.095003702970027 - type: nauc_mrr_at_5_std value: 24.442255620074484 - type: nauc_ndcg_at_1000_diff1 value: 11.434789444093685 - type: nauc_ndcg_at_1000_max value: 19.86851237564927 - type: nauc_ndcg_at_1000_std value: 40.75483058183507 - type: nauc_ndcg_at_100_diff1 value: 8.47533044191214 - type: nauc_ndcg_at_100_max value: 13.575094411118432 - type: nauc_ndcg_at_100_std value: 35.56446142831008 - type: nauc_ndcg_at_10_diff1 value: 3.8335346471495133 - type: nauc_ndcg_at_10_max value: 13.604843458587629 - type: nauc_ndcg_at_10_std value: 14.026218004849186 - type: nauc_ndcg_at_1_diff1 value: -7.753442025343042 - type: nauc_ndcg_at_1_max value: 16.06299655203062 - type: nauc_ndcg_at_1_std value: 21.58079636889492 - type: nauc_ndcg_at_20_diff1 value: 7.440981403494449 - type: nauc_ndcg_at_20_max value: 10.960931136182648 - type: nauc_ndcg_at_20_std value: 14.132481758302665 - type: nauc_ndcg_at_3_diff1 value: 8.097829929493612 - type: nauc_ndcg_at_3_max value: 13.50824542271782 - type: nauc_ndcg_at_3_std value: 13.275247050693869 - type: nauc_ndcg_at_5_diff1 value: 11.971002032611313 - type: nauc_ndcg_at_5_max value: 7.246169276334145 - type: nauc_ndcg_at_5_std value: 13.975255468959613 - type: nauc_precision_at_1000_diff1 value: 5.616105475807897 - type: nauc_precision_at_1000_max value: 25.581074402479505 - type: nauc_precision_at_1000_std value: 28.030885522347404 - type: nauc_precision_at_100_diff1 value: 5.563452871367157 - type: nauc_precision_at_100_max value: 20.2742392314572 - type: nauc_precision_at_100_std value: 69.72201297915448 - type: nauc_precision_at_10_diff1 value: 1.3379158842989611 - type: nauc_precision_at_10_max value: 12.076929332870746 - type: nauc_precision_at_10_std value: 19.420680340269207 - type: nauc_precision_at_1_diff1 value: -10.071571494502948 - type: nauc_precision_at_1_max value: 17.28112978017911 - type: nauc_precision_at_1_std value: 21.458308992920365 - type: nauc_precision_at_20_diff1 value: 4.240200829917038 - type: nauc_precision_at_20_max value: 8.993878588160804 - type: nauc_precision_at_20_std value: 30.80491219798138 - type: nauc_precision_at_3_diff1 value: 12.886975175455992 - type: nauc_precision_at_3_max value: 11.298461410464169 - type: nauc_precision_at_3_std value: 10.518238245615933 - type: nauc_precision_at_5_diff1 value: 17.435560313660595 - type: nauc_precision_at_5_max value: 2.2155983021008256 - type: nauc_precision_at_5_std value: 11.998919133184952 - type: nauc_recall_at_1000_diff1 value: 20.776784820989995 - type: nauc_recall_at_1000_max value: 7.82142405608866 - type: nauc_recall_at_1000_std value: 61.814763636984914 - type: nauc_recall_at_100_diff1 value: 3.8928372388427777 - type: nauc_recall_at_100_max value: 0.9218533326334627 - type: nauc_recall_at_100_std value: 37.90057790091917 - type: nauc_recall_at_10_diff1 value: 1.620404946253575 - type: nauc_recall_at_10_max value: 7.657179453157968 - type: nauc_recall_at_10_std value: 1.4727974035045146 - type: nauc_recall_at_1_diff1 value: 2.5191045547339246 - type: nauc_recall_at_1_max value: 26.262684180827723 - type: nauc_recall_at_1_std value: 11.673938727339559 - type: nauc_recall_at_20_diff1 value: 1.7449368151648579 - type: nauc_recall_at_20_max value: 1.7480936001393137 - type: nauc_recall_at_20_std value: 4.712827220834413 - type: nauc_recall_at_3_diff1 value: 11.857687870032672 - type: nauc_recall_at_3_max value: 7.014395082015944 - type: nauc_recall_at_3_std value: -2.9526240248867732 - type: nauc_recall_at_5_diff1 value: 15.486663741149448 - type: nauc_recall_at_5_max value: 2.6981538842605866 - type: nauc_recall_at_5_std value: -2.279294596760028 - type: ndcg_at_1 value: 26.531 - type: ndcg_at_10 value: 24.7 - type: ndcg_at_100 value: 37.21 - type: ndcg_at_1000 value: 48.687999999999995 - type: ndcg_at_20 value: 25.365 - type: ndcg_at_3 value: 26.8 - type: ndcg_at_5 value: 24.618000000000002 - type: precision_at_1 value: 28.571 - type: precision_at_10 value: 22.041 - type: precision_at_100 value: 8.122 - type: precision_at_1000 value: 1.5879999999999999 - type: precision_at_20 value: 16.735 - type: precision_at_3 value: 27.211000000000002 - type: precision_at_5 value: 24.082 - type: recall_at_1 value: 1.8350000000000002 - type: recall_at_10 value: 16.039 - type: recall_at_100 value: 49.82 - type: recall_at_1000 value: 85.979 - type: recall_at_20 value: 24.169999999999998 - type: recall_at_3 value: 5.789 - type: recall_at_5 value: 8.725 task: type: Retrieval - dataset: config: default name: MTEB ToxicConversationsClassification (default) revision: edfaf9da55d3dd50d43143d90c1ac476895ae6de split: test type: mteb/toxic_conversations_50k metrics: - type: accuracy value: 67.314453125 - type: ap value: 13.15420979399679 - type: ap_weighted value: 13.15420979399679 - type: f1 value: 52.03706668900905 - type: f1_weighted value: 74.55554872499289 - type: main_score value: 67.314453125 task: type: Classification - dataset: config: default name: MTEB TweetSentimentExtractionClassification (default) revision: d604517c81ca91fe16a244d1248fc021f9ecee7a split: test type: mteb/tweet_sentiment_extraction metrics: - type: accuracy value: 64.03225806451613 - type: f1 value: 64.06843534121843 - type: f1_weighted value: 62.74796899202356 - type: main_score value: 64.03225806451613 task: type: Classification - dataset: config: default name: MTEB TwentyNewsgroupsClustering (default) revision: 6125ec4e24fa026cec8a478383ee943acfbd5449 split: test type: mteb/twentynewsgroups-clustering metrics: - type: main_score value: 53.5182998227727 - type: v_measure value: 53.5182998227727 - type: v_measure_std value: 1.8758215247032688 task: type: Clustering - dataset: config: default name: MTEB TwitterSemEval2015 (default) revision: 70970daeab8776df92f5ea462b6173c0b46fd2d1 split: test type: mteb/twittersemeval2015-pairclassification metrics: - type: cosine_accuracy value: 88.77630088812064 - type: cosine_accuracy_threshold value: 78.23218107223511 - type: cosine_ap value: 82.3910336126561 - type: cosine_f1 value: 74.85029940119759 - type: cosine_f1_threshold value: 74.12334680557251 - type: cosine_precision value: 70.9891150023663 - type: cosine_recall value: 79.15567282321899 - type: dot_accuracy value: 88.77630088812064 - type: dot_accuracy_threshold value: 78.23217511177063 - type: dot_ap value: 82.39102411942014 - type: dot_f1 value: 74.85029940119759 - type: dot_f1_threshold value: 74.12335872650146 - type: dot_precision value: 70.9891150023663 - type: dot_recall value: 79.15567282321899 - type: euclidean_accuracy value: 88.77630088812064 - type: euclidean_accuracy_threshold value: 65.98154306411743 - type: euclidean_ap value: 82.39103194961726 - type: euclidean_f1 value: 74.85029940119759 - type: euclidean_f1_threshold value: 71.93976640701294 - type: euclidean_precision value: 70.9891150023663 - type: euclidean_recall value: 79.15567282321899 - type: main_score value: 82.39103516928604 - type: manhattan_accuracy value: 88.72265601716636 - type: manhattan_accuracy_threshold value: 3392.3141479492188 - type: manhattan_ap value: 82.37303670339044 - type: manhattan_f1 value: 74.91452450297581 - type: manhattan_f1_threshold value: 3631.7977905273438 - type: manhattan_precision value: 72.02337472607742 - type: manhattan_recall value: 78.04749340369393 - type: max_accuracy value: 88.77630088812064 - type: max_ap value: 82.39103516928604 - type: max_f1 value: 74.91452450297581 - type: max_precision value: 72.02337472607742 - type: max_recall value: 79.15567282321899 - type: similarity_accuracy value: 88.77630088812064 - type: similarity_accuracy_threshold value: 1776.1724472045898 - type: similarity_ap value: 82.39103516928604 - type: similarity_f1 value: 74.85029940119759 - type: similarity_f1_threshold value: 1682.8865051269531 - type: similarity_precision value: 70.9891150023663 - type: similarity_recall value: 79.15567282321899 task: type: PairClassification - dataset: config: default name: MTEB TwitterURLCorpus (default) revision: 8b6510b0b1fa4e4c4f879467980e9be563ec1cdf split: test type: mteb/twitterurlcorpus-pairclassification metrics: - type: cosine_accuracy value: 89.86688399891334 - type: cosine_accuracy_threshold value: 72.39344120025635 - type: cosine_ap value: 87.60291936910795 - type: cosine_f1 value: 80.1902671057446 - type: cosine_f1_threshold value: 69.8064923286438 - type: cosine_precision value: 76.40496444010599 - type: cosine_recall value: 84.3701878657222 - type: dot_accuracy value: 89.86688399891334 - type: dot_accuracy_threshold value: 72.39343523979187 - type: dot_ap value: 87.60292256175723 - type: dot_f1 value: 80.1902671057446 - type: dot_f1_threshold value: 69.80648040771484 - type: dot_precision value: 76.40496444010599 - type: dot_recall value: 84.3701878657222 - type: euclidean_accuracy value: 89.86688399891334 - type: euclidean_accuracy_threshold value: 74.30553436279297 - type: euclidean_ap value: 87.60291559616975 - type: euclidean_f1 value: 80.1902671057446 - type: euclidean_f1_threshold value: 77.70909070968628 - type: euclidean_precision value: 76.40496444010599 - type: euclidean_recall value: 84.3701878657222 - type: main_score value: 87.60538403560003 - type: manhattan_accuracy value: 89.88822913028291 - type: manhattan_accuracy_threshold value: 3783.2366943359375 - type: manhattan_ap value: 87.60538403560003 - type: manhattan_f1 value: 80.16710642040458 - type: manhattan_f1_threshold value: 3948.430633544922 - type: manhattan_precision value: 76.4895104895105 - type: manhattan_recall value: 84.21619956883278 - type: max_accuracy value: 89.88822913028291 - type: max_ap value: 87.60538403560003 - type: max_f1 value: 80.1902671057446 - type: max_precision value: 76.4895104895105 - type: max_recall value: 84.3701878657222 - type: similarity_accuracy value: 89.86688399891334 - type: similarity_accuracy_threshold value: 1643.6103820800781 - type: similarity_ap value: 87.6029128769812 - type: similarity_f1 value: 80.1902671057446 - type: similarity_f1_threshold value: 1584.8767280578613 - type: similarity_precision value: 76.40496444010599 - type: similarity_recall value: 84.3701878657222 task: type: PairClassification --- # BinGE: TODO TODO: 2 line summary and link to paper ## Usage ```python import torch from transformers import AutoTokenizer, AutoModel, AutoConfig from peft import PeftModel if __name__ == "__main__": # Loading base Meta-Llama-3 model, along with custom code that enables bidirectional connections in decoder-only LLMs. tokenizer = AutoTokenizer.from_pretrained( "McGill-NLP/LLM2Vec-Meta-Llama-3-8B-Instruct-mntp" ) config = AutoConfig.from_pretrained( "McGill-NLP/LLM2Vec-Meta-Llama-3-8B-Instruct-mntp", trust_remote_code=True ) model = AutoModel.from_pretrained( "McGill-NLP/LLM2Vec-Meta-Llama-3-8B-Instruct-mntp", trust_remote_code=True, config=config, torch_dtype=torch.bfloat16, device_map="cuda" if torch.cuda.is_available() else "cpu", ) # Loading MNTP (Masked Next Token Prediction) model. model = PeftModel.from_pretrained( model, "McGill-NLP/LLM2Vec-Meta-Llama-3-8B-Instruct-mntp", ) model = model.merge_and_unload() # This can take several minutes on cpu # Loading BinGSE model. This loads the trained LoRA weights on top of MNTP model. Hence the final weights are -- Base model + MNTP (LoRA) + BinGSE (LoRA). model = PeftModel.from_pretrained( model, model_path ) ``` TODO: initialize wrapper, provide example to check loading happened properly - see https://huggingface.co/McGill-NLP/LLM2Vec-Meta-Llama-3-8B-Instruct-mntp-unsup-simcse