Hugging Face's logo

nickprock
/
mmarco-sentence-flare-it

Sentence Similarity
sentence-transformers
PyTorch
Transformers
Italian
xlm-roberta
feature-extraction
mteb
Eval Results
Inference Endpoints
text-embeddings-inference
Model card Files Community
nickprock commited on
Commit
4eb357a
•
1 Parent(s): daa5869

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +2962 -0
README.md CHANGED
@@ -1,4 +1,2966 @@
1
  ---
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
2
  pipeline_tag: sentence-similarity
3
  tags:
4
  - sentence-transformers
 
1
  ---
2
+ tags:
3
+ - mteb
4
+ model-index:
5
+ - name: mmarco-sentence-flare-it
6
+ results:
7
+ - task:
8
+ type: Classification
9
+ dataset:
10
+ type: mteb/amazon_counterfactual
11
+ name: MTEB AmazonCounterfactualClassification (en)
12
+ config: en
13
+ split: test
14
+ revision: e8379541af4e31359cca9fbcf4b00f2671dba205
15
+ metrics:
16
+ - type: accuracy
17
+ value: 66.28358208955223
18
+ - type: ap
19
+ value: 28.583712225399804
20
+ - type: f1
21
+ value: 59.773975520814645
22
+ - task:
23
+ type: Classification
24
+ dataset:
25
+ type: mteb/amazon_counterfactual
26
+ name: MTEB AmazonCounterfactualClassification (de)
27
+ config: de
28
+ split: test
29
+ revision: e8379541af4e31359cca9fbcf4b00f2671dba205
30
+ metrics:
31
+ - type: accuracy
32
+ value: 49.28265524625267
33
+ - type: ap
34
+ value: 70.12705711793366
35
+ - type: f1
36
+ value: 46.9152621753021
37
+ - task:
38
+ type: Classification
39
+ dataset:
40
+ type: mteb/amazon_counterfactual
41
+ name: MTEB AmazonCounterfactualClassification (en-ext)
42
+ config: en-ext
43
+ split: test
44
+ revision: e8379541af4e31359cca9fbcf4b00f2671dba205
45
+ metrics:
46
+ - type: accuracy
47
+ value: 61.13943028485757
48
+ - type: ap
49
+ value: 15.393299134540122
50
+ - type: f1
51
+ value: 50.441499676740754
52
+ - task:
53
+ type: Classification
54
+ dataset:
55
+ type: mteb/amazon_counterfactual
56
+ name: MTEB AmazonCounterfactualClassification (ja)
57
+ config: ja
58
+ split: test
59
+ revision: e8379541af4e31359cca9fbcf4b00f2671dba205
60
+ metrics:
61
+ - type: accuracy
62
+ value: 44.85010706638115
63
+ - type: ap
64
+ value: 11.24959111812915
65
+ - type: f1
66
+ value: 38.4896899038441
67
+ - task:
68
+ type: Classification
69
+ dataset:
70
+ type: mteb/amazon_polarity
71
+ name: MTEB AmazonPolarityClassification
72
+ config: default
73
+ split: test
74
+ revision: e2d317d38cd51312af73b3d32a06d1a08b442046
75
+ metrics:
76
+ - type: accuracy
77
+ value: 53.786350000000006
78
+ - type: ap
79
+ value: 52.711619488611895
80
+ - type: f1
81
+ value: 52.08639681443221
82
+ - task:
83
+ type: Classification
84
+ dataset:
85
+ type: mteb/amazon_reviews_multi
86
+ name: MTEB AmazonReviewsClassification (en)
87
+ config: en
88
+ split: test
89
+ revision: 1399c76144fd37290681b995c656ef9b2e06e26d
90
+ metrics:
91
+ - type: accuracy
92
+ value: 22.954
93
+ - type: f1
94
+ value: 20.895324325359304
95
+ - task:
96
+ type: Classification
97
+ dataset:
98
+ type: mteb/amazon_reviews_multi
99
+ name: MTEB AmazonReviewsClassification (de)
100
+ config: de
101
+ split: test
102
+ revision: 1399c76144fd37290681b995c656ef9b2e06e26d
103
+ metrics:
104
+ - type: accuracy
105
+ value: 22.016
106
+ - type: f1
107
+ value: 20.141551433471214
108
+ - task:
109
+ type: Classification
110
+ dataset:
111
+ type: mteb/amazon_reviews_multi
112
+ name: MTEB AmazonReviewsClassification (es)
113
+ config: es
114
+ split: test
115
+ revision: 1399c76144fd37290681b995c656ef9b2e06e26d
116
+ metrics:
117
+ - type: accuracy
118
+ value: 23.842000000000002
119
+ - type: f1
120
+ value: 22.360764368564414
121
+ - task:
122
+ type: Classification
123
+ dataset:
124
+ type: mteb/amazon_reviews_multi
125
+ name: MTEB AmazonReviewsClassification (fr)
126
+ config: fr
127
+ split: test
128
+ revision: 1399c76144fd37290681b995c656ef9b2e06e26d
129
+ metrics:
130
+ - type: accuracy
131
+ value: 24.534000000000002
132
+ - type: f1
133
+ value: 23.348432665500937
134
+ - task:
135
+ type: Classification
136
+ dataset:
137
+ type: mteb/amazon_reviews_multi
138
+ name: MTEB AmazonReviewsClassification (ja)
139
+ config: ja
140
+ split: test
141
+ revision: 1399c76144fd37290681b995c656ef9b2e06e26d
142
+ metrics:
143
+ - type: accuracy
144
+ value: 20.183999999999997
145
+ - type: f1
146
+ value: 17.025753479408394
147
+ - task:
148
+ type: Classification
149
+ dataset:
150
+ type: mteb/amazon_reviews_multi
151
+ name: MTEB AmazonReviewsClassification (zh)
152
+ config: zh
153
+ split: test
154
+ revision: 1399c76144fd37290681b995c656ef9b2e06e26d
155
+ metrics:
156
+ - type: accuracy
157
+ value: 20.226000000000003
158
+ - type: f1
159
+ value: 17.949454130689396
160
+ - task:
161
+ type: Retrieval
162
+ dataset:
163
+ type: arguana
164
+ name: MTEB ArguAna
165
+ config: default
166
+ split: test
167
+ revision: None
168
+ metrics:
169
+ - type: map_at_1
170
+ value: 0.64
171
+ - type: map_at_10
172
+ value: 1.109
173
+ - type: map_at_100
174
+ value: 1.214
175
+ - type: map_at_1000
176
+ value: 1.273
177
+ - type: map_at_3
178
+ value: 0.936
179
+ - type: map_at_5
180
+ value: 1.032
181
+ - type: mrr_at_1
182
+ value: 0.64
183
+ - type: mrr_at_10
184
+ value: 1.109
185
+ - type: mrr_at_100
186
+ value: 1.214
187
+ - type: mrr_at_1000
188
+ value: 1.273
189
+ - type: mrr_at_3
190
+ value: 0.936
191
+ - type: mrr_at_5
192
+ value: 1.032
193
+ - type: ndcg_at_1
194
+ value: 0.64
195
+ - type: ndcg_at_10
196
+ value: 1.401
197
+ - type: ndcg_at_100
198
+ value: 2.106
199
+ - type: ndcg_at_1000
200
+ value: 4.484
201
+ - type: ndcg_at_3
202
+ value: 1.042
203
+ - type: ndcg_at_5
204
+ value: 1.217
205
+ - type: precision_at_1
206
+ value: 0.64
207
+ - type: precision_at_10
208
+ value: 0.23500000000000001
209
+ - type: precision_at_100
210
+ value: 0.061
211
+ - type: precision_at_1000
212
+ value: 0.027
213
+ - type: precision_at_3
214
+ value: 0.44999999999999996
215
+ - type: precision_at_5
216
+ value: 0.356
217
+ - type: recall_at_1
218
+ value: 0.64
219
+ - type: recall_at_10
220
+ value: 2.347
221
+ - type: recall_at_100
222
+ value: 6.117
223
+ - type: recall_at_1000
224
+ value: 26.671
225
+ - type: recall_at_3
226
+ value: 1.351
227
+ - type: recall_at_5
228
+ value: 1.778
229
+ - task:
230
+ type: Clustering
231
+ dataset:
232
+ type: mteb/arxiv-clustering-p2p
233
+ name: MTEB ArxivClusteringP2P
234
+ config: default
235
+ split: test
236
+ revision: a122ad7f3f0291bf49cc6f4d32aa80929df69d5d
237
+ metrics:
238
+ - type: v_measure
239
+ value: 10.337297492580117
240
+ - task:
241
+ type: Clustering
242
+ dataset:
243
+ type: mteb/arxiv-clustering-s2s
244
+ name: MTEB ArxivClusteringS2S
245
+ config: default
246
+ split: test
247
+ revision: f910caf1a6075f7329cdf8c1a6135696f37dbd53
248
+ metrics:
249
+ - type: v_measure
250
+ value: 8.41067718068448
251
+ - task:
252
+ type: Reranking
253
+ dataset:
254
+ type: mteb/askubuntudupquestions-reranking
255
+ name: MTEB AskUbuntuDupQuestions
256
+ config: default
257
+ split: test
258
+ revision: 2000358ca161889fa9c082cb41daa8dcfb161a54
259
+ metrics:
260
+ - type: map
261
+ value: 39.45372039138711
262
+ - type: mrr
263
+ value: 49.48005979861936
264
+ - task:
265
+ type: STS
266
+ dataset:
267
+ type: mteb/biosses-sts
268
+ name: MTEB BIOSSES
269
+ config: default
270
+ split: test
271
+ revision: d3fb88f8f02e40887cd149695127462bbcf29b4a
272
+ metrics:
273
+ - type: cos_sim_pearson
274
+ value: 2.1060056434908527
275
+ - type: cos_sim_spearman
276
+ value: -6.396531291473412
277
+ - type: euclidean_pearson
278
+ value: -1.0319749731423296
279
+ - type: euclidean_spearman
280
+ value: -5.283855335987313
281
+ - type: manhattan_pearson
282
+ value: -5.66609061890471
283
+ - type: manhattan_spearman
284
+ value: -7.173055009189482
285
+ - task:
286
+ type: Classification
287
+ dataset:
288
+ type: mteb/banking77
289
+ name: MTEB Banking77Classification
290
+ config: default
291
+ split: test
292
+ revision: 0fd18e25b25c072e09e0d92ab615fda904d66300
293
+ metrics:
294
+ - type: accuracy
295
+ value: 17.435064935064933
296
+ - type: f1
297
+ value: 15.631665237965379
298
+ - task:
299
+ type: Clustering
300
+ dataset:
301
+ type: mteb/biorxiv-clustering-p2p
302
+ name: MTEB BiorxivClusteringP2P
303
+ config: default
304
+ split: test
305
+ revision: 65b79d1d13f80053f67aca9498d9402c2d9f1f40
306
+ metrics:
307
+ - type: v_measure
308
+ value: 4.01285243931824
309
+ - task:
310
+ type: Clustering
311
+ dataset:
312
+ type: mteb/biorxiv-clustering-s2s
313
+ name: MTEB BiorxivClusteringS2S
314
+ config: default
315
+ split: test
316
+ revision: 258694dd0231531bc1fd9de6ceb52a0853c6d908
317
+ metrics:
318
+ - type: v_measure
319
+ value: 3.0046123718115685
320
+ - task:
321
+ type: Retrieval
322
+ dataset:
323
+ type: BeIR/cqadupstack
324
+ name: MTEB CQADupstackAndroidRetrieval
325
+ config: default
326
+ split: test
327
+ revision: None
328
+ metrics:
329
+ - type: map_at_1
330
+ value: 0.001
331
+ - type: map_at_10
332
+ value: 0.023
333
+ - type: map_at_100
334
+ value: 0.034999999999999996
335
+ - type: map_at_1000
336
+ value: 0.042
337
+ - type: map_at_3
338
+ value: 0.001
339
+ - type: map_at_5
340
+ value: 0.001
341
+ - type: mrr_at_1
342
+ value: 0.14300000000000002
343
+ - type: mrr_at_10
344
+ value: 0.173
345
+ - type: mrr_at_100
346
+ value: 0.203
347
+ - type: mrr_at_1000
348
+ value: 0.216
349
+ - type: mrr_at_3
350
+ value: 0.14300000000000002
351
+ - type: mrr_at_5
352
+ value: 0.14300000000000002
353
+ - type: ndcg_at_1
354
+ value: 0.14300000000000002
355
+ - type: ndcg_at_10
356
+ value: 0.11
357
+ - type: ndcg_at_100
358
+ value: 0.174
359
+ - type: ndcg_at_1000
360
+ value: 0.526
361
+ - type: ndcg_at_3
362
+ value: 0.067
363
+ - type: ndcg_at_5
364
+ value: 0.049
365
+ - type: precision_at_1
366
+ value: 0.14300000000000002
367
+ - type: precision_at_10
368
+ value: 0.056999999999999995
369
+ - type: precision_at_100
370
+ value: 0.02
371
+ - type: precision_at_1000
372
+ value: 0.01
373
+ - type: precision_at_3
374
+ value: 0.048
375
+ - type: precision_at_5
376
+ value: 0.029
377
+ - type: recall_at_1
378
+ value: 0.001
379
+ - type: recall_at_10
380
+ value: 0.216
381
+ - type: recall_at_100
382
+ value: 0.629
383
+ - type: recall_at_1000
384
+ value: 3.1940000000000004
385
+ - type: recall_at_3
386
+ value: 0.001
387
+ - type: recall_at_5
388
+ value: 0.001
389
+ - task:
390
+ type: Retrieval
391
+ dataset:
392
+ type: BeIR/cqadupstack
393
+ name: MTEB CQADupstackEnglishRetrieval
394
+ config: default
395
+ split: test
396
+ revision: None
397
+ metrics:
398
+ - type: map_at_1
399
+ value: 0.0
400
+ - type: map_at_10
401
+ value: 0.04
402
+ - type: map_at_100
403
+ value: 0.058
404
+ - type: map_at_1000
405
+ value: 0.065
406
+ - type: map_at_3
407
+ value: 0.033
408
+ - type: map_at_5
409
+ value: 0.04
410
+ - type: mrr_at_1
411
+ value: 0.0
412
+ - type: mrr_at_10
413
+ value: 0.066
414
+ - type: mrr_at_100
415
+ value: 0.099
416
+ - type: mrr_at_1000
417
+ value: 0.11
418
+ - type: mrr_at_3
419
+ value: 0.053
420
+ - type: mrr_at_5
421
+ value: 0.066
422
+ - type: ndcg_at_1
423
+ value: 0.0
424
+ - type: ndcg_at_10
425
+ value: 0.062
426
+ - type: ndcg_at_100
427
+ value: 0.182
428
+ - type: ndcg_at_1000
429
+ value: 0.494
430
+ - type: ndcg_at_3
431
+ value: 0.055
432
+ - type: ndcg_at_5
433
+ value: 0.066
434
+ - type: precision_at_1
435
+ value: 0.0
436
+ - type: precision_at_10
437
+ value: 0.019
438
+ - type: precision_at_100
439
+ value: 0.012
440
+ - type: precision_at_1000
441
+ value: 0.006
442
+ - type: precision_at_3
443
+ value: 0.042
444
+ - type: precision_at_5
445
+ value: 0.038
446
+ - type: recall_at_1
447
+ value: 0.0
448
+ - type: recall_at_10
449
+ value: 0.1
450
+ - type: recall_at_100
451
+ value: 0.626
452
+ - type: recall_at_1000
453
+ value: 3.012
454
+ - type: recall_at_3
455
+ value: 0.068
456
+ - type: recall_at_5
457
+ value: 0.1
458
+ - task:
459
+ type: Retrieval
460
+ dataset:
461
+ type: BeIR/cqadupstack
462
+ name: MTEB CQADupstackGamingRetrieval
463
+ config: default
464
+ split: test
465
+ revision: None
466
+ metrics:
467
+ - type: map_at_1
468
+ value: 0.0
469
+ - type: map_at_10
470
+ value: 0.01
471
+ - type: map_at_100
472
+ value: 0.019
473
+ - type: map_at_1000
474
+ value: 0.026
475
+ - type: map_at_3
476
+ value: 0.0
477
+ - type: map_at_5
478
+ value: 0.0
479
+ - type: mrr_at_1
480
+ value: 0.0
481
+ - type: mrr_at_10
482
+ value: 0.01
483
+ - type: mrr_at_100
484
+ value: 0.019
485
+ - type: mrr_at_1000
486
+ value: 0.027
487
+ - type: mrr_at_3
488
+ value: 0.0
489
+ - type: mrr_at_5
490
+ value: 0.0
491
+ - type: ndcg_at_1
492
+ value: 0.0
493
+ - type: ndcg_at_10
494
+ value: 0.022000000000000002
495
+ - type: ndcg_at_100
496
+ value: 0.09
497
+ - type: ndcg_at_1000
498
+ value: 0.35500000000000004
499
+ - type: ndcg_at_3
500
+ value: 0.0
501
+ - type: ndcg_at_5
502
+ value: 0.0
503
+ - type: precision_at_1
504
+ value: 0.0
505
+ - type: precision_at_10
506
+ value: 0.006
507
+ - type: precision_at_100
508
+ value: 0.004
509
+ - type: precision_at_1000
510
+ value: 0.003
511
+ - type: precision_at_3
512
+ value: 0.0
513
+ - type: precision_at_5
514
+ value: 0.0
515
+ - type: recall_at_1
516
+ value: 0.0
517
+ - type: recall_at_10
518
+ value: 0.063
519
+ - type: recall_at_100
520
+ value: 0.439
521
+ - type: recall_at_1000
522
+ value: 2.576
523
+ - type: recall_at_3
524
+ value: 0.0
525
+ - type: recall_at_5
526
+ value: 0.0
527
+ - task:
528
+ type: Retrieval
529
+ dataset:
530
+ type: BeIR/cqadupstack
531
+ name: MTEB CQADupstackGisRetrieval
532
+ config: default
533
+ split: test
534
+ revision: None
535
+ metrics:
536
+ - type: map_at_1
537
+ value: 0.22599999999999998
538
+ - type: map_at_10
539
+ value: 0.22599999999999998
540
+ - type: map_at_100
541
+ value: 0.23500000000000001
542
+ - type: map_at_1000
543
+ value: 0.241
544
+ - type: map_at_3
545
+ value: 0.22599999999999998
546
+ - type: map_at_5
547
+ value: 0.22599999999999998
548
+ - type: mrr_at_1
549
+ value: 0.22599999999999998
550
+ - type: mrr_at_10
551
+ value: 0.22599999999999998
552
+ - type: mrr_at_100
553
+ value: 0.23800000000000002
554
+ - type: mrr_at_1000
555
+ value: 0.244
556
+ - type: mrr_at_3
557
+ value: 0.22599999999999998
558
+ - type: mrr_at_5
559
+ value: 0.22599999999999998
560
+ - type: ndcg_at_1
561
+ value: 0.22599999999999998
562
+ - type: ndcg_at_10
563
+ value: 0.22599999999999998
564
+ - type: ndcg_at_100
565
+ value: 0.317
566
+ - type: ndcg_at_1000
567
+ value: 0.584
568
+ - type: ndcg_at_3
569
+ value: 0.22599999999999998
570
+ - type: ndcg_at_5
571
+ value: 0.22599999999999998
572
+ - type: precision_at_1
573
+ value: 0.22599999999999998
574
+ - type: precision_at_10
575
+ value: 0.023
576
+ - type: precision_at_100
577
+ value: 0.009000000000000001
578
+ - type: precision_at_1000
579
+ value: 0.004
580
+ - type: precision_at_3
581
+ value: 0.075
582
+ - type: precision_at_5
583
+ value: 0.045
584
+ - type: recall_at_1
585
+ value: 0.22599999999999998
586
+ - type: recall_at_10
587
+ value: 0.22599999999999998
588
+ - type: recall_at_100
589
+ value: 0.732
590
+ - type: recall_at_1000
591
+ value: 2.951
592
+ - type: recall_at_3
593
+ value: 0.22599999999999998
594
+ - type: recall_at_5
595
+ value: 0.22599999999999998
596
+ - task:
597
+ type: Retrieval
598
+ dataset:
599
+ type: BeIR/cqadupstack
600
+ name: MTEB CQADupstackMathematicaRetrieval
601
+ config: default
602
+ split: test
603
+ revision: None
604
+ metrics:
605
+ - type: map_at_1
606
+ value: 0.062
607
+ - type: map_at_10
608
+ value: 0.062
609
+ - type: map_at_100
610
+ value: 0.08800000000000001
611
+ - type: map_at_1000
612
+ value: 0.1
613
+ - type: map_at_3
614
+ value: 0.062
615
+ - type: map_at_5
616
+ value: 0.062
617
+ - type: mrr_at_1
618
+ value: 0.124
619
+ - type: mrr_at_10
620
+ value: 0.124
621
+ - type: mrr_at_100
622
+ value: 0.173
623
+ - type: mrr_at_1000
624
+ value: 0.191
625
+ - type: mrr_at_3
626
+ value: 0.124
627
+ - type: mrr_at_5
628
+ value: 0.124
629
+ - type: ndcg_at_1
630
+ value: 0.124
631
+ - type: ndcg_at_10
632
+ value: 0.076
633
+ - type: ndcg_at_100
634
+ value: 0.27
635
+ - type: ndcg_at_1000
636
+ value: 0.7849999999999999
637
+ - type: ndcg_at_3
638
+ value: 0.076
639
+ - type: ndcg_at_5
640
+ value: 0.076
641
+ - type: precision_at_1
642
+ value: 0.124
643
+ - type: precision_at_10
644
+ value: 0.012
645
+ - type: precision_at_100
646
+ value: 0.02
647
+ - type: precision_at_1000
648
+ value: 0.009000000000000001
649
+ - type: precision_at_3
650
+ value: 0.041
651
+ - type: precision_at_5
652
+ value: 0.025
653
+ - type: recall_at_1
654
+ value: 0.062
655
+ - type: recall_at_10
656
+ value: 0.062
657
+ - type: recall_at_100
658
+ value: 0.9119999999999999
659
+ - type: recall_at_1000
660
+ value: 4.809
661
+ - type: recall_at_3
662
+ value: 0.062
663
+ - type: recall_at_5
664
+ value: 0.062
665
+ - task:
666
+ type: Retrieval
667
+ dataset:
668
+ type: BeIR/cqadupstack
669
+ name: MTEB CQADupstackPhysicsRetrieval
670
+ config: default
671
+ split: test
672
+ revision: None
673
+ metrics:
674
+ - type: map_at_1
675
+ value: 0.0
676
+ - type: map_at_10
677
+ value: 0.043
678
+ - type: map_at_100
679
+ value: 0.061
680
+ - type: map_at_1000
681
+ value: 0.06999999999999999
682
+ - type: map_at_3
683
+ value: 0.0
684
+ - type: map_at_5
685
+ value: 0.043
686
+ - type: mrr_at_1
687
+ value: 0.0
688
+ - type: mrr_at_10
689
+ value: 0.043
690
+ - type: mrr_at_100
691
+ value: 0.06899999999999999
692
+ - type: mrr_at_1000
693
+ value: 0.079
694
+ - type: mrr_at_3
695
+ value: 0.0
696
+ - type: mrr_at_5
697
+ value: 0.043
698
+ - type: ndcg_at_1
699
+ value: 0.0
700
+ - type: ndcg_at_10
701
+ value: 0.079
702
+ - type: ndcg_at_100
703
+ value: 0.22599999999999998
704
+ - type: ndcg_at_1000
705
+ value: 0.5579999999999999
706
+ - type: ndcg_at_3
707
+ value: 0.0
708
+ - type: ndcg_at_5
709
+ value: 0.079
710
+ - type: precision_at_1
711
+ value: 0.0
712
+ - type: precision_at_10
713
+ value: 0.019
714
+ - type: precision_at_100
715
+ value: 0.013
716
+ - type: precision_at_1000
717
+ value: 0.005
718
+ - type: precision_at_3
719
+ value: 0.0
720
+ - type: precision_at_5
721
+ value: 0.038
722
+ - type: recall_at_1
723
+ value: 0.0
724
+ - type: recall_at_10
725
+ value: 0.192
726
+ - type: recall_at_100
727
+ value: 0.918
728
+ - type: recall_at_1000
729
+ value: 3.5909999999999997
730
+ - type: recall_at_3
731
+ value: 0.0
732
+ - type: recall_at_5
733
+ value: 0.192
734
+ - task:
735
+ type: Retrieval
736
+ dataset:
737
+ type: BeIR/cqadupstack
738
+ name: MTEB CQADupstackProgrammersRetrieval
739
+ config: default
740
+ split: test
741
+ revision: None
742
+ metrics:
743
+ - type: map_at_1
744
+ value: 0.0
745
+ - type: map_at_10
746
+ value: 0.04
747
+ - type: map_at_100
748
+ value: 0.044000000000000004
749
+ - type: map_at_1000
750
+ value: 0.052
751
+ - type: map_at_3
752
+ value: 0.038
753
+ - type: map_at_5
754
+ value: 0.038
755
+ - type: mrr_at_1
756
+ value: 0.0
757
+ - type: mrr_at_10
758
+ value: 0.054
759
+ - type: mrr_at_100
760
+ value: 0.07200000000000001
761
+ - type: mrr_at_1000
762
+ value: 0.084
763
+ - type: mrr_at_3
764
+ value: 0.038
765
+ - type: mrr_at_5
766
+ value: 0.038
767
+ - type: ndcg_at_1
768
+ value: 0.0
769
+ - type: ndcg_at_10
770
+ value: 0.067
771
+ - type: ndcg_at_100
772
+ value: 0.10300000000000001
773
+ - type: ndcg_at_1000
774
+ value: 0.488
775
+ - type: ndcg_at_3
776
+ value: 0.056999999999999995
777
+ - type: ndcg_at_5
778
+ value: 0.056999999999999995
779
+ - type: precision_at_1
780
+ value: 0.0
781
+ - type: precision_at_10
782
+ value: 0.023
783
+ - type: precision_at_100
784
+ value: 0.006999999999999999
785
+ - type: precision_at_1000
786
+ value: 0.006
787
+ - type: precision_at_3
788
+ value: 0.038
789
+ - type: precision_at_5
790
+ value: 0.023
791
+ - type: recall_at_1
792
+ value: 0.0
793
+ - type: recall_at_10
794
+ value: 0.128
795
+ - type: recall_at_100
796
+ value: 0.248
797
+ - type: recall_at_1000
798
+ value: 3.36
799
+ - type: recall_at_3
800
+ value: 0.11399999999999999
801
+ - type: recall_at_5
802
+ value: 0.11399999999999999
803
+ - task:
804
+ type: Retrieval
805
+ dataset:
806
+ type: BeIR/cqadupstack
807
+ name: MTEB CQADupstackStatsRetrieval
808
+ config: default
809
+ split: test
810
+ revision: None
811
+ metrics:
812
+ - type: map_at_1
813
+ value: 0.0
814
+ - type: map_at_10
815
+ value: 0.0
816
+ - type: map_at_100
817
+ value: 0.013999999999999999
818
+ - type: map_at_1000
819
+ value: 0.023
820
+ - type: map_at_3
821
+ value: 0.0
822
+ - type: map_at_5
823
+ value: 0.0
824
+ - type: mrr_at_1
825
+ value: 0.0
826
+ - type: mrr_at_10
827
+ value: 0.0
828
+ - type: mrr_at_100
829
+ value: 0.03
830
+ - type: mrr_at_1000
831
+ value: 0.045
832
+ - type: mrr_at_3
833
+ value: 0.0
834
+ - type: mrr_at_5
835
+ value: 0.0
836
+ - type: ndcg_at_1
837
+ value: 0.0
838
+ - type: ndcg_at_10
839
+ value: 0.0
840
+ - type: ndcg_at_100
841
+ value: 0.066
842
+ - type: ndcg_at_1000
843
+ value: 0.445
844
+ - type: ndcg_at_3
845
+ value: 0.0
846
+ - type: ndcg_at_5
847
+ value: 0.0
848
+ - type: precision_at_1
849
+ value: 0.0
850
+ - type: precision_at_10
851
+ value: 0.0
852
+ - type: precision_at_100
853
+ value: 0.005
854
+ - type: precision_at_1000
855
+ value: 0.005
856
+ - type: precision_at_3
857
+ value: 0.0
858
+ - type: precision_at_5
859
+ value: 0.0
860
+ - type: recall_at_1
861
+ value: 0.0
862
+ - type: recall_at_10
863
+ value: 0.0
864
+ - type: recall_at_100
865
+ value: 0.24
866
+ - type: recall_at_1000
867
+ value: 3.1559999999999997
868
+ - type: recall_at_3
869
+ value: 0.0
870
+ - type: recall_at_5
871
+ value: 0.0
872
+ - task:
873
+ type: Retrieval
874
+ dataset:
875
+ type: BeIR/cqadupstack
876
+ name: MTEB CQADupstackTexRetrieval
877
+ config: default
878
+ split: test
879
+ revision: None
880
+ metrics:
881
+ - type: map_at_1
882
+ value: 0.0
883
+ - type: map_at_10
884
+ value: 0.0
885
+ - type: map_at_100
886
+ value: 0.006999999999999999
887
+ - type: map_at_1000
888
+ value: 0.012
889
+ - type: map_at_3
890
+ value: 0.0
891
+ - type: map_at_5
892
+ value: 0.0
893
+ - type: mrr_at_1
894
+ value: 0.0
895
+ - type: mrr_at_10
896
+ value: 0.0
897
+ - type: mrr_at_100
898
+ value: 0.011000000000000001
899
+ - type: mrr_at_1000
900
+ value: 0.018000000000000002
901
+ - type: mrr_at_3
902
+ value: 0.0
903
+ - type: mrr_at_5
904
+ value: 0.0
905
+ - type: ndcg_at_1
906
+ value: 0.0
907
+ - type: ndcg_at_10
908
+ value: 0.0
909
+ - type: ndcg_at_100
910
+ value: 0.055
911
+ - type: ndcg_at_1000
912
+ value: 0.254
913
+ - type: ndcg_at_3
914
+ value: 0.0
915
+ - type: ndcg_at_5
916
+ value: 0.0
917
+ - type: precision_at_1
918
+ value: 0.0
919
+ - type: precision_at_10
920
+ value: 0.0
921
+ - type: precision_at_100
922
+ value: 0.004
923
+ - type: precision_at_1000
924
+ value: 0.003
925
+ - type: precision_at_3
926
+ value: 0.0
927
+ - type: precision_at_5
928
+ value: 0.0
929
+ - type: recall_at_1
930
+ value: 0.0
931
+ - type: recall_at_10
932
+ value: 0.0
933
+ - type: recall_at_100
934
+ value: 0.27599999999999997
935
+ - type: recall_at_1000
936
+ value: 1.828
937
+ - type: recall_at_3
938
+ value: 0.0
939
+ - type: recall_at_5
940
+ value: 0.0
941
+ - task:
942
+ type: Retrieval
943
+ dataset:
944
+ type: BeIR/cqadupstack
945
+ name: MTEB CQADupstackUnixRetrieval
946
+ config: default
947
+ split: test
948
+ revision: None
949
+ metrics:
950
+ - type: map_at_1
951
+ value: 0.0
952
+ - type: map_at_10
953
+ value: 0.023
954
+ - type: map_at_100
955
+ value: 0.031
956
+ - type: map_at_1000
957
+ value: 0.038
958
+ - type: map_at_3
959
+ value: 0.0
960
+ - type: map_at_5
961
+ value: 0.023
962
+ - type: mrr_at_1
963
+ value: 0.0
964
+ - type: mrr_at_10
965
+ value: 0.023
966
+ - type: mrr_at_100
967
+ value: 0.039
968
+ - type: mrr_at_1000
969
+ value: 0.048
970
+ - type: mrr_at_3
971
+ value: 0.0
972
+ - type: mrr_at_5
973
+ value: 0.023
974
+ - type: ndcg_at_1
975
+ value: 0.0
976
+ - type: ndcg_at_10
977
+ value: 0.04
978
+ - type: ndcg_at_100
979
+ value: 0.133
980
+ - type: ndcg_at_1000
981
+ value: 0.395
982
+ - type: ndcg_at_3
983
+ value: 0.0
984
+ - type: ndcg_at_5
985
+ value: 0.04
986
+ - type: precision_at_1
987
+ value: 0.0
988
+ - type: precision_at_10
989
+ value: 0.009000000000000001
990
+ - type: precision_at_100
991
+ value: 0.009000000000000001
992
+ - type: precision_at_1000
993
+ value: 0.004
994
+ - type: precision_at_3
995
+ value: 0.0
996
+ - type: precision_at_5
997
+ value: 0.019
998
+ - type: recall_at_1
999
+ value: 0.0
1000
+ - type: recall_at_10
1001
+ value: 0.093
1002
+ - type: recall_at_100
1003
+ value: 0.598
1004
+ - type: recall_at_1000
1005
+ value: 2.59
1006
+ - type: recall_at_3
1007
+ value: 0.0
1008
+ - type: recall_at_5
1009
+ value: 0.093
1010
+ - task:
1011
+ type: Retrieval
1012
+ dataset:
1013
+ type: BeIR/cqadupstack
1014
+ name: MTEB CQADupstackWebmastersRetrieval
1015
+ config: default
1016
+ split: test
1017
+ revision: None
1018
+ metrics:
1019
+ - type: map_at_1
1020
+ value: 0.0
1021
+ - type: map_at_10
1022
+ value: 0.0
1023
+ - type: map_at_100
1024
+ value: 0.015
1025
+ - type: map_at_1000
1026
+ value: 0.03
1027
+ - type: map_at_3
1028
+ value: 0.0
1029
+ - type: map_at_5
1030
+ value: 0.0
1031
+ - type: mrr_at_1
1032
+ value: 0.0
1033
+ - type: mrr_at_10
1034
+ value: 0.0
1035
+ - type: mrr_at_100
1036
+ value: 0.062
1037
+ - type: mrr_at_1000
1038
+ value: 0.083
1039
+ - type: mrr_at_3
1040
+ value: 0.0
1041
+ - type: mrr_at_5
1042
+ value: 0.0
1043
+ - type: ndcg_at_1
1044
+ value: 0.0
1045
+ - type: ndcg_at_10
1046
+ value: 0.0
1047
+ - type: ndcg_at_100
1048
+ value: 0.17700000000000002
1049
+ - type: ndcg_at_1000
1050
+ value: 0.9299999999999999
1051
+ - type: ndcg_at_3
1052
+ value: 0.0
1053
+ - type: ndcg_at_5
1054
+ value: 0.0
1055
+ - type: precision_at_1
1056
+ value: 0.0
1057
+ - type: precision_at_10
1058
+ value: 0.0
1059
+ - type: precision_at_100
1060
+ value: 0.027999999999999997
1061
+ - type: precision_at_1000
1062
+ value: 0.023
1063
+ - type: precision_at_3
1064
+ value: 0.0
1065
+ - type: precision_at_5
1066
+ value: 0.0
1067
+ - type: recall_at_1
1068
+ value: 0.0
1069
+ - type: recall_at_10
1070
+ value: 0.0
1071
+ - type: recall_at_100
1072
+ value: 0.894
1073
+ - type: recall_at_1000
1074
+ value: 6.639
1075
+ - type: recall_at_3
1076
+ value: 0.0
1077
+ - type: recall_at_5
1078
+ value: 0.0
1079
+ - task:
1080
+ type: Retrieval
1081
+ dataset:
1082
+ type: BeIR/cqadupstack
1083
+ name: MTEB CQADupstackWordpressRetrieval
1084
+ config: default
1085
+ split: test
1086
+ revision: None
1087
+ metrics:
1088
+ - type: map_at_1
1089
+ value: 0.0
1090
+ - type: map_at_10
1091
+ value: 0.0
1092
+ - type: map_at_100
1093
+ value: 0.0
1094
+ - type: map_at_1000
1095
+ value: 0.006999999999999999
1096
+ - type: map_at_3
1097
+ value: 0.0
1098
+ - type: map_at_5
1099
+ value: 0.0
1100
+ - type: mrr_at_1
1101
+ value: 0.0
1102
+ - type: mrr_at_10
1103
+ value: 0.0
1104
+ - type: mrr_at_100
1105
+ value: 0.0
1106
+ - type: mrr_at_1000
1107
+ value: 0.009000000000000001
1108
+ - type: mrr_at_3
1109
+ value: 0.0
1110
+ - type: mrr_at_5
1111
+ value: 0.0
1112
+ - type: ndcg_at_1
1113
+ value: 0.0
1114
+ - type: ndcg_at_10
1115
+ value: 0.0
1116
+ - type: ndcg_at_100
1117
+ value: 0.0
1118
+ - type: ndcg_at_1000
1119
+ value: 0.35200000000000004
1120
+ - type: ndcg_at_3
1121
+ value: 0.0
1122
+ - type: ndcg_at_5
1123
+ value: 0.0
1124
+ - type: precision_at_1
1125
+ value: 0.0
1126
+ - type: precision_at_10
1127
+ value: 0.0
1128
+ - type: precision_at_100
1129
+ value: 0.0
1130
+ - type: precision_at_1000
1131
+ value: 0.004
1132
+ - type: precision_at_3
1133
+ value: 0.0
1134
+ - type: precision_at_5
1135
+ value: 0.0
1136
+ - type: recall_at_1
1137
+ value: 0.0
1138
+ - type: recall_at_10
1139
+ value: 0.0
1140
+ - type: recall_at_100
1141
+ value: 0.0
1142
+ - type: recall_at_1000
1143
+ value: 2.864
1144
+ - type: recall_at_3
1145
+ value: 0.0
1146
+ - type: recall_at_5
1147
+ value: 0.0
1148
+ - task:
1149
+ type: Retrieval
1150
+ dataset:
1151
+ type: climate-fever
1152
+ name: MTEB ClimateFEVER
1153
+ config: default
1154
+ split: test
1155
+ revision: None
1156
+ metrics:
1157
+ - type: map_at_1
1158
+ value: 0.033
1159
+ - type: map_at_10
1160
+ value: 0.034
1161
+ - type: map_at_100
1162
+ value: 0.037
1163
+ - type: map_at_1000
1164
+ value: 0.039
1165
+ - type: map_at_3
1166
+ value: 0.033
1167
+ - type: map_at_5
1168
+ value: 0.033
1169
+ - type: mrr_at_1
1170
+ value: 0.065
1171
+ - type: mrr_at_10
1172
+ value: 0.07200000000000001
1173
+ - type: mrr_at_100
1174
+ value: 0.08
1175
+ - type: mrr_at_1000
1176
+ value: 0.086
1177
+ - type: mrr_at_3
1178
+ value: 0.065
1179
+ - type: mrr_at_5
1180
+ value: 0.065
1181
+ - type: ndcg_at_1
1182
+ value: 0.065
1183
+ - type: ndcg_at_10
1184
+ value: 0.047
1185
+ - type: ndcg_at_100
1186
+ value: 0.079
1187
+ - type: ndcg_at_1000
1188
+ value: 0.19
1189
+ - type: ndcg_at_3
1190
+ value: 0.04
1191
+ - type: ndcg_at_5
1192
+ value: 0.04
1193
+ - type: precision_at_1
1194
+ value: 0.065
1195
+ - type: precision_at_10
1196
+ value: 0.013
1197
+ - type: precision_at_100
1198
+ value: 0.005
1199
+ - type: precision_at_1000
1200
+ value: 0.002
1201
+ - type: precision_at_3
1202
+ value: 0.022000000000000002
1203
+ - type: precision_at_5
1204
+ value: 0.013
1205
+ - type: recall_at_1
1206
+ value: 0.033
1207
+ - type: recall_at_10
1208
+ value: 0.049
1209
+ - type: recall_at_100
1210
+ value: 0.186
1211
+ - type: recall_at_1000
1212
+ value: 0.9199999999999999
1213
+ - type: recall_at_3
1214
+ value: 0.033
1215
+ - type: recall_at_5
1216
+ value: 0.033
1217
+ - task:
1218
+ type: Retrieval
1219
+ dataset:
1220
+ type: dbpedia-entity
1221
+ name: MTEB DBPedia
1222
+ config: default
1223
+ split: test
1224
+ revision: None
1225
+ metrics:
1226
+ - type: map_at_1
1227
+ value: 0.0
1228
+ - type: map_at_10
1229
+ value: 0.0
1230
+ - type: map_at_100
1231
+ value: 0.008
1232
+ - type: map_at_1000
1233
+ value: 0.008
1234
+ - type: map_at_3
1235
+ value: 0.0
1236
+ - type: map_at_5
1237
+ value: 0.0
1238
+ - type: mrr_at_1
1239
+ value: 0.0
1240
+ - type: mrr_at_10
1241
+ value: 0.0
1242
+ - type: mrr_at_100
1243
+ value: 0.066
1244
+ - type: mrr_at_1000
1245
+ value: 0.077
1246
+ - type: mrr_at_3
1247
+ value: 0.0
1248
+ - type: mrr_at_5
1249
+ value: 0.0
1250
+ - type: ndcg_at_1
1251
+ value: 0.0
1252
+ - type: ndcg_at_10
1253
+ value: 0.0
1254
+ - type: ndcg_at_100
1255
+ value: 0.08
1256
+ - type: ndcg_at_1000
1257
+ value: 0.131
1258
+ - type: ndcg_at_3
1259
+ value: 0.0
1260
+ - type: ndcg_at_5
1261
+ value: 0.0
1262
+ - type: precision_at_1
1263
+ value: 0.0
1264
+ - type: precision_at_10
1265
+ value: 0.0
1266
+ - type: precision_at_100
1267
+ value: 0.018000000000000002
1268
+ - type: precision_at_1000
1269
+ value: 0.006999999999999999
1270
+ - type: precision_at_3
1271
+ value: 0.0
1272
+ - type: precision_at_5
1273
+ value: 0.0
1274
+ - type: recall_at_1
1275
+ value: 0.0
1276
+ - type: recall_at_10
1277
+ value: 0.0
1278
+ - type: recall_at_100
1279
+ value: 0.133
1280
+ - type: recall_at_1000
1281
+ value: 0.293
1282
+ - type: recall_at_3
1283
+ value: 0.0
1284
+ - type: recall_at_5
1285
+ value: 0.0
1286
+ - task:
1287
+ type: Classification
1288
+ dataset:
1289
+ type: mteb/emotion
1290
+ name: MTEB EmotionClassification
1291
+ config: default
1292
+ split: test
1293
+ revision: 4f58c6b202a23cf9a4da393831edf4f9183cad37
1294
+ metrics:
1295
+ - type: accuracy
1296
+ value: 19.57
1297
+ - type: f1
1298
+ value: 16.51103261738041
1299
+ - task:
1300
+ type: Retrieval
1301
+ dataset:
1302
+ type: fever
1303
+ name: MTEB FEVER
1304
+ config: default
1305
+ split: test
1306
+ revision: None
1307
+ metrics:
1308
+ - type: map_at_1
1309
+ value: 0.068
1310
+ - type: map_at_10
1311
+ value: 0.104
1312
+ - type: map_at_100
1313
+ value: 0.11
1314
+ - type: map_at_1000
1315
+ value: 0.11299999999999999
1316
+ - type: map_at_3
1317
+ value: 0.095
1318
+ - type: map_at_5
1319
+ value: 0.098
1320
+ - type: mrr_at_1
1321
+ value: 0.075
1322
+ - type: mrr_at_10
1323
+ value: 0.11299999999999999
1324
+ - type: mrr_at_100
1325
+ value: 0.11900000000000001
1326
+ - type: mrr_at_1000
1327
+ value: 0.123
1328
+ - type: mrr_at_3
1329
+ value: 0.10300000000000001
1330
+ - type: mrr_at_5
1331
+ value: 0.106
1332
+ - type: ndcg_at_1
1333
+ value: 0.075
1334
+ - type: ndcg_at_10
1335
+ value: 0.128
1336
+ - type: ndcg_at_100
1337
+ value: 0.167
1338
+ - type: ndcg_at_1000
1339
+ value: 0.291
1340
+ - type: ndcg_at_3
1341
+ value: 0.105
1342
+ - type: ndcg_at_5
1343
+ value: 0.11100000000000002
1344
+ - type: precision_at_1
1345
+ value: 0.075
1346
+ - type: precision_at_10
1347
+ value: 0.021
1348
+ - type: precision_at_100
1349
+ value: 0.004
1350
+ - type: precision_at_1000
1351
+ value: 0.002
1352
+ - type: precision_at_3
1353
+ value: 0.045
1354
+ - type: precision_at_5
1355
+ value: 0.03
1356
+ - type: recall_at_1
1357
+ value: 0.068
1358
+ - type: recall_at_10
1359
+ value: 0.19499999999999998
1360
+ - type: recall_at_100
1361
+ value: 0.40299999999999997
1362
+ - type: recall_at_1000
1363
+ value: 1.448
1364
+ - type: recall_at_3
1365
+ value: 0.128
1366
+ - type: recall_at_5
1367
+ value: 0.14300000000000002
1368
+ - task:
1369
+ type: Retrieval
1370
+ dataset:
1371
+ type: fiqa
1372
+ name: MTEB FiQA2018
1373
+ config: default
1374
+ split: test
1375
+ revision: None
1376
+ metrics:
1377
+ - type: map_at_1
1378
+ value: 0.0
1379
+ - type: map_at_10
1380
+ value: 0.0
1381
+ - type: map_at_100
1382
+ value: 0.005
1383
+ - type: map_at_1000
1384
+ value: 0.01
1385
+ - type: map_at_3
1386
+ value: 0.0
1387
+ - type: map_at_5
1388
+ value: 0.0
1389
+ - type: mrr_at_1
1390
+ value: 0.0
1391
+ - type: mrr_at_10
1392
+ value: 0.0
1393
+ - type: mrr_at_100
1394
+ value: 0.011000000000000001
1395
+ - type: mrr_at_1000
1396
+ value: 0.026
1397
+ - type: mrr_at_3
1398
+ value: 0.0
1399
+ - type: mrr_at_5
1400
+ value: 0.0
1401
+ - type: ndcg_at_1
1402
+ value: 0.0
1403
+ - type: ndcg_at_10
1404
+ value: 0.0
1405
+ - type: ndcg_at_100
1406
+ value: 0.06999999999999999
1407
+ - type: ndcg_at_1000
1408
+ value: 0.38899999999999996
1409
+ - type: ndcg_at_3
1410
+ value: 0.0
1411
+ - type: ndcg_at_5
1412
+ value: 0.0
1413
+ - type: precision_at_1
1414
+ value: 0.0
1415
+ - type: precision_at_10
1416
+ value: 0.0
1417
+ - type: precision_at_100
1418
+ value: 0.008
1419
+ - type: precision_at_1000
1420
+ value: 0.006999999999999999
1421
+ - type: precision_at_3
1422
+ value: 0.0
1423
+ - type: precision_at_5
1424
+ value: 0.0
1425
+ - type: recall_at_1
1426
+ value: 0.0
1427
+ - type: recall_at_10
1428
+ value: 0.0
1429
+ - type: recall_at_100
1430
+ value: 0.383
1431
+ - type: recall_at_1000
1432
+ value: 2.435
1433
+ - type: recall_at_3
1434
+ value: 0.0
1435
+ - type: recall_at_5
1436
+ value: 0.0
1437
+ - task:
1438
+ type: Retrieval
1439
+ dataset:
1440
+ type: hotpotqa
1441
+ name: MTEB HotpotQA
1442
+ config: default
1443
+ split: test
1444
+ revision: None
1445
+ metrics:
1446
+ - type: map_at_1
1447
+ value: 0.013999999999999999
1448
+ - type: map_at_10
1449
+ value: 0.018000000000000002
1450
+ - type: map_at_100
1451
+ value: 0.019
1452
+ - type: map_at_1000
1453
+ value: 0.02
1454
+ - type: map_at_3
1455
+ value: 0.016
1456
+ - type: map_at_5
1457
+ value: 0.016
1458
+ - type: mrr_at_1
1459
+ value: 0.027
1460
+ - type: mrr_at_10
1461
+ value: 0.034999999999999996
1462
+ - type: mrr_at_100
1463
+ value: 0.038
1464
+ - type: mrr_at_1000
1465
+ value: 0.039
1466
+ - type: mrr_at_3
1467
+ value: 0.032
1468
+ - type: mrr_at_5
1469
+ value: 0.032
1470
+ - type: ndcg_at_1
1471
+ value: 0.027
1472
+ - type: ndcg_at_10
1473
+ value: 0.026
1474
+ - type: ndcg_at_100
1475
+ value: 0.038
1476
+ - type: ndcg_at_1000
1477
+ value: 0.064
1478
+ - type: ndcg_at_3
1479
+ value: 0.021
1480
+ - type: ndcg_at_5
1481
+ value: 0.021
1482
+ - type: precision_at_1
1483
+ value: 0.027
1484
+ - type: precision_at_10
1485
+ value: 0.006999999999999999
1486
+ - type: precision_at_100
1487
+ value: 0.002
1488
+ - type: precision_at_1000
1489
+ value: 0.001
1490
+ - type: precision_at_3
1491
+ value: 0.013999999999999999
1492
+ - type: precision_at_5
1493
+ value: 0.008
1494
+ - type: recall_at_1
1495
+ value: 0.013999999999999999
1496
+ - type: recall_at_10
1497
+ value: 0.034
1498
+ - type: recall_at_100
1499
+ value: 0.08800000000000001
1500
+ - type: recall_at_1000
1501
+ value: 0.27
1502
+ - type: recall_at_3
1503
+ value: 0.02
1504
+ - type: recall_at_5
1505
+ value: 0.02
1506
+ - task:
1507
+ type: Classification
1508
+ dataset:
1509
+ type: mteb/imdb
1510
+ name: MTEB ImdbClassification
1511
+ config: default
1512
+ split: test
1513
+ revision: 3d86128a09e091d6018b6d26cad27f2739fc2db7
1514
+ metrics:
1515
+ - type: accuracy
1516
+ value: 55.4624
1517
+ - type: ap
1518
+ value: 53.20545827965495
1519
+ - type: f1
1520
+ value: 54.40019244805333
1521
+ - task:
1522
+ type: Retrieval
1523
+ dataset:
1524
+ type: msmarco
1525
+ name: MTEB MSMARCO
1526
+ config: default
1527
+ split: dev
1528
+ revision: None
1529
+ metrics:
1530
+ - type: map_at_1
1531
+ value: 0.086
1532
+ - type: map_at_10
1533
+ value: 0.109
1534
+ - type: map_at_100
1535
+ value: 0.11499999999999999
1536
+ - type: map_at_1000
1537
+ value: 0.11800000000000001
1538
+ - type: map_at_3
1539
+ value: 0.091
1540
+ - type: map_at_5
1541
+ value: 0.101
1542
+ - type: mrr_at_1
1543
+ value: 0.086
1544
+ - type: mrr_at_10
1545
+ value: 0.109
1546
+ - type: mrr_at_100
1547
+ value: 0.11499999999999999
1548
+ - type: mrr_at_1000
1549
+ value: 0.11800000000000001
1550
+ - type: mrr_at_3
1551
+ value: 0.091
1552
+ - type: mrr_at_5
1553
+ value: 0.101
1554
+ - type: ndcg_at_1
1555
+ value: 0.086
1556
+ - type: ndcg_at_10
1557
+ value: 0.133
1558
+ - type: ndcg_at_100
1559
+ value: 0.168
1560
+ - type: ndcg_at_1000
1561
+ value: 0.259
1562
+ - type: ndcg_at_3
1563
+ value: 0.093
1564
+ - type: ndcg_at_5
1565
+ value: 0.11100000000000002
1566
+ - type: precision_at_1
1567
+ value: 0.086
1568
+ - type: precision_at_10
1569
+ value: 0.021
1570
+ - type: precision_at_100
1571
+ value: 0.004
1572
+ - type: precision_at_1000
1573
+ value: 0.001
1574
+ - type: precision_at_3
1575
+ value: 0.033
1576
+ - type: precision_at_5
1577
+ value: 0.029
1578
+ - type: recall_at_1
1579
+ value: 0.086
1580
+ - type: recall_at_10
1581
+ value: 0.215
1582
+ - type: recall_at_100
1583
+ value: 0.387
1584
+ - type: recall_at_1000
1585
+ value: 1.1560000000000001
1586
+ - type: recall_at_3
1587
+ value: 0.1
1588
+ - type: recall_at_5
1589
+ value: 0.14300000000000002
1590
+ - task:
1591
+ type: Classification
1592
+ dataset:
1593
+ type: mteb/mtop_domain
1594
+ name: MTEB MTOPDomainClassification (en)
1595
+ config: en
1596
+ split: test
1597
+ revision: d80d48c1eb48d3562165c59d59d0034df9fff0bf
1598
+ metrics:
1599
+ - type: accuracy
1600
+ value: 34.42544459644323
1601
+ - type: f1
1602
+ value: 33.610930846065315
1603
+ - task:
1604
+ type: Classification
1605
+ dataset:
1606
+ type: mteb/mtop_domain
1607
+ name: MTEB MTOPDomainClassification (de)
1608
+ config: de
1609
+ split: test
1610
+ revision: d80d48c1eb48d3562165c59d59d0034df9fff0bf
1611
+ metrics:
1612
+ - type: accuracy
1613
+ value: 27.511975204282894
1614
+ - type: f1
1615
+ value: 25.84277270994464
1616
+ - task:
1617
+ type: Classification
1618
+ dataset:
1619
+ type: mteb/mtop_domain
1620
+ name: MTEB MTOPDomainClassification (es)
1621
+ config: es
1622
+ split: test
1623
+ revision: d80d48c1eb48d3562165c59d59d0034df9fff0bf
1624
+ metrics:
1625
+ - type: accuracy
1626
+ value: 29.51967978652435
1627
+ - type: f1
1628
+ value: 27.67290779782277
1629
+ - task:
1630
+ type: Classification
1631
+ dataset:
1632
+ type: mteb/mtop_domain
1633
+ name: MTEB MTOPDomainClassification (fr)
1634
+ config: fr
1635
+ split: test
1636
+ revision: d80d48c1eb48d3562165c59d59d0034df9fff0bf
1637
+ metrics:
1638
+ - type: accuracy
1639
+ value: 26.003758221108676
1640
+ - type: f1
1641
+ value: 23.831315642315282
1642
+ - task:
1643
+ type: Classification
1644
+ dataset:
1645
+ type: mteb/mtop_domain
1646
+ name: MTEB MTOPDomainClassification (hi)
1647
+ config: hi
1648
+ split: test
1649
+ revision: d80d48c1eb48d3562165c59d59d0034df9fff0bf
1650
+ metrics:
1651
+ - type: accuracy
1652
+ value: 20.132664037289356
1653
+ - type: f1
1654
+ value: 16.737043939830457
1655
+ - task:
1656
+ type: Classification
1657
+ dataset:
1658
+ type: mteb/mtop_domain
1659
+ name: MTEB MTOPDomainClassification (th)
1660
+ config: th
1661
+ split: test
1662
+ revision: d80d48c1eb48d3562165c59d59d0034df9fff0bf
1663
+ metrics:
1664
+ - type: accuracy
1665
+ value: 14.701627486437612
1666
+ - type: f1
1667
+ value: 10.849797498613762
1668
+ - task:
1669
+ type: Classification
1670
+ dataset:
1671
+ type: mteb/mtop_intent
1672
+ name: MTEB MTOPIntentClassification (en)
1673
+ config: en
1674
+ split: test
1675
+ revision: ae001d0e6b1228650b7bd1c2c65fb50ad11a8aba
1676
+ metrics:
1677
+ - type: accuracy
1678
+ value: 13.948928408572733
1679
+ - type: f1
1680
+ value: 8.562615846233708
1681
+ - task:
1682
+ type: Classification
1683
+ dataset:
1684
+ type: mteb/mtop_intent
1685
+ name: MTEB MTOPIntentClassification (de)
1686
+ config: de
1687
+ split: test
1688
+ revision: ae001d0e6b1228650b7bd1c2c65fb50ad11a8aba
1689
+ metrics:
1690
+ - type: accuracy
1691
+ value: 11.814595660749507
1692
+ - type: f1
1693
+ value: 5.353787568647624
1694
+ - task:
1695
+ type: Classification
1696
+ dataset:
1697
+ type: mteb/mtop_intent
1698
+ name: MTEB MTOPIntentClassification (es)
1699
+ config: es
1700
+ split: test
1701
+ revision: ae001d0e6b1228650b7bd1c2c65fb50ad11a8aba
1702
+ metrics:
1703
+ - type: accuracy
1704
+ value: 12.468312208138759
1705
+ - type: f1
1706
+ value: 7.566990405355253
1707
+ - task:
1708
+ type: Classification
1709
+ dataset:
1710
+ type: mteb/mtop_intent
1711
+ name: MTEB MTOPIntentClassification (fr)
1712
+ config: fr
1713
+ split: test
1714
+ revision: ae001d0e6b1228650b7bd1c2c65fb50ad11a8aba
1715
+ metrics:
1716
+ - type: accuracy
1717
+ value: 9.320388349514563
1718
+ - type: f1
1719
+ value: 5.916218245687591
1720
+ - task:
1721
+ type: Classification
1722
+ dataset:
1723
+ type: mteb/mtop_intent
1724
+ name: MTEB MTOPIntentClassification (hi)
1725
+ config: hi
1726
+ split: test
1727
+ revision: ae001d0e6b1228650b7bd1c2c65fb50ad11a8aba
1728
+ metrics:
1729
+ - type: accuracy
1730
+ value: 3.2377196127644314
1731
+ - type: f1
1732
+ value: 1.2053714075016808
1733
+ - task:
1734
+ type: Classification
1735
+ dataset:
1736
+ type: mteb/mtop_intent
1737
+ name: MTEB MTOPIntentClassification (th)
1738
+ config: th
1739
+ split: test
1740
+ revision: ae001d0e6b1228650b7bd1c2c65fb50ad11a8aba
1741
+ metrics:
1742
+ - type: accuracy
1743
+ value: 4.4159132007233275
1744
+ - type: f1
1745
+ value: 1.1992998118559788
1746
+ - task:
1747
+ type: Classification
1748
+ dataset:
1749
+ type: mteb/amazon_massive_intent
1750
+ name: MTEB MassiveIntentClassification (it)
1751
+ config: it
1752
+ split: test
1753
+ revision: 31efe3c427b0bae9c22cbb560b8f15491cc6bed7
1754
+ metrics:
1755
+ - type: accuracy
1756
+ value: 22.299932750504368
1757
+ - type: f1
1758
+ value: 20.147804322480262
1759
+ - task:
1760
+ type: Classification
1761
+ dataset:
1762
+ type: mteb/amazon_massive_scenario
1763
+ name: MTEB MassiveScenarioClassification (it)
1764
+ config: it
1765
+ split: test
1766
+ revision: 7d571f92784cd94a019292a1f45445077d0ef634
1767
+ metrics:
1768
+ - type: accuracy
1769
+ value: 27.40753194351042
1770
+ - type: f1
1771
+ value: 25.187141587127705
1772
+ - task:
1773
+ type: Clustering
1774
+ dataset:
1775
+ type: mteb/medrxiv-clustering-p2p
1776
+ name: MTEB MedrxivClusteringP2P
1777
+ config: default
1778
+ split: test
1779
+ revision: e7a26af6f3ae46b30dde8737f02c07b1505bcc73
1780
+ metrics:
1781
+ - type: v_measure
1782
+ value: 11.797082944399047
1783
+ - task:
1784
+ type: Clustering
1785
+ dataset:
1786
+ type: mteb/medrxiv-clustering-s2s
1787
+ name: MTEB MedrxivClusteringS2S
1788
+ config: default
1789
+ split: test
1790
+ revision: 35191c8c0dca72d8ff3efcd72aa802307d469663
1791
+ metrics:
1792
+ - type: v_measure
1793
+ value: 11.059126362649126
1794
+ - task:
1795
+ type: Reranking
1796
+ dataset:
1797
+ type: mteb/mind_small
1798
+ name: MTEB MindSmallReranking
1799
+ config: default
1800
+ split: test
1801
+ revision: 3bdac13927fdc888b903db93b2ffdbd90b295a69
1802
+ metrics:
1803
+ - type: map
1804
+ value: 24.452301182586798
1805
+ - type: mrr
1806
+ value: 24.374807287562085
1807
+ - task:
1808
+ type: Retrieval
1809
+ dataset:
1810
+ type: nfcorpus
1811
+ name: MTEB NFCorpus
1812
+ config: default
1813
+ split: test
1814
+ revision: None
1815
+ metrics:
1816
+ - type: map_at_1
1817
+ value: 0.03
1818
+ - type: map_at_10
1819
+ value: 0.075
1820
+ - type: map_at_100
1821
+ value: 0.208
1822
+ - type: map_at_1000
1823
+ value: 0.529
1824
+ - type: map_at_3
1825
+ value: 0.051000000000000004
1826
+ - type: map_at_5
1827
+ value: 0.055999999999999994
1828
+ - type: mrr_at_1
1829
+ value: 1.238
1830
+ - type: mrr_at_10
1831
+ value: 2.939
1832
+ - type: mrr_at_100
1833
+ value: 3.927
1834
+ - type: mrr_at_1000
1835
+ value: 4.117
1836
+ - type: mrr_at_3
1837
+ value: 1.806
1838
+ - type: mrr_at_5
1839
+ value: 2.286
1840
+ - type: ndcg_at_1
1841
+ value: 1.084
1842
+ - type: ndcg_at_10
1843
+ value: 1.133
1844
+ - type: ndcg_at_100
1845
+ value: 2.1399999999999997
1846
+ - type: ndcg_at_1000
1847
+ value: 9.362
1848
+ - type: ndcg_at_3
1849
+ value: 0.9299999999999999
1850
+ - type: ndcg_at_5
1851
+ value: 0.958
1852
+ - type: precision_at_1
1853
+ value: 1.238
1854
+ - type: precision_at_10
1855
+ value: 1.269
1856
+ - type: precision_at_100
1857
+ value: 1.155
1858
+ - type: precision_at_1000
1859
+ value: 1.0250000000000001
1860
+ - type: precision_at_3
1861
+ value: 1.032
1862
+ - type: precision_at_5
1863
+ value: 1.053
1864
+ - type: recall_at_1
1865
+ value: 0.03
1866
+ - type: recall_at_10
1867
+ value: 0.22200000000000003
1868
+ - type: recall_at_100
1869
+ value: 3.779
1870
+ - type: recall_at_1000
1871
+ value: 29.471000000000004
1872
+ - type: recall_at_3
1873
+ value: 0.087
1874
+ - type: recall_at_5
1875
+ value: 0.11199999999999999
1876
+ - task:
1877
+ type: Retrieval
1878
+ dataset:
1879
+ type: nq
1880
+ name: MTEB NQ
1881
+ config: default
1882
+ split: test
1883
+ revision: None
1884
+ metrics:
1885
+ - type: map_at_1
1886
+ value: 0.0
1887
+ - type: map_at_10
1888
+ value: 0.012
1889
+ - type: map_at_100
1890
+ value: 0.025
1891
+ - type: map_at_1000
1892
+ value: 0.027
1893
+ - type: map_at_3
1894
+ value: 0.0
1895
+ - type: map_at_5
1896
+ value: 0.006999999999999999
1897
+ - type: mrr_at_1
1898
+ value: 0.0
1899
+ - type: mrr_at_10
1900
+ value: 0.012
1901
+ - type: mrr_at_100
1902
+ value: 0.026
1903
+ - type: mrr_at_1000
1904
+ value: 0.029
1905
+ - type: mrr_at_3
1906
+ value: 0.0
1907
+ - type: mrr_at_5
1908
+ value: 0.006999999999999999
1909
+ - type: ndcg_at_1
1910
+ value: 0.0
1911
+ - type: ndcg_at_10
1912
+ value: 0.023
1913
+ - type: ndcg_at_100
1914
+ value: 0.092
1915
+ - type: ndcg_at_1000
1916
+ value: 0.16999999999999998
1917
+ - type: ndcg_at_3
1918
+ value: 0.0
1919
+ - type: ndcg_at_5
1920
+ value: 0.012
1921
+ - type: precision_at_1
1922
+ value: 0.0
1923
+ - type: precision_at_10
1924
+ value: 0.006
1925
+ - type: precision_at_100
1926
+ value: 0.004
1927
+ - type: precision_at_1000
1928
+ value: 0.001
1929
+ - type: precision_at_3
1930
+ value: 0.0
1931
+ - type: precision_at_5
1932
+ value: 0.006
1933
+ - type: recall_at_1
1934
+ value: 0.0
1935
+ - type: recall_at_10
1936
+ value: 0.058
1937
+ - type: recall_at_100
1938
+ value: 0.377
1939
+ - type: recall_at_1000
1940
+ value: 1.009
1941
+ - type: recall_at_3
1942
+ value: 0.0
1943
+ - type: recall_at_5
1944
+ value: 0.029
1945
+ - task:
1946
+ type: Retrieval
1947
+ dataset:
1948
+ type: quora
1949
+ name: MTEB QuoraRetrieval
1950
+ config: default
1951
+ split: test
1952
+ revision: None
1953
+ metrics:
1954
+ - type: map_at_1
1955
+ value: 8.943
1956
+ - type: map_at_10
1957
+ value: 10.557
1958
+ - type: map_at_100
1959
+ value: 10.777000000000001
1960
+ - type: map_at_1000
1961
+ value: 10.812
1962
+ - type: map_at_3
1963
+ value: 10.137
1964
+ - type: map_at_5
1965
+ value: 10.351
1966
+ - type: mrr_at_1
1967
+ value: 10.51
1968
+ - type: mrr_at_10
1969
+ value: 12.229
1970
+ - type: mrr_at_100
1971
+ value: 12.468
1972
+ - type: mrr_at_1000
1973
+ value: 12.504999999999999
1974
+ - type: mrr_at_3
1975
+ value: 11.777
1976
+ - type: mrr_at_5
1977
+ value: 12.014
1978
+ - type: ndcg_at_1
1979
+ value: 10.5
1980
+ - type: ndcg_at_10
1981
+ value: 11.715
1982
+ - type: ndcg_at_100
1983
+ value: 12.925
1984
+ - type: ndcg_at_1000
1985
+ value: 14.163
1986
+ - type: ndcg_at_3
1987
+ value: 10.968
1988
+ - type: ndcg_at_5
1989
+ value: 11.264000000000001
1990
+ - type: precision_at_1
1991
+ value: 10.5
1992
+ - type: precision_at_10
1993
+ value: 1.696
1994
+ - type: precision_at_100
1995
+ value: 0.248
1996
+ - type: precision_at_1000
1997
+ value: 0.039
1998
+ - type: precision_at_3
1999
+ value: 4.623
2000
+ - type: precision_at_5
2001
+ value: 3.012
2002
+ - type: recall_at_1
2003
+ value: 8.943
2004
+ - type: recall_at_10
2005
+ value: 13.746
2006
+ - type: recall_at_100
2007
+ value: 19.521
2008
+ - type: recall_at_1000
2009
+ value: 29.255
2010
+ - type: recall_at_3
2011
+ value: 11.448
2012
+ - type: recall_at_5
2013
+ value: 12.332
2014
+ - task:
2015
+ type: Clustering
2016
+ dataset:
2017
+ type: mteb/reddit-clustering
2018
+ name: MTEB RedditClustering
2019
+ config: default
2020
+ split: test
2021
+ revision: 24640382cdbf8abc73003fb0fa6d111a705499eb
2022
+ metrics:
2023
+ - type: v_measure
2024
+ value: 4.845410629021448
2025
+ - task:
2026
+ type: Clustering
2027
+ dataset:
2028
+ type: mteb/reddit-clustering-p2p
2029
+ name: MTEB RedditClusteringP2P
2030
+ config: default
2031
+ split: test
2032
+ revision: 282350215ef01743dc01b456c7f5241fa8937f16
2033
+ metrics:
2034
+ - type: v_measure
2035
+ value: 11.661900277329933
2036
+ - task:
2037
+ type: Retrieval
2038
+ dataset:
2039
+ type: scidocs
2040
+ name: MTEB SCIDOCS
2041
+ config: default
2042
+ split: test
2043
+ revision: None
2044
+ metrics:
2045
+ - type: map_at_1
2046
+ value: 0.02
2047
+ - type: map_at_10
2048
+ value: 0.036000000000000004
2049
+ - type: map_at_100
2050
+ value: 0.056999999999999995
2051
+ - type: map_at_1000
2052
+ value: 0.07200000000000001
2053
+ - type: map_at_3
2054
+ value: 0.03
2055
+ - type: map_at_5
2056
+ value: 0.03
2057
+ - type: mrr_at_1
2058
+ value: 0.1
2059
+ - type: mrr_at_10
2060
+ value: 0.181
2061
+ - type: mrr_at_100
2062
+ value: 0.27899999999999997
2063
+ - type: mrr_at_1000
2064
+ value: 0.335
2065
+ - type: mrr_at_3
2066
+ value: 0.15
2067
+ - type: mrr_at_5
2068
+ value: 0.15
2069
+ - type: ndcg_at_1
2070
+ value: 0.1
2071
+ - type: ndcg_at_10
2072
+ value: 0.079
2073
+ - type: ndcg_at_100
2074
+ value: 0.28200000000000003
2075
+ - type: ndcg_at_1000
2076
+ value: 1.228
2077
+ - type: ndcg_at_3
2078
+ value: 0.077
2079
+ - type: ndcg_at_5
2080
+ value: 0.055
2081
+ - type: precision_at_1
2082
+ value: 0.1
2083
+ - type: precision_at_10
2084
+ value: 0.04
2085
+ - type: precision_at_100
2086
+ value: 0.034
2087
+ - type: precision_at_1000
2088
+ value: 0.027999999999999997
2089
+ - type: precision_at_3
2090
+ value: 0.067
2091
+ - type: precision_at_5
2092
+ value: 0.04
2093
+ - type: recall_at_1
2094
+ value: 0.02
2095
+ - type: recall_at_10
2096
+ value: 0.08
2097
+ - type: recall_at_100
2098
+ value: 0.703
2099
+ - type: recall_at_1000
2100
+ value: 5.632000000000001
2101
+ - type: recall_at_3
2102
+ value: 0.04
2103
+ - type: recall_at_5
2104
+ value: 0.04
2105
+ - task:
2106
+ type: STS
2107
+ dataset:
2108
+ type: mteb/sickr-sts
2109
+ name: MTEB SICK-R
2110
+ config: default
2111
+ split: test
2112
+ revision: a6ea5a8cab320b040a23452cc28066d9beae2cee
2113
+ metrics:
2114
+ - type: cos_sim_pearson
2115
+ value: 22.985682461739827
2116
+ - type: cos_sim_spearman
2117
+ value: 36.63211990852576
2118
+ - type: euclidean_pearson
2119
+ value: 30.883409587497358
2120
+ - type: euclidean_spearman
2121
+ value: 36.94600975857584
2122
+ - type: manhattan_pearson
2123
+ value: 36.736693988156894
2124
+ - type: manhattan_spearman
2125
+ value: 38.98446799028811
2126
+ - task:
2127
+ type: STS
2128
+ dataset:
2129
+ type: mteb/sts12-sts
2130
+ name: MTEB STS12
2131
+ config: default
2132
+ split: test
2133
+ revision: a0d554a64d88156834ff5ae9920b964011b16384
2134
+ metrics:
2135
+ - type: cos_sim_pearson
2136
+ value: 2.5604178523517063
2137
+ - type: cos_sim_spearman
2138
+ value: 13.628378324133767
2139
+ - type: euclidean_pearson
2140
+ value: 7.9904894312005865
2141
+ - type: euclidean_spearman
2142
+ value: 15.090689818973416
2143
+ - type: manhattan_pearson
2144
+ value: 14.011092205465575
2145
+ - type: manhattan_spearman
2146
+ value: 18.04386210573924
2147
+ - task:
2148
+ type: STS
2149
+ dataset:
2150
+ type: mteb/sts13-sts
2151
+ name: MTEB STS13
2152
+ config: default
2153
+ split: test
2154
+ revision: 7e90230a92c190f1bf69ae9002b8cea547a64cca
2155
+ metrics:
2156
+ - type: cos_sim_pearson
2157
+ value: 18.59414271348264
2158
+ - type: cos_sim_spearman
2159
+ value: 23.758346452530105
2160
+ - type: euclidean_pearson
2161
+ value: 22.985667268384162
2162
+ - type: euclidean_spearman
2163
+ value: 25.143580728437183
2164
+ - type: manhattan_pearson
2165
+ value: 28.109316236003
2166
+ - type: manhattan_spearman
2167
+ value: 29.403691387442727
2168
+ - task:
2169
+ type: STS
2170
+ dataset:
2171
+ type: mteb/sts14-sts
2172
+ name: MTEB STS14
2173
+ config: default
2174
+ split: test
2175
+ revision: 6031580fec1f6af667f0bd2da0a551cf4f0b2375
2176
+ metrics:
2177
+ - type: cos_sim_pearson
2178
+ value: 8.292349216262673
2179
+ - type: cos_sim_spearman
2180
+ value: 15.648383623810028
2181
+ - type: euclidean_pearson
2182
+ value: 12.136605941196938
2183
+ - type: euclidean_spearman
2184
+ value: 16.37547051924145
2185
+ - type: manhattan_pearson
2186
+ value: 21.049918496319524
2187
+ - type: manhattan_spearman
2188
+ value: 22.168125518695295
2189
+ - task:
2190
+ type: STS
2191
+ dataset:
2192
+ type: mteb/sts15-sts
2193
+ name: MTEB STS15
2194
+ config: default
2195
+ split: test
2196
+ revision: ae752c7c21bf194d8b67fd573edf7ae58183cbe3
2197
+ metrics:
2198
+ - type: cos_sim_pearson
2199
+ value: 21.0574858763695
2200
+ - type: cos_sim_spearman
2201
+ value: 28.24306393347735
2202
+ - type: euclidean_pearson
2203
+ value: 25.67620587891895
2204
+ - type: euclidean_spearman
2205
+ value: 28.802005577995292
2206
+ - type: manhattan_pearson
2207
+ value: 33.333168689238846
2208
+ - type: manhattan_spearman
2209
+ value: 33.7249701052437
2210
+ - task:
2211
+ type: STS
2212
+ dataset:
2213
+ type: mteb/sts16-sts
2214
+ name: MTEB STS16
2215
+ config: default
2216
+ split: test
2217
+ revision: 4d8694f8f0e0100860b497b999b3dbed754a0513
2218
+ metrics:
2219
+ - type: cos_sim_pearson
2220
+ value: 23.345013082866608
2221
+ - type: cos_sim_spearman
2222
+ value: 32.08654087568418
2223
+ - type: euclidean_pearson
2224
+ value: 29.1302480053082
2225
+ - type: euclidean_spearman
2226
+ value: 32.723960824054224
2227
+ - type: manhattan_pearson
2228
+ value: 34.73363269084969
2229
+ - type: manhattan_spearman
2230
+ value: 35.946509333697016
2231
+ - task:
2232
+ type: STS
2233
+ dataset:
2234
+ type: mteb/sts17-crosslingual-sts
2235
+ name: MTEB STS17 (ko-ko)
2236
+ config: ko-ko
2237
+ split: test
2238
+ revision: af5e6fb845001ecf41f4c1e033ce921939a2a68d
2239
+ metrics:
2240
+ - type: cos_sim_pearson
2241
+ value: -1.0510693376748088
2242
+ - type: cos_sim_spearman
2243
+ value: 3.7330446273344897
2244
+ - type: euclidean_pearson
2245
+ value: -0.2108306777168949
2246
+ - type: euclidean_spearman
2247
+ value: 3.627369552634812
2248
+ - type: manhattan_pearson
2249
+ value: 1.5031538964733262
2250
+ - type: manhattan_spearman
2251
+ value: 5.004910973166412
2252
+ - task:
2253
+ type: STS
2254
+ dataset:
2255
+ type: mteb/sts17-crosslingual-sts
2256
+ name: MTEB STS17 (ar-ar)
2257
+ config: ar-ar
2258
+ split: test
2259
+ revision: af5e6fb845001ecf41f4c1e033ce921939a2a68d
2260
+ metrics:
2261
+ - type: cos_sim_pearson
2262
+ value: 11.578393453214481
2263
+ - type: cos_sim_spearman
2264
+ value: 21.790827126422034
2265
+ - type: euclidean_pearson
2266
+ value: 19.06071141618503
2267
+ - type: euclidean_spearman
2268
+ value: 22.161779839314196
2269
+ - type: manhattan_pearson
2270
+ value: 17.725623325242474
2271
+ - type: manhattan_spearman
2272
+ value: 20.43157514666076
2273
+ - task:
2274
+ type: STS
2275
+ dataset:
2276
+ type: mteb/sts17-crosslingual-sts
2277
+ name: MTEB STS17 (en-ar)
2278
+ config: en-ar
2279
+ split: test
2280
+ revision: af5e6fb845001ecf41f4c1e033ce921939a2a68d
2281
+ metrics:
2282
+ - type: cos_sim_pearson
2283
+ value: 17.29239442081278
2284
+ - type: cos_sim_spearman
2285
+ value: 16.292056207420202
2286
+ - type: euclidean_pearson
2287
+ value: 16.503491974377727
2288
+ - type: euclidean_spearman
2289
+ value: 15.541440440884651
2290
+ - type: manhattan_pearson
2291
+ value: 21.158901085317268
2292
+ - type: manhattan_spearman
2293
+ value: 21.781541830999963
2294
+ - task:
2295
+ type: STS
2296
+ dataset:
2297
+ type: mteb/sts17-crosslingual-sts
2298
+ name: MTEB STS17 (en-de)
2299
+ config: en-de
2300
+ split: test
2301
+ revision: af5e6fb845001ecf41f4c1e033ce921939a2a68d
2302
+ metrics:
2303
+ - type: cos_sim_pearson
2304
+ value: 22.589778414759955
2305
+ - type: cos_sim_spearman
2306
+ value: 18.997838545450612
2307
+ - type: euclidean_pearson
2308
+ value: 21.90016323186628
2309
+ - type: euclidean_spearman
2310
+ value: 18.905160536986692
2311
+ - type: manhattan_pearson
2312
+ value: 16.913499882046576
2313
+ - type: manhattan_spearman
2314
+ value: 15.669617887287327
2315
+ - task:
2316
+ type: STS
2317
+ dataset:
2318
+ type: mteb/sts17-crosslingual-sts
2319
+ name: MTEB STS17 (en-en)
2320
+ config: en-en
2321
+ split: test
2322
+ revision: af5e6fb845001ecf41f4c1e033ce921939a2a68d
2323
+ metrics:
2324
+ - type: cos_sim_pearson
2325
+ value: 33.05366168385912
2326
+ - type: cos_sim_spearman
2327
+ value: 37.781952608504135
2328
+ - type: euclidean_pearson
2329
+ value: 37.085941074268966
2330
+ - type: euclidean_spearman
2331
+ value: 37.8364215997913
2332
+ - type: manhattan_pearson
2333
+ value: 42.316206028770736
2334
+ - type: manhattan_spearman
2335
+ value: 41.74208275697782
2336
+ - task:
2337
+ type: STS
2338
+ dataset:
2339
+ type: mteb/sts17-crosslingual-sts
2340
+ name: MTEB STS17 (en-tr)
2341
+ config: en-tr
2342
+ split: test
2343
+ revision: af5e6fb845001ecf41f4c1e033ce921939a2a68d
2344
+ metrics:
2345
+ - type: cos_sim_pearson
2346
+ value: 23.991765076225654
2347
+ - type: cos_sim_spearman
2348
+ value: 20.588105042260104
2349
+ - type: euclidean_pearson
2350
+ value: 19.712724760717997
2351
+ - type: euclidean_spearman
2352
+ value: 19.253030106327383
2353
+ - type: manhattan_pearson
2354
+ value: 16.84198288544301
2355
+ - type: manhattan_spearman
2356
+ value: 17.61549197324614
2357
+ - task:
2358
+ type: STS
2359
+ dataset:
2360
+ type: mteb/sts17-crosslingual-sts
2361
+ name: MTEB STS17 (es-en)
2362
+ config: es-en
2363
+ split: test
2364
+ revision: af5e6fb845001ecf41f4c1e033ce921939a2a68d
2365
+ metrics:
2366
+ - type: cos_sim_pearson
2367
+ value: 24.968980346008365
2368
+ - type: cos_sim_spearman
2369
+ value: 27.252856647926286
2370
+ - type: euclidean_pearson
2371
+ value: 24.58496162769602
2372
+ - type: euclidean_spearman
2373
+ value: 26.034323771297824
2374
+ - type: manhattan_pearson
2375
+ value: 22.40058722998031
2376
+ - type: manhattan_spearman
2377
+ value: 24.459230575688714
2378
+ - task:
2379
+ type: STS
2380
+ dataset:
2381
+ type: mteb/sts17-crosslingual-sts
2382
+ name: MTEB STS17 (es-es)
2383
+ config: es-es
2384
+ split: test
2385
+ revision: af5e6fb845001ecf41f4c1e033ce921939a2a68d
2386
+ metrics:
2387
+ - type: cos_sim_pearson
2388
+ value: 32.94464820482554
2389
+ - type: cos_sim_spearman
2390
+ value: 40.462017863354085
2391
+ - type: euclidean_pearson
2392
+ value: 38.88676423326504
2393
+ - type: euclidean_spearman
2394
+ value: 40.47190508444464
2395
+ - type: manhattan_pearson
2396
+ value: 39.692234741895874
2397
+ - type: manhattan_spearman
2398
+ value: 39.478725017400166
2399
+ - task:
2400
+ type: STS
2401
+ dataset:
2402
+ type: mteb/sts17-crosslingual-sts
2403
+ name: MTEB STS17 (fr-en)
2404
+ config: fr-en
2405
+ split: test
2406
+ revision: af5e6fb845001ecf41f4c1e033ce921939a2a68d
2407
+ metrics:
2408
+ - type: cos_sim_pearson
2409
+ value: 23.342163445031748
2410
+ - type: cos_sim_spearman
2411
+ value: 27.27097487628386
2412
+ - type: euclidean_pearson
2413
+ value: 24.76789948947651
2414
+ - type: euclidean_spearman
2415
+ value: 25.81926286149811
2416
+ - type: manhattan_pearson
2417
+ value: 25.475723026689685
2418
+ - type: manhattan_spearman
2419
+ value: 25.813420033921787
2420
+ - task:
2421
+ type: STS
2422
+ dataset:
2423
+ type: mteb/sts17-crosslingual-sts
2424
+ name: MTEB STS17 (it-en)
2425
+ config: it-en
2426
+ split: test
2427
+ revision: af5e6fb845001ecf41f4c1e033ce921939a2a68d
2428
+ metrics:
2429
+ - type: cos_sim_pearson
2430
+ value: 7.717130622537516
2431
+ - type: cos_sim_spearman
2432
+ value: 9.437435044381033
2433
+ - type: euclidean_pearson
2434
+ value: 7.9423592614250555
2435
+ - type: euclidean_spearman
2436
+ value: 8.010785684725303
2437
+ - type: manhattan_pearson
2438
+ value: 8.217253457576026
2439
+ - type: manhattan_spearman
2440
+ value: 8.247881477427692
2441
+ - task:
2442
+ type: STS
2443
+ dataset:
2444
+ type: mteb/sts17-crosslingual-sts
2445
+ name: MTEB STS17 (nl-en)
2446
+ config: nl-en
2447
+ split: test
2448
+ revision: af5e6fb845001ecf41f4c1e033ce921939a2a68d
2449
+ metrics:
2450
+ - type: cos_sim_pearson
2451
+ value: 16.16499446427259
2452
+ - type: cos_sim_spearman
2453
+ value: 17.21754435880688
2454
+ - type: euclidean_pearson
2455
+ value: 16.325415307049443
2456
+ - type: euclidean_spearman
2457
+ value: 16.293339731059863
2458
+ - type: manhattan_pearson
2459
+ value: 19.410680542804005
2460
+ - type: manhattan_spearman
2461
+ value: 18.95736158862253
2462
+ - task:
2463
+ type: STS
2464
+ dataset:
2465
+ type: mteb/sts22-crosslingual-sts
2466
+ name: MTEB STS22 (it)
2467
+ config: it
2468
+ split: test
2469
+ revision: 6d1ba47164174a496b7fa5d3569dae26a6813b80
2470
+ metrics:
2471
+ - type: cos_sim_pearson
2472
+ value: 30.67175493186678
2473
+ - type: cos_sim_spearman
2474
+ value: 37.92638638971281
2475
+ - type: euclidean_pearson
2476
+ value: 37.47072224334179
2477
+ - type: euclidean_spearman
2478
+ value: 39.23036609148336
2479
+ - type: manhattan_pearson
2480
+ value: 42.92657347688227
2481
+ - type: manhattan_spearman
2482
+ value: 43.93955531904934
2483
+ - task:
2484
+ type: STS
2485
+ dataset:
2486
+ type: mteb/stsbenchmark-sts
2487
+ name: MTEB STSBenchmark
2488
+ config: default
2489
+ split: test
2490
+ revision: b0fddb56ed78048fa8b90373c8a3cfc37b684831
2491
+ metrics:
2492
+ - type: cos_sim_pearson
2493
+ value: 15.51842992576514
2494
+ - type: cos_sim_spearman
2495
+ value: 22.738964472371965
2496
+ - type: euclidean_pearson
2497
+ value: 20.68981398028433
2498
+ - type: euclidean_spearman
2499
+ value: 23.35518754871016
2500
+ - type: manhattan_pearson
2501
+ value: 26.488467867333142
2502
+ - type: manhattan_spearman
2503
+ value: 27.318174781418055
2504
+ - task:
2505
+ type: Reranking
2506
+ dataset:
2507
+ type: mteb/scidocs-reranking
2508
+ name: MTEB SciDocsRR
2509
+ config: default
2510
+ split: test
2511
+ revision: d3c5e1fc0b855ab6097bf1cda04dd73947d7caab
2512
+ metrics:
2513
+ - type: map
2514
+ value: 30.728870040728083
2515
+ - type: mrr
2516
+ value: 45.1852849401869
2517
+ - task:
2518
+ type: Retrieval
2519
+ dataset:
2520
+ type: scifact
2521
+ name: MTEB SciFact
2522
+ config: default
2523
+ split: test
2524
+ revision: None
2525
+ metrics:
2526
+ - type: map_at_1
2527
+ value: 0.0
2528
+ - type: map_at_10
2529
+ value: 0.104
2530
+ - type: map_at_100
2531
+ value: 0.265
2532
+ - type: map_at_1000
2533
+ value: 0.334
2534
+ - type: map_at_3
2535
+ value: 0.0
2536
+ - type: map_at_5
2537
+ value: 0.067
2538
+ - type: mrr_at_1
2539
+ value: 0.0
2540
+ - type: mrr_at_10
2541
+ value: 0.104
2542
+ - type: mrr_at_100
2543
+ value: 0.27
2544
+ - type: mrr_at_1000
2545
+ value: 0.345
2546
+ - type: mrr_at_3
2547
+ value: 0.0
2548
+ - type: mrr_at_5
2549
+ value: 0.067
2550
+ - type: ndcg_at_1
2551
+ value: 0.0
2552
+ - type: ndcg_at_10
2553
+ value: 0.22899999999999998
2554
+ - type: ndcg_at_100
2555
+ value: 1.044
2556
+ - type: ndcg_at_1000
2557
+ value: 3.911
2558
+ - type: ndcg_at_3
2559
+ value: 0.0
2560
+ - type: ndcg_at_5
2561
+ value: 0.129
2562
+ - type: precision_at_1
2563
+ value: 0.0
2564
+ - type: precision_at_10
2565
+ value: 0.067
2566
+ - type: precision_at_100
2567
+ value: 0.05
2568
+ - type: precision_at_1000
2569
+ value: 0.032
2570
+ - type: precision_at_3
2571
+ value: 0.0
2572
+ - type: precision_at_5
2573
+ value: 0.067
2574
+ - type: recall_at_1
2575
+ value: 0.0
2576
+ - type: recall_at_10
2577
+ value: 0.6669999999999999
2578
+ - type: recall_at_100
2579
+ value: 4.583
2580
+ - type: recall_at_1000
2581
+ value: 28.910999999999998
2582
+ - type: recall_at_3
2583
+ value: 0.0
2584
+ - type: recall_at_5
2585
+ value: 0.333
2586
+ - task:
2587
+ type: PairClassification
2588
+ dataset:
2589
+ type: mteb/sprintduplicatequestions-pairclassification
2590
+ name: MTEB SprintDuplicateQuestions
2591
+ config: default
2592
+ split: test
2593
+ revision: d66bd1f72af766a5cc4b0ca5e00c162f89e8cc46
2594
+ metrics:
2595
+ - type: cos_sim_accuracy
2596
+ value: 99.01089108910891
2597
+ - type: cos_sim_ap
2598
+ value: 2.649295518982714
2599
+ - type: cos_sim_f1
2600
+ value: 6.26322471434617
2601
+ - type: cos_sim_precision
2602
+ value: 3.972088030059045
2603
+ - type: cos_sim_recall
2604
+ value: 14.799999999999999
2605
+ - type: dot_accuracy
2606
+ value: 99.0089108910891
2607
+ - type: dot_ap
2608
+ value: 1.713700413108619
2609
+ - type: dot_f1
2610
+ value: 5.073705862187179
2611
+ - type: dot_precision
2612
+ value: 3.061646669424907
2613
+ - type: dot_recall
2614
+ value: 14.799999999999999
2615
+ - type: euclidean_accuracy
2616
+ value: 99.01089108910891
2617
+ - type: euclidean_ap
2618
+ value: 2.744099763470491
2619
+ - type: euclidean_f1
2620
+ value: 6.291706387035273
2621
+ - type: euclidean_precision
2622
+ value: 4.611085235211924
2623
+ - type: euclidean_recall
2624
+ value: 9.9
2625
+ - type: manhattan_accuracy
2626
+ value: 99.01089108910891
2627
+ - type: manhattan_ap
2628
+ value: 3.2781717730991327
2629
+ - type: manhattan_f1
2630
+ value: 7.68245838668374
2631
+ - type: manhattan_precision
2632
+ value: 10.676156583629894
2633
+ - type: manhattan_recall
2634
+ value: 6.0
2635
+ - type: max_accuracy
2636
+ value: 99.01089108910891
2637
+ - type: max_ap
2638
+ value: 3.2781717730991327
2639
+ - type: max_f1
2640
+ value: 7.68245838668374
2641
+ - task:
2642
+ type: Clustering
2643
+ dataset:
2644
+ type: mteb/stackexchange-clustering
2645
+ name: MTEB StackExchangeClustering
2646
+ config: default
2647
+ split: test
2648
+ revision: 6cbc1f7b2bc0622f2e39d2c77fa502909748c259
2649
+ metrics:
2650
+ - type: v_measure
2651
+ value: 8.602221384568187
2652
+ - task:
2653
+ type: Clustering
2654
+ dataset:
2655
+ type: mteb/stackexchange-clustering-p2p
2656
+ name: MTEB StackExchangeClusteringP2P
2657
+ config: default
2658
+ split: test
2659
+ revision: 815ca46b2622cec33ccafc3735d572c266efdb44
2660
+ metrics:
2661
+ - type: v_measure
2662
+ value: 25.619483506865205
2663
+ - task:
2664
+ type: Reranking
2665
+ dataset:
2666
+ type: mteb/stackoverflowdupquestions-reranking
2667
+ name: MTEB StackOverflowDupQuestions
2668
+ config: default
2669
+ split: test
2670
+ revision: e185fbe320c72810689fc5848eb6114e1ef5ec69
2671
+ metrics:
2672
+ - type: map
2673
+ value: 20.02237719216371
2674
+ - type: mrr
2675
+ value: 18.132453739071387
2676
+ - task:
2677
+ type: Retrieval
2678
+ dataset:
2679
+ type: trec-covid
2680
+ name: MTEB TRECCOVID
2681
+ config: default
2682
+ split: test
2683
+ revision: None
2684
+ metrics:
2685
+ - type: map_at_1
2686
+ value: 0.004
2687
+ - type: map_at_10
2688
+ value: 0.009000000000000001
2689
+ - type: map_at_100
2690
+ value: 0.011000000000000001
2691
+ - type: map_at_1000
2692
+ value: 0.016
2693
+ - type: map_at_3
2694
+ value: 0.004
2695
+ - type: map_at_5
2696
+ value: 0.006999999999999999
2697
+ - type: mrr_at_1
2698
+ value: 2.0
2699
+ - type: mrr_at_10
2700
+ value: 4.233
2701
+ - type: mrr_at_100
2702
+ value: 4.936
2703
+ - type: mrr_at_1000
2704
+ value: 5.103
2705
+ - type: mrr_at_3
2706
+ value: 2.0
2707
+ - type: mrr_at_5
2708
+ value: 3.9
2709
+ - type: ndcg_at_1
2710
+ value: 1.0
2711
+ - type: ndcg_at_10
2712
+ value: 1.1039999999999999
2713
+ - type: ndcg_at_100
2714
+ value: 0.486
2715
+ - type: ndcg_at_1000
2716
+ value: 0.666
2717
+ - type: ndcg_at_3
2718
+ value: 0.469
2719
+ - type: ndcg_at_5
2720
+ value: 1.347
2721
+ - type: precision_at_1
2722
+ value: 2.0
2723
+ - type: precision_at_10
2724
+ value: 1.4000000000000001
2725
+ - type: precision_at_100
2726
+ value: 0.52
2727
+ - type: precision_at_1000
2728
+ value: 0.37
2729
+ - type: precision_at_3
2730
+ value: 0.6669999999999999
2731
+ - type: precision_at_5
2732
+ value: 2.0
2733
+ - type: recall_at_1
2734
+ value: 0.004
2735
+ - type: recall_at_10
2736
+ value: 0.024
2737
+ - type: recall_at_100
2738
+ value: 0.09
2739
+ - type: recall_at_1000
2740
+ value: 0.807
2741
+ - type: recall_at_3
2742
+ value: 0.004
2743
+ - type: recall_at_5
2744
+ value: 0.016
2745
+ - task:
2746
+ type: Retrieval
2747
+ dataset:
2748
+ type: webis-touche2020
2749
+ name: MTEB Touche2020
2750
+ config: default
2751
+ split: test
2752
+ revision: None
2753
+ metrics:
2754
+ - type: map_at_1
2755
+ value: 0.0
2756
+ - type: map_at_10
2757
+ value: 0.0
2758
+ - type: map_at_100
2759
+ value: 0.0
2760
+ - type: map_at_1000
2761
+ value: 0.0
2762
+ - type: map_at_3
2763
+ value: 0.0
2764
+ - type: map_at_5
2765
+ value: 0.0
2766
+ - type: mrr_at_1
2767
+ value: 0.0
2768
+ - type: mrr_at_10
2769
+ value: 0.0
2770
+ - type: mrr_at_100
2771
+ value: 0.0
2772
+ - type: mrr_at_1000
2773
+ value: 0.0
2774
+ - type: mrr_at_3
2775
+ value: 0.0
2776
+ - type: mrr_at_5
2777
+ value: 0.0
2778
+ - type: ndcg_at_1
2779
+ value: 0.0
2780
+ - type: ndcg_at_10
2781
+ value: 0.0
2782
+ - type: ndcg_at_100
2783
+ value: 0.0
2784
+ - type: ndcg_at_1000
2785
+ value: 0.0
2786
+ - type: ndcg_at_3
2787
+ value: 0.0
2788
+ - type: ndcg_at_5
2789
+ value: 0.0
2790
+ - type: precision_at_1
2791
+ value: 0.0
2792
+ - type: precision_at_10
2793
+ value: 0.0
2794
+ - type: precision_at_100
2795
+ value: 0.0
2796
+ - type: precision_at_1000
2797
+ value: 0.0
2798
+ - type: precision_at_3
2799
+ value: 0.0
2800
+ - type: precision_at_5
2801
+ value: 0.0
2802
+ - type: recall_at_1
2803
+ value: 0.0
2804
+ - type: recall_at_10
2805
+ value: 0.0
2806
+ - type: recall_at_100
2807
+ value: 0.0
2808
+ - type: recall_at_1000
2809
+ value: 0.0
2810
+ - type: recall_at_3
2811
+ value: 0.0
2812
+ - type: recall_at_5
2813
+ value: 0.0
2814
+ - task:
2815
+ type: Classification
2816
+ dataset:
2817
+ type: mteb/toxic_conversations_50k
2818
+ name: MTEB ToxicConversationsClassification
2819
+ config: default
2820
+ split: test
2821
+ revision: d7c0de2777da35d6aae2200a62c6e0e5af397c4c
2822
+ metrics:
2823
+ - type: accuracy
2824
+ value: 55.065799999999996
2825
+ - type: ap
2826
+ value: 8.3123142340845
2827
+ - type: f1
2828
+ value: 41.78797425886187
2829
+ - task:
2830
+ type: Classification
2831
+ dataset:
2832
+ type: mteb/tweet_sentiment_extraction
2833
+ name: MTEB TweetSentimentExtractionClassification
2834
+ config: default
2835
+ split: test
2836
+ revision: d604517c81ca91fe16a244d1248fc021f9ecee7a
2837
+ metrics:
2838
+ - type: accuracy
2839
+ value: 41.177136389360506
2840
+ - type: f1
2841
+ value: 40.882588170909244
2842
+ - task:
2843
+ type: Clustering
2844
+ dataset:
2845
+ type: mteb/twentynewsgroups-clustering
2846
+ name: MTEB TwentyNewsgroupsClustering
2847
+ config: default
2848
+ split: test
2849
+ revision: 6125ec4e24fa026cec8a478383ee943acfbd5449
2850
+ metrics:
2851
+ - type: v_measure
2852
+ value: 10.046104242285523
2853
+ - task:
2854
+ type: PairClassification
2855
+ dataset:
2856
+ type: mteb/twittersemeval2015-pairclassification
2857
+ name: MTEB TwitterSemEval2015
2858
+ config: default
2859
+ split: test
2860
+ revision: 70970daeab8776df92f5ea462b6173c0b46fd2d1
2861
+ metrics:
2862
+ - type: cos_sim_accuracy
2863
+ value: 77.63604935328128
2864
+ - type: cos_sim_ap
2865
+ value: 32.380569186976906
2866
+ - type: cos_sim_f1
2867
+ value: 38.29160530191458
2868
+ - type: cos_sim_precision
2869
+ value: 27.110086708374492
2870
+ - type: cos_sim_recall
2871
+ value: 65.17150395778364
2872
+ - type: dot_accuracy
2873
+ value: 77.40954878702986
2874
+ - type: dot_ap
2875
+ value: 28.34039741384004
2876
+ - type: dot_f1
2877
+ value: 37.45059908412271
2878
+ - type: dot_precision
2879
+ value: 24.565879351493706
2880
+ - type: dot_recall
2881
+ value: 78.7598944591029
2882
+ - type: euclidean_accuracy
2883
+ value: 77.63604935328128
2884
+ - type: euclidean_ap
2885
+ value: 32.40705726976434
2886
+ - type: euclidean_f1
2887
+ value: 38.365584519430676
2888
+ - type: euclidean_precision
2889
+ value: 27.524093620927033
2890
+ - type: euclidean_recall
2891
+ value: 63.298153034300796
2892
+ - type: manhattan_accuracy
2893
+ value: 77.70757584788699
2894
+ - type: manhattan_ap
2895
+ value: 33.03410839977045
2896
+ - type: manhattan_f1
2897
+ value: 39.04353514063523
2898
+ - type: manhattan_precision
2899
+ value: 26.943524927274552
2900
+ - type: manhattan_recall
2901
+ value: 70.87071240105541
2902
+ - type: max_accuracy
2903
+ value: 77.70757584788699
2904
+ - type: max_ap
2905
+ value: 33.03410839977045
2906
+ - type: max_f1
2907
+ value: 39.04353514063523
2908
+ - task:
2909
+ type: PairClassification
2910
+ dataset:
2911
+ type: mteb/twitterurlcorpus-pairclassification
2912
+ name: MTEB TwitterURLCorpus
2913
+ config: default
2914
+ split: test
2915
+ revision: 8b6510b0b1fa4e4c4f879467980e9be563ec1cdf
2916
+ metrics:
2917
+ - type: cos_sim_accuracy
2918
+ value: 75.80626382582373
2919
+ - type: cos_sim_ap
2920
+ value: 41.94768516713251
2921
+ - type: cos_sim_f1
2922
+ value: 44.69374385849984
2923
+ - type: cos_sim_precision
2924
+ value: 34.620263870094725
2925
+ - type: cos_sim_recall
2926
+ value: 63.035109331690784
2927
+ - type: dot_accuracy
2928
+ value: 74.79528078550084
2929
+ - type: dot_ap
2930
+ value: 33.69361208467778
2931
+ - type: dot_f1
2932
+ value: 44.620064092118845
2933
+ - type: dot_precision
2934
+ value: 34.467567340773684
2935
+ - type: dot_recall
2936
+ value: 63.250692947336006
2937
+ - type: euclidean_accuracy
2938
+ value: 75.98866767570924
2939
+ - type: euclidean_ap
2940
+ value: 42.65497342948604
2941
+ - type: euclidean_f1
2942
+ value: 44.794497753619176
2943
+ - type: euclidean_precision
2944
+ value: 35.006501950585175
2945
+ - type: euclidean_recall
2946
+ value: 62.180474283954425
2947
+ - type: manhattan_accuracy
2948
+ value: 76.37870143982613
2949
+ - type: manhattan_ap
2950
+ value: 46.65401496383161
2951
+ - type: manhattan_f1
2952
+ value: 48.14085011643678
2953
+ - type: manhattan_precision
2954
+ value: 36.0535091417839
2955
+ - type: manhattan_recall
2956
+ value: 72.42069602710194
2957
+ - type: max_accuracy
2958
+ value: 76.37870143982613
2959
+ - type: max_ap
2960
+ value: 46.65401496383161
2961
+ - type: max_f1
2962
+ value: 48.14085011643678
2963
+ ---
2964
  pipeline_tag: sentence-similarity
2965
  tags:
2966
  - sentence-transformers