avsolatorio commited on
Commit
bcb505b
1 Parent(s): 2bef05e

Add model readme

Browse files

Signed-off-by: Aivin V. Solatorio <avsolatorio@gmail.com>

Files changed (1) hide show
  1. README.md +2682 -0
README.md CHANGED
@@ -1,3 +1,2685 @@
1
  ---
 
 
 
2
  license: mit
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
3
  ---
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
  ---
2
+ language:
3
+ - en
4
+ library_name: sentence-transformers
5
  license: mit
6
+ pipeline_tag: sentence-similarity
7
+ tags:
8
+ - feature-extraction
9
+ - mteb
10
+ - sentence-similarity
11
+ - sentence-transformers
12
+
13
+ model-index:
14
+ - name: GIST-all-MiniLM-L6-v2
15
+ results:
16
+ - task:
17
+ type: Classification
18
+ dataset:
19
+ type: mteb/amazon_counterfactual
20
+ name: MTEB AmazonCounterfactualClassification (en)
21
+ config: en
22
+ split: test
23
+ revision: e8379541af4e31359cca9fbcf4b00f2671dba205
24
+ metrics:
25
+ - type: accuracy
26
+ value: 69.68656716417911
27
+ - type: ap
28
+ value: 31.84640905923114
29
+ - type: f1
30
+ value: 63.4379647836158
31
+ - task:
32
+ type: Classification
33
+ dataset:
34
+ type: mteb/amazon_polarity
35
+ name: MTEB AmazonPolarityClassification
36
+ config: default
37
+ split: test
38
+ revision: e2d317d38cd51312af73b3d32a06d1a08b442046
39
+ metrics:
40
+ - type: accuracy
41
+ value: 82.078025
42
+ - type: ap
43
+ value: 77.3451894150185
44
+ - type: f1
45
+ value: 81.97258648080654
46
+ - task:
47
+ type: Classification
48
+ dataset:
49
+ type: mteb/amazon_reviews_multi
50
+ name: MTEB AmazonReviewsClassification (en)
51
+ config: en
52
+ split: test
53
+ revision: 1399c76144fd37290681b995c656ef9b2e06e26d
54
+ metrics:
55
+ - type: accuracy
56
+ value: 38.254
57
+ - type: f1
58
+ value: 37.940387801030376
59
+ - task:
60
+ type: Retrieval
61
+ dataset:
62
+ type: arguana
63
+ name: MTEB ArguAna
64
+ config: default
65
+ split: test
66
+ revision: None
67
+ metrics:
68
+ - type: map_at_1
69
+ value: 28.876
70
+ - type: map_at_10
71
+ value: 44.741
72
+ - type: map_at_100
73
+ value: 45.688
74
+ - type: map_at_1000
75
+ value: 45.695
76
+ - type: map_at_3
77
+ value: 39.829
78
+ - type: map_at_5
79
+ value: 42.646
80
+ - type: mrr_at_1
81
+ value: 30.156
82
+ - type: mrr_at_10
83
+ value: 45.196
84
+ - type: mrr_at_100
85
+ value: 46.149
86
+ - type: mrr_at_1000
87
+ value: 46.156000000000006
88
+ - type: mrr_at_3
89
+ value: 40.339000000000006
90
+ - type: mrr_at_5
91
+ value: 43.120000000000005
92
+ - type: ndcg_at_1
93
+ value: 28.876
94
+ - type: ndcg_at_10
95
+ value: 53.581
96
+ - type: ndcg_at_100
97
+ value: 57.428000000000004
98
+ - type: ndcg_at_1000
99
+ value: 57.599000000000004
100
+ - type: ndcg_at_3
101
+ value: 43.46
102
+ - type: ndcg_at_5
103
+ value: 48.501
104
+ - type: precision_at_1
105
+ value: 28.876
106
+ - type: precision_at_10
107
+ value: 8.186
108
+ - type: precision_at_100
109
+ value: 0.9820000000000001
110
+ - type: precision_at_1000
111
+ value: 0.1
112
+ - type: precision_at_3
113
+ value: 17.994
114
+ - type: precision_at_5
115
+ value: 13.229
116
+ - type: recall_at_1
117
+ value: 28.876
118
+ - type: recall_at_10
119
+ value: 81.863
120
+ - type: recall_at_100
121
+ value: 98.222
122
+ - type: recall_at_1000
123
+ value: 99.502
124
+ - type: recall_at_3
125
+ value: 53.983000000000004
126
+ - type: recall_at_5
127
+ value: 66.145
128
+ - task:
129
+ type: Clustering
130
+ dataset:
131
+ type: mteb/arxiv-clustering-p2p
132
+ name: MTEB ArxivClusteringP2P
133
+ config: default
134
+ split: test
135
+ revision: a122ad7f3f0291bf49cc6f4d32aa80929df69d5d
136
+ metrics:
137
+ - type: v_measure
138
+ value: 44.81109445338116
139
+ - task:
140
+ type: Clustering
141
+ dataset:
142
+ type: mteb/arxiv-clustering-s2s
143
+ name: MTEB ArxivClusteringS2S
144
+ config: default
145
+ split: test
146
+ revision: f910caf1a6075f7329cdf8c1a6135696f37dbd53
147
+ metrics:
148
+ - type: v_measure
149
+ value: 35.705350248894476
150
+ - task:
151
+ type: Reranking
152
+ dataset:
153
+ type: mteb/askubuntudupquestions-reranking
154
+ name: MTEB AskUbuntuDupQuestions
155
+ config: default
156
+ split: test
157
+ revision: 2000358ca161889fa9c082cb41daa8dcfb161a54
158
+ metrics:
159
+ - type: map
160
+ value: 63.13335364248881
161
+ - type: mrr
162
+ value: 76.80605021325243
163
+ - task:
164
+ type: STS
165
+ dataset:
166
+ type: mteb/biosses-sts
167
+ name: MTEB BIOSSES
168
+ config: default
169
+ split: test
170
+ revision: d3fb88f8f02e40887cd149695127462bbcf29b4a
171
+ metrics:
172
+ - type: cos_sim_pearson
173
+ value: 83.33741812376516
174
+ - type: cos_sim_spearman
175
+ value: 80.51267790947811
176
+ - type: euclidean_pearson
177
+ value: 67.49002803470997
178
+ - type: euclidean_spearman
179
+ value: 65.39064659674824
180
+ - type: manhattan_pearson
181
+ value: 67.3390206944745
182
+ - type: manhattan_spearman
183
+ value: 65.35329634810715
184
+ - task:
185
+ type: Classification
186
+ dataset:
187
+ type: mteb/banking77
188
+ name: MTEB Banking77Classification
189
+ config: default
190
+ split: test
191
+ revision: 0fd18e25b25c072e09e0d92ab615fda904d66300
192
+ metrics:
193
+ - type: accuracy
194
+ value: 83.13636363636364
195
+ - type: f1
196
+ value: 83.10810612376775
197
+ - task:
198
+ type: Clustering
199
+ dataset:
200
+ type: mteb/biorxiv-clustering-p2p
201
+ name: MTEB BiorxivClusteringP2P
202
+ config: default
203
+ split: test
204
+ revision: 65b79d1d13f80053f67aca9498d9402c2d9f1f40
205
+ metrics:
206
+ - type: v_measure
207
+ value: 38.47849860204599
208
+ - task:
209
+ type: Clustering
210
+ dataset:
211
+ type: mteb/biorxiv-clustering-s2s
212
+ name: MTEB BiorxivClusteringS2S
213
+ config: default
214
+ split: test
215
+ revision: 258694dd0231531bc1fd9de6ceb52a0853c6d908
216
+ metrics:
217
+ - type: v_measure
218
+ value: 31.159196233892057
219
+ - task:
220
+ type: Retrieval
221
+ dataset:
222
+ type: BeIR/cqadupstack
223
+ name: MTEB CQADupstackAndroidRetrieval
224
+ config: default
225
+ split: test
226
+ revision: None
227
+ metrics:
228
+ - type: map_at_1
229
+ value: 34.096
230
+ - type: map_at_10
231
+ value: 46.61
232
+ - type: map_at_100
233
+ value: 48.163
234
+ - type: map_at_1000
235
+ value: 48.272
236
+ - type: map_at_3
237
+ value: 43.03
238
+ - type: map_at_5
239
+ value: 45.036
240
+ - type: mrr_at_1
241
+ value: 42.489
242
+ - type: mrr_at_10
243
+ value: 52.83
244
+ - type: mrr_at_100
245
+ value: 53.525
246
+ - type: mrr_at_1000
247
+ value: 53.561
248
+ - type: mrr_at_3
249
+ value: 50.453
250
+ - type: mrr_at_5
251
+ value: 51.991
252
+ - type: ndcg_at_1
253
+ value: 42.489
254
+ - type: ndcg_at_10
255
+ value: 53.21900000000001
256
+ - type: ndcg_at_100
257
+ value: 58.277
258
+ - type: ndcg_at_1000
259
+ value: 59.836999999999996
260
+ - type: ndcg_at_3
261
+ value: 48.64
262
+ - type: ndcg_at_5
263
+ value: 50.800999999999995
264
+ - type: precision_at_1
265
+ value: 42.489
266
+ - type: precision_at_10
267
+ value: 10.343
268
+ - type: precision_at_100
269
+ value: 1.624
270
+ - type: precision_at_1000
271
+ value: 0.20400000000000001
272
+ - type: precision_at_3
273
+ value: 23.605
274
+ - type: precision_at_5
275
+ value: 16.881
276
+ - type: recall_at_1
277
+ value: 34.096
278
+ - type: recall_at_10
279
+ value: 65.003
280
+ - type: recall_at_100
281
+ value: 86.211
282
+ - type: recall_at_1000
283
+ value: 96.017
284
+ - type: recall_at_3
285
+ value: 51.307
286
+ - type: recall_at_5
287
+ value: 57.873
288
+ - task:
289
+ type: Retrieval
290
+ dataset:
291
+ type: BeIR/cqadupstack
292
+ name: MTEB CQADupstackEnglishRetrieval
293
+ config: default
294
+ split: test
295
+ revision: None
296
+ metrics:
297
+ - type: map_at_1
298
+ value: 29.482000000000003
299
+ - type: map_at_10
300
+ value: 39.793
301
+ - type: map_at_100
302
+ value: 41.028
303
+ - type: map_at_1000
304
+ value: 41.163
305
+ - type: map_at_3
306
+ value: 36.674
307
+ - type: map_at_5
308
+ value: 38.379999999999995
309
+ - type: mrr_at_1
310
+ value: 37.197
311
+ - type: mrr_at_10
312
+ value: 45.991
313
+ - type: mrr_at_100
314
+ value: 46.599000000000004
315
+ - type: mrr_at_1000
316
+ value: 46.649
317
+ - type: mrr_at_3
318
+ value: 43.662
319
+ - type: mrr_at_5
320
+ value: 45.054
321
+ - type: ndcg_at_1
322
+ value: 37.197
323
+ - type: ndcg_at_10
324
+ value: 45.73
325
+ - type: ndcg_at_100
326
+ value: 50.074
327
+ - type: ndcg_at_1000
328
+ value: 52.312000000000005
329
+ - type: ndcg_at_3
330
+ value: 41.308
331
+ - type: ndcg_at_5
332
+ value: 43.323
333
+ - type: precision_at_1
334
+ value: 37.197
335
+ - type: precision_at_10
336
+ value: 8.854
337
+ - type: precision_at_100
338
+ value: 1.411
339
+ - type: precision_at_1000
340
+ value: 0.191
341
+ - type: precision_at_3
342
+ value: 20.085
343
+ - type: precision_at_5
344
+ value: 14.42
345
+ - type: recall_at_1
346
+ value: 29.482000000000003
347
+ - type: recall_at_10
348
+ value: 56.077999999999996
349
+ - type: recall_at_100
350
+ value: 74.83800000000001
351
+ - type: recall_at_1000
352
+ value: 89.128
353
+ - type: recall_at_3
354
+ value: 42.971
355
+ - type: recall_at_5
356
+ value: 48.577
357
+ - task:
358
+ type: Retrieval
359
+ dataset:
360
+ type: BeIR/cqadupstack
361
+ name: MTEB CQADupstackGamingRetrieval
362
+ config: default
363
+ split: test
364
+ revision: None
365
+ metrics:
366
+ - type: map_at_1
367
+ value: 38.679
368
+ - type: map_at_10
369
+ value: 50.854
370
+ - type: map_at_100
371
+ value: 51.849000000000004
372
+ - type: map_at_1000
373
+ value: 51.909000000000006
374
+ - type: map_at_3
375
+ value: 47.82
376
+ - type: map_at_5
377
+ value: 49.479
378
+ - type: mrr_at_1
379
+ value: 44.263000000000005
380
+ - type: mrr_at_10
381
+ value: 54.161
382
+ - type: mrr_at_100
383
+ value: 54.833
384
+ - type: mrr_at_1000
385
+ value: 54.86600000000001
386
+ - type: mrr_at_3
387
+ value: 51.912000000000006
388
+ - type: mrr_at_5
389
+ value: 53.201
390
+ - type: ndcg_at_1
391
+ value: 44.263000000000005
392
+ - type: ndcg_at_10
393
+ value: 56.486000000000004
394
+ - type: ndcg_at_100
395
+ value: 60.553999999999995
396
+ - type: ndcg_at_1000
397
+ value: 61.77
398
+ - type: ndcg_at_3
399
+ value: 51.456999999999994
400
+ - type: ndcg_at_5
401
+ value: 53.83
402
+ - type: precision_at_1
403
+ value: 44.263000000000005
404
+ - type: precision_at_10
405
+ value: 9.041
406
+ - type: precision_at_100
407
+ value: 1.204
408
+ - type: precision_at_1000
409
+ value: 0.135
410
+ - type: precision_at_3
411
+ value: 22.989
412
+ - type: precision_at_5
413
+ value: 15.598999999999998
414
+ - type: recall_at_1
415
+ value: 38.679
416
+ - type: recall_at_10
417
+ value: 69.77799999999999
418
+ - type: recall_at_100
419
+ value: 87.59
420
+ - type: recall_at_1000
421
+ value: 96.202
422
+ - type: recall_at_3
423
+ value: 56.351
424
+ - type: recall_at_5
425
+ value: 62.16199999999999
426
+ - task:
427
+ type: Retrieval
428
+ dataset:
429
+ type: BeIR/cqadupstack
430
+ name: MTEB CQADupstackGisRetrieval
431
+ config: default
432
+ split: test
433
+ revision: None
434
+ metrics:
435
+ - type: map_at_1
436
+ value: 27.245
437
+ - type: map_at_10
438
+ value: 36.104
439
+ - type: map_at_100
440
+ value: 37.207
441
+ - type: map_at_1000
442
+ value: 37.288
443
+ - type: map_at_3
444
+ value: 33.427
445
+ - type: map_at_5
446
+ value: 34.866
447
+ - type: mrr_at_1
448
+ value: 29.604999999999997
449
+ - type: mrr_at_10
450
+ value: 38.346999999999994
451
+ - type: mrr_at_100
452
+ value: 39.274
453
+ - type: mrr_at_1000
454
+ value: 39.336
455
+ - type: mrr_at_3
456
+ value: 35.876000000000005
457
+ - type: mrr_at_5
458
+ value: 37.164
459
+ - type: ndcg_at_1
460
+ value: 29.604999999999997
461
+ - type: ndcg_at_10
462
+ value: 41.253
463
+ - type: ndcg_at_100
464
+ value: 46.511
465
+ - type: ndcg_at_1000
466
+ value: 48.503
467
+ - type: ndcg_at_3
468
+ value: 35.975
469
+ - type: ndcg_at_5
470
+ value: 38.35
471
+ - type: precision_at_1
472
+ value: 29.604999999999997
473
+ - type: precision_at_10
474
+ value: 6.305
475
+ - type: precision_at_100
476
+ value: 0.9440000000000001
477
+ - type: precision_at_1000
478
+ value: 0.11499999999999999
479
+ - type: precision_at_3
480
+ value: 15.179
481
+ - type: precision_at_5
482
+ value: 10.508000000000001
483
+ - type: recall_at_1
484
+ value: 27.245
485
+ - type: recall_at_10
486
+ value: 55.07300000000001
487
+ - type: recall_at_100
488
+ value: 79.036
489
+ - type: recall_at_1000
490
+ value: 93.809
491
+ - type: recall_at_3
492
+ value: 40.593
493
+ - type: recall_at_5
494
+ value: 46.318
495
+ - task:
496
+ type: Retrieval
497
+ dataset:
498
+ type: BeIR/cqadupstack
499
+ name: MTEB CQADupstackMathematicaRetrieval
500
+ config: default
501
+ split: test
502
+ revision: None
503
+ metrics:
504
+ - type: map_at_1
505
+ value: 15.440000000000001
506
+ - type: map_at_10
507
+ value: 23.758000000000003
508
+ - type: map_at_100
509
+ value: 25.1
510
+ - type: map_at_1000
511
+ value: 25.230000000000004
512
+ - type: map_at_3
513
+ value: 21.093
514
+ - type: map_at_5
515
+ value: 22.431
516
+ - type: mrr_at_1
517
+ value: 19.279
518
+ - type: mrr_at_10
519
+ value: 28.077
520
+ - type: mrr_at_100
521
+ value: 29.164
522
+ - type: mrr_at_1000
523
+ value: 29.237000000000002
524
+ - type: mrr_at_3
525
+ value: 25.497999999999998
526
+ - type: mrr_at_5
527
+ value: 26.76
528
+ - type: ndcg_at_1
529
+ value: 19.279
530
+ - type: ndcg_at_10
531
+ value: 29.025000000000002
532
+ - type: ndcg_at_100
533
+ value: 35.244
534
+ - type: ndcg_at_1000
535
+ value: 38.112
536
+ - type: ndcg_at_3
537
+ value: 24.079
538
+ - type: ndcg_at_5
539
+ value: 26.064999999999998
540
+ - type: precision_at_1
541
+ value: 19.279
542
+ - type: precision_at_10
543
+ value: 5.498
544
+ - type: precision_at_100
545
+ value: 0.985
546
+ - type: precision_at_1000
547
+ value: 0.136
548
+ - type: precision_at_3
549
+ value: 11.692
550
+ - type: precision_at_5
551
+ value: 8.383000000000001
552
+ - type: recall_at_1
553
+ value: 15.440000000000001
554
+ - type: recall_at_10
555
+ value: 40.855999999999995
556
+ - type: recall_at_100
557
+ value: 67.916
558
+ - type: recall_at_1000
559
+ value: 88.11
560
+ - type: recall_at_3
561
+ value: 27.387
562
+ - type: recall_at_5
563
+ value: 32.387
564
+ - task:
565
+ type: Retrieval
566
+ dataset:
567
+ type: BeIR/cqadupstack
568
+ name: MTEB CQADupstackPhysicsRetrieval
569
+ config: default
570
+ split: test
571
+ revision: None
572
+ metrics:
573
+ - type: map_at_1
574
+ value: 29.351
575
+ - type: map_at_10
576
+ value: 40.477999999999994
577
+ - type: map_at_100
578
+ value: 41.8
579
+ - type: map_at_1000
580
+ value: 41.926
581
+ - type: map_at_3
582
+ value: 37.246
583
+ - type: map_at_5
584
+ value: 39.206
585
+ - type: mrr_at_1
586
+ value: 36.092
587
+ - type: mrr_at_10
588
+ value: 46.319
589
+ - type: mrr_at_100
590
+ value: 47.087
591
+ - type: mrr_at_1000
592
+ value: 47.13
593
+ - type: mrr_at_3
594
+ value: 43.808
595
+ - type: mrr_at_5
596
+ value: 45.406
597
+ - type: ndcg_at_1
598
+ value: 36.092
599
+ - type: ndcg_at_10
600
+ value: 46.707
601
+ - type: ndcg_at_100
602
+ value: 52.266
603
+ - type: ndcg_at_1000
604
+ value: 54.303000000000004
605
+ - type: ndcg_at_3
606
+ value: 41.858000000000004
607
+ - type: ndcg_at_5
608
+ value: 44.407999999999994
609
+ - type: precision_at_1
610
+ value: 36.092
611
+ - type: precision_at_10
612
+ value: 8.527
613
+ - type: precision_at_100
614
+ value: 1.34
615
+ - type: precision_at_1000
616
+ value: 0.172
617
+ - type: precision_at_3
618
+ value: 20.212
619
+ - type: precision_at_5
620
+ value: 14.456
621
+ - type: recall_at_1
622
+ value: 29.351
623
+ - type: recall_at_10
624
+ value: 59.254
625
+ - type: recall_at_100
626
+ value: 83.047
627
+ - type: recall_at_1000
628
+ value: 95.911
629
+ - type: recall_at_3
630
+ value: 45.488
631
+ - type: recall_at_5
632
+ value: 52.186
633
+ - task:
634
+ type: Retrieval
635
+ dataset:
636
+ type: BeIR/cqadupstack
637
+ name: MTEB CQADupstackProgrammersRetrieval
638
+ config: default
639
+ split: test
640
+ revision: None
641
+ metrics:
642
+ - type: map_at_1
643
+ value: 25.601000000000003
644
+ - type: map_at_10
645
+ value: 34.589999999999996
646
+ - type: map_at_100
647
+ value: 35.917
648
+ - type: map_at_1000
649
+ value: 36.032
650
+ - type: map_at_3
651
+ value: 31.338
652
+ - type: map_at_5
653
+ value: 33.128
654
+ - type: mrr_at_1
655
+ value: 31.163999999999998
656
+ - type: mrr_at_10
657
+ value: 39.646
658
+ - type: mrr_at_100
659
+ value: 40.491
660
+ - type: mrr_at_1000
661
+ value: 40.549
662
+ - type: mrr_at_3
663
+ value: 36.91
664
+ - type: mrr_at_5
665
+ value: 38.446000000000005
666
+ - type: ndcg_at_1
667
+ value: 31.163999999999998
668
+ - type: ndcg_at_10
669
+ value: 40.321
670
+ - type: ndcg_at_100
671
+ value: 45.894
672
+ - type: ndcg_at_1000
673
+ value: 48.233
674
+ - type: ndcg_at_3
675
+ value: 34.871
676
+ - type: ndcg_at_5
677
+ value: 37.302
678
+ - type: precision_at_1
679
+ value: 31.163999999999998
680
+ - type: precision_at_10
681
+ value: 7.523000000000001
682
+ - type: precision_at_100
683
+ value: 1.188
684
+ - type: precision_at_1000
685
+ value: 0.157
686
+ - type: precision_at_3
687
+ value: 16.591
688
+ - type: precision_at_5
689
+ value: 12.055
690
+ - type: recall_at_1
691
+ value: 25.601000000000003
692
+ - type: recall_at_10
693
+ value: 52.422000000000004
694
+ - type: recall_at_100
695
+ value: 76.426
696
+ - type: recall_at_1000
697
+ value: 92.142
698
+ - type: recall_at_3
699
+ value: 37.141000000000005
700
+ - type: recall_at_5
701
+ value: 43.449
702
+ - task:
703
+ type: Retrieval
704
+ dataset:
705
+ type: BeIR/cqadupstack
706
+ name: MTEB CQADupstackRetrieval
707
+ config: default
708
+ split: test
709
+ revision: None
710
+ metrics:
711
+ - type: map_at_1
712
+ value: 26.267916666666668
713
+ - type: map_at_10
714
+ value: 35.758250000000004
715
+ - type: map_at_100
716
+ value: 37.0185
717
+ - type: map_at_1000
718
+ value: 37.136916666666664
719
+ - type: map_at_3
720
+ value: 32.85125
721
+ - type: map_at_5
722
+ value: 34.4165
723
+ - type: mrr_at_1
724
+ value: 31.131083333333343
725
+ - type: mrr_at_10
726
+ value: 39.95941666666667
727
+ - type: mrr_at_100
728
+ value: 40.81541666666666
729
+ - type: mrr_at_1000
730
+ value: 40.87358333333332
731
+ - type: mrr_at_3
732
+ value: 37.5175
733
+ - type: mrr_at_5
734
+ value: 38.86833333333334
735
+ - type: ndcg_at_1
736
+ value: 31.131083333333343
737
+ - type: ndcg_at_10
738
+ value: 41.26174999999999
739
+ - type: ndcg_at_100
740
+ value: 46.55975
741
+ - type: ndcg_at_1000
742
+ value: 48.80016666666666
743
+ - type: ndcg_at_3
744
+ value: 36.37566666666667
745
+ - type: ndcg_at_5
746
+ value: 38.55166666666667
747
+ - type: precision_at_1
748
+ value: 31.131083333333343
749
+ - type: precision_at_10
750
+ value: 7.315916666666666
751
+ - type: precision_at_100
752
+ value: 1.1813333333333333
753
+ - type: precision_at_1000
754
+ value: 0.15666666666666665
755
+ - type: precision_at_3
756
+ value: 16.818166666666663
757
+ - type: precision_at_5
758
+ value: 11.923
759
+ - type: recall_at_1
760
+ value: 26.267916666666668
761
+ - type: recall_at_10
762
+ value: 53.28391666666666
763
+ - type: recall_at_100
764
+ value: 76.53983333333332
765
+ - type: recall_at_1000
766
+ value: 91.93008333333334
767
+ - type: recall_at_3
768
+ value: 39.60583333333334
769
+ - type: recall_at_5
770
+ value: 45.25741666666667
771
+ - task:
772
+ type: Retrieval
773
+ dataset:
774
+ type: BeIR/cqadupstack
775
+ name: MTEB CQADupstackStatsRetrieval
776
+ config: default
777
+ split: test
778
+ revision: None
779
+ metrics:
780
+ - type: map_at_1
781
+ value: 23.372
782
+ - type: map_at_10
783
+ value: 30.916
784
+ - type: map_at_100
785
+ value: 31.980999999999998
786
+ - type: map_at_1000
787
+ value: 32.07
788
+ - type: map_at_3
789
+ value: 28.778
790
+ - type: map_at_5
791
+ value: 29.872
792
+ - type: mrr_at_1
793
+ value: 26.074
794
+ - type: mrr_at_10
795
+ value: 33.451
796
+ - type: mrr_at_100
797
+ value: 34.366
798
+ - type: mrr_at_1000
799
+ value: 34.424
800
+ - type: mrr_at_3
801
+ value: 31.569999999999997
802
+ - type: mrr_at_5
803
+ value: 32.467
804
+ - type: ndcg_at_1
805
+ value: 26.074
806
+ - type: ndcg_at_10
807
+ value: 35.119
808
+ - type: ndcg_at_100
809
+ value: 40.357
810
+ - type: ndcg_at_1000
811
+ value: 42.548
812
+ - type: ndcg_at_3
813
+ value: 31.281
814
+ - type: ndcg_at_5
815
+ value: 32.866
816
+ - type: precision_at_1
817
+ value: 26.074
818
+ - type: precision_at_10
819
+ value: 5.583
820
+ - type: precision_at_100
821
+ value: 0.899
822
+ - type: precision_at_1000
823
+ value: 0.116
824
+ - type: precision_at_3
825
+ value: 13.700999999999999
826
+ - type: precision_at_5
827
+ value: 9.447999999999999
828
+ - type: recall_at_1
829
+ value: 23.372
830
+ - type: recall_at_10
831
+ value: 45.396
832
+ - type: recall_at_100
833
+ value: 69.26
834
+ - type: recall_at_1000
835
+ value: 85.438
836
+ - type: recall_at_3
837
+ value: 34.373
838
+ - type: recall_at_5
839
+ value: 38.509
840
+ - task:
841
+ type: Retrieval
842
+ dataset:
843
+ type: BeIR/cqadupstack
844
+ name: MTEB CQADupstackTexRetrieval
845
+ config: default
846
+ split: test
847
+ revision: None
848
+ metrics:
849
+ - type: map_at_1
850
+ value: 17.483999999999998
851
+ - type: map_at_10
852
+ value: 25.191999999999997
853
+ - type: map_at_100
854
+ value: 26.432
855
+ - type: map_at_1000
856
+ value: 26.566000000000003
857
+ - type: map_at_3
858
+ value: 22.697
859
+ - type: map_at_5
860
+ value: 24.101
861
+ - type: mrr_at_1
862
+ value: 21.645
863
+ - type: mrr_at_10
864
+ value: 29.243000000000002
865
+ - type: mrr_at_100
866
+ value: 30.232
867
+ - type: mrr_at_1000
868
+ value: 30.312
869
+ - type: mrr_at_3
870
+ value: 26.967000000000002
871
+ - type: mrr_at_5
872
+ value: 28.262999999999998
873
+ - type: ndcg_at_1
874
+ value: 21.645
875
+ - type: ndcg_at_10
876
+ value: 30.087999999999997
877
+ - type: ndcg_at_100
878
+ value: 35.806
879
+ - type: ndcg_at_1000
880
+ value: 38.763
881
+ - type: ndcg_at_3
882
+ value: 25.746999999999996
883
+ - type: ndcg_at_5
884
+ value: 27.765
885
+ - type: precision_at_1
886
+ value: 21.645
887
+ - type: precision_at_10
888
+ value: 5.6129999999999995
889
+ - type: precision_at_100
890
+ value: 1.004
891
+ - type: precision_at_1000
892
+ value: 0.14400000000000002
893
+ - type: precision_at_3
894
+ value: 12.331
895
+ - type: precision_at_5
896
+ value: 9.009
897
+ - type: recall_at_1
898
+ value: 17.483999999999998
899
+ - type: recall_at_10
900
+ value: 40.723
901
+ - type: recall_at_100
902
+ value: 66.226
903
+ - type: recall_at_1000
904
+ value: 87.312
905
+ - type: recall_at_3
906
+ value: 28.481
907
+ - type: recall_at_5
908
+ value: 33.777
909
+ - task:
910
+ type: Retrieval
911
+ dataset:
912
+ type: BeIR/cqadupstack
913
+ name: MTEB CQADupstackUnixRetrieval
914
+ config: default
915
+ split: test
916
+ revision: None
917
+ metrics:
918
+ - type: map_at_1
919
+ value: 26.735
920
+ - type: map_at_10
921
+ value: 36.431000000000004
922
+ - type: map_at_100
923
+ value: 37.696000000000005
924
+ - type: map_at_1000
925
+ value: 37.793
926
+ - type: map_at_3
927
+ value: 33.416000000000004
928
+ - type: map_at_5
929
+ value: 34.934
930
+ - type: mrr_at_1
931
+ value: 31.25
932
+ - type: mrr_at_10
933
+ value: 40.516000000000005
934
+ - type: mrr_at_100
935
+ value: 41.392
936
+ - type: mrr_at_1000
937
+ value: 41.449000000000005
938
+ - type: mrr_at_3
939
+ value: 37.842
940
+ - type: mrr_at_5
941
+ value: 39.265
942
+ - type: ndcg_at_1
943
+ value: 31.25
944
+ - type: ndcg_at_10
945
+ value: 42.191
946
+ - type: ndcg_at_100
947
+ value: 47.683
948
+ - type: ndcg_at_1000
949
+ value: 49.815
950
+ - type: ndcg_at_3
951
+ value: 36.744
952
+ - type: ndcg_at_5
953
+ value: 39.007
954
+ - type: precision_at_1
955
+ value: 31.25
956
+ - type: precision_at_10
957
+ value: 7.276000000000001
958
+ - type: precision_at_100
959
+ value: 1.125
960
+ - type: precision_at_1000
961
+ value: 0.14100000000000001
962
+ - type: precision_at_3
963
+ value: 16.76
964
+ - type: precision_at_5
965
+ value: 11.791
966
+ - type: recall_at_1
967
+ value: 26.735
968
+ - type: recall_at_10
969
+ value: 55.444
970
+ - type: recall_at_100
971
+ value: 79.098
972
+ - type: recall_at_1000
973
+ value: 93.815
974
+ - type: recall_at_3
975
+ value: 40.623
976
+ - type: recall_at_5
977
+ value: 46.322
978
+ - task:
979
+ type: Retrieval
980
+ dataset:
981
+ type: BeIR/cqadupstack
982
+ name: MTEB CQADupstackWebmastersRetrieval
983
+ config: default
984
+ split: test
985
+ revision: None
986
+ metrics:
987
+ - type: map_at_1
988
+ value: 26.495
989
+ - type: map_at_10
990
+ value: 35.648
991
+ - type: map_at_100
992
+ value: 37.275000000000006
993
+ - type: map_at_1000
994
+ value: 37.494
995
+ - type: map_at_3
996
+ value: 32.446999999999996
997
+ - type: map_at_5
998
+ value: 34.233000000000004
999
+ - type: mrr_at_1
1000
+ value: 31.225
1001
+ - type: mrr_at_10
1002
+ value: 40.127
1003
+ - type: mrr_at_100
1004
+ value: 41.092
1005
+ - type: mrr_at_1000
1006
+ value: 41.148
1007
+ - type: mrr_at_3
1008
+ value: 37.153999999999996
1009
+ - type: mrr_at_5
1010
+ value: 38.873999999999995
1011
+ - type: ndcg_at_1
1012
+ value: 31.225
1013
+ - type: ndcg_at_10
1014
+ value: 41.665
1015
+ - type: ndcg_at_100
1016
+ value: 47.557
1017
+ - type: ndcg_at_1000
1018
+ value: 49.992
1019
+ - type: ndcg_at_3
1020
+ value: 36.114000000000004
1021
+ - type: ndcg_at_5
1022
+ value: 38.675
1023
+ - type: precision_at_1
1024
+ value: 31.225
1025
+ - type: precision_at_10
1026
+ value: 7.904999999999999
1027
+ - type: precision_at_100
1028
+ value: 1.5890000000000002
1029
+ - type: precision_at_1000
1030
+ value: 0.246
1031
+ - type: precision_at_3
1032
+ value: 16.535
1033
+ - type: precision_at_5
1034
+ value: 12.134
1035
+ - type: recall_at_1
1036
+ value: 26.495
1037
+ - type: recall_at_10
1038
+ value: 53.727000000000004
1039
+ - type: recall_at_100
1040
+ value: 79.34400000000001
1041
+ - type: recall_at_1000
1042
+ value: 94.35900000000001
1043
+ - type: recall_at_3
1044
+ value: 38.432
1045
+ - type: recall_at_5
1046
+ value: 45.050000000000004
1047
+ - task:
1048
+ type: Retrieval
1049
+ dataset:
1050
+ type: BeIR/cqadupstack
1051
+ name: MTEB CQADupstackWordpressRetrieval
1052
+ config: default
1053
+ split: test
1054
+ revision: None
1055
+ metrics:
1056
+ - type: map_at_1
1057
+ value: 21.235
1058
+ - type: map_at_10
1059
+ value: 28.725
1060
+ - type: map_at_100
1061
+ value: 29.774
1062
+ - type: map_at_1000
1063
+ value: 29.9
1064
+ - type: map_at_3
1065
+ value: 26.249
1066
+ - type: map_at_5
1067
+ value: 27.332
1068
+ - type: mrr_at_1
1069
+ value: 23.29
1070
+ - type: mrr_at_10
1071
+ value: 30.805
1072
+ - type: mrr_at_100
1073
+ value: 31.730000000000004
1074
+ - type: mrr_at_1000
1075
+ value: 31.822
1076
+ - type: mrr_at_3
1077
+ value: 28.558
1078
+ - type: mrr_at_5
1079
+ value: 29.529
1080
+ - type: ndcg_at_1
1081
+ value: 23.29
1082
+ - type: ndcg_at_10
1083
+ value: 33.337
1084
+ - type: ndcg_at_100
1085
+ value: 38.494
1086
+ - type: ndcg_at_1000
1087
+ value: 41.414
1088
+ - type: ndcg_at_3
1089
+ value: 28.433999999999997
1090
+ - type: ndcg_at_5
1091
+ value: 30.227999999999998
1092
+ - type: precision_at_1
1093
+ value: 23.29
1094
+ - type: precision_at_10
1095
+ value: 5.323
1096
+ - type: precision_at_100
1097
+ value: 0.8630000000000001
1098
+ - type: precision_at_1000
1099
+ value: 0.123
1100
+ - type: precision_at_3
1101
+ value: 12.138
1102
+ - type: precision_at_5
1103
+ value: 8.392
1104
+ - type: recall_at_1
1105
+ value: 21.235
1106
+ - type: recall_at_10
1107
+ value: 45.653
1108
+ - type: recall_at_100
1109
+ value: 69.486
1110
+ - type: recall_at_1000
1111
+ value: 90.91799999999999
1112
+ - type: recall_at_3
1113
+ value: 32.123000000000005
1114
+ - type: recall_at_5
1115
+ value: 36.479
1116
+ - task:
1117
+ type: Retrieval
1118
+ dataset:
1119
+ type: climate-fever
1120
+ name: MTEB ClimateFEVER
1121
+ config: default
1122
+ split: test
1123
+ revision: None
1124
+ metrics:
1125
+ - type: map_at_1
1126
+ value: 9.180000000000001
1127
+ - type: map_at_10
1128
+ value: 16.461000000000002
1129
+ - type: map_at_100
1130
+ value: 18.093999999999998
1131
+ - type: map_at_1000
1132
+ value: 18.297
1133
+ - type: map_at_3
1134
+ value: 13.475000000000001
1135
+ - type: map_at_5
1136
+ value: 15.02
1137
+ - type: mrr_at_1
1138
+ value: 21.303
1139
+ - type: mrr_at_10
1140
+ value: 31.755
1141
+ - type: mrr_at_100
1142
+ value: 32.826
1143
+ - type: mrr_at_1000
1144
+ value: 32.873000000000005
1145
+ - type: mrr_at_3
1146
+ value: 28.469
1147
+ - type: mrr_at_5
1148
+ value: 30.325999999999997
1149
+ - type: ndcg_at_1
1150
+ value: 21.303
1151
+ - type: ndcg_at_10
1152
+ value: 23.892
1153
+ - type: ndcg_at_100
1154
+ value: 30.848
1155
+ - type: ndcg_at_1000
1156
+ value: 34.577999999999996
1157
+ - type: ndcg_at_3
1158
+ value: 18.88
1159
+ - type: ndcg_at_5
1160
+ value: 20.683
1161
+ - type: precision_at_1
1162
+ value: 21.303
1163
+ - type: precision_at_10
1164
+ value: 7.693999999999999
1165
+ - type: precision_at_100
1166
+ value: 1.517
1167
+ - type: precision_at_1000
1168
+ value: 0.22
1169
+ - type: precision_at_3
1170
+ value: 14.180000000000001
1171
+ - type: precision_at_5
1172
+ value: 11.231
1173
+ - type: recall_at_1
1174
+ value: 9.180000000000001
1175
+ - type: recall_at_10
1176
+ value: 29.813000000000002
1177
+ - type: recall_at_100
1178
+ value: 54.116
1179
+ - type: recall_at_1000
1180
+ value: 75.248
1181
+ - type: recall_at_3
1182
+ value: 17.684
1183
+ - type: recall_at_5
1184
+ value: 22.557
1185
+ - task:
1186
+ type: Retrieval
1187
+ dataset:
1188
+ type: dbpedia-entity
1189
+ name: MTEB DBPedia
1190
+ config: default
1191
+ split: test
1192
+ revision: None
1193
+ metrics:
1194
+ - type: map_at_1
1195
+ value: 8.508000000000001
1196
+ - type: map_at_10
1197
+ value: 16.39
1198
+ - type: map_at_100
1199
+ value: 21.981
1200
+ - type: map_at_1000
1201
+ value: 23.253
1202
+ - type: map_at_3
1203
+ value: 12.465
1204
+ - type: map_at_5
1205
+ value: 14.194999999999999
1206
+ - type: mrr_at_1
1207
+ value: 60.0
1208
+ - type: mrr_at_10
1209
+ value: 68.499
1210
+ - type: mrr_at_100
1211
+ value: 69.014
1212
+ - type: mrr_at_1000
1213
+ value: 69.024
1214
+ - type: mrr_at_3
1215
+ value: 66.625
1216
+ - type: mrr_at_5
1217
+ value: 67.887
1218
+ - type: ndcg_at_1
1219
+ value: 48.5
1220
+ - type: ndcg_at_10
1221
+ value: 34.870000000000005
1222
+ - type: ndcg_at_100
1223
+ value: 38.448
1224
+ - type: ndcg_at_1000
1225
+ value: 45.668
1226
+ - type: ndcg_at_3
1227
+ value: 39.931
1228
+ - type: ndcg_at_5
1229
+ value: 37.007
1230
+ - type: precision_at_1
1231
+ value: 60.0
1232
+ - type: precision_at_10
1233
+ value: 26.924999999999997
1234
+ - type: precision_at_100
1235
+ value: 8.358
1236
+ - type: precision_at_1000
1237
+ value: 1.7850000000000001
1238
+ - type: precision_at_3
1239
+ value: 43.0
1240
+ - type: precision_at_5
1241
+ value: 35.449999999999996
1242
+ - type: recall_at_1
1243
+ value: 8.508000000000001
1244
+ - type: recall_at_10
1245
+ value: 21.089
1246
+ - type: recall_at_100
1247
+ value: 43.146
1248
+ - type: recall_at_1000
1249
+ value: 66.776
1250
+ - type: recall_at_3
1251
+ value: 13.33
1252
+ - type: recall_at_5
1253
+ value: 16.225
1254
+ - task:
1255
+ type: Classification
1256
+ dataset:
1257
+ type: mteb/emotion
1258
+ name: MTEB EmotionClassification
1259
+ config: default
1260
+ split: test
1261
+ revision: 4f58c6b202a23cf9a4da393831edf4f9183cad37
1262
+ metrics:
1263
+ - type: accuracy
1264
+ value: 46.735
1265
+ - type: f1
1266
+ value: 42.30853263256299
1267
+ - task:
1268
+ type: Retrieval
1269
+ dataset:
1270
+ type: fever
1271
+ name: MTEB FEVER
1272
+ config: default
1273
+ split: test
1274
+ revision: None
1275
+ metrics:
1276
+ - type: map_at_1
1277
+ value: 54.54
1278
+ - type: map_at_10
1279
+ value: 65.24600000000001
1280
+ - type: map_at_100
1281
+ value: 65.69
1282
+ - type: map_at_1000
1283
+ value: 65.71000000000001
1284
+ - type: map_at_3
1285
+ value: 63.234
1286
+ - type: map_at_5
1287
+ value: 64.455
1288
+ - type: mrr_at_1
1289
+ value: 58.821
1290
+ - type: mrr_at_10
1291
+ value: 69.616
1292
+ - type: mrr_at_100
1293
+ value: 69.98
1294
+ - type: mrr_at_1000
1295
+ value: 69.992
1296
+ - type: mrr_at_3
1297
+ value: 67.782
1298
+ - type: mrr_at_5
1299
+ value: 68.917
1300
+ - type: ndcg_at_1
1301
+ value: 58.821
1302
+ - type: ndcg_at_10
1303
+ value: 70.798
1304
+ - type: ndcg_at_100
1305
+ value: 72.719
1306
+ - type: ndcg_at_1000
1307
+ value: 73.19600000000001
1308
+ - type: ndcg_at_3
1309
+ value: 67.037
1310
+ - type: ndcg_at_5
1311
+ value: 69.048
1312
+ - type: precision_at_1
1313
+ value: 58.821
1314
+ - type: precision_at_10
1315
+ value: 9.182
1316
+ - type: precision_at_100
1317
+ value: 1.024
1318
+ - type: precision_at_1000
1319
+ value: 0.108
1320
+ - type: precision_at_3
1321
+ value: 26.662999999999997
1322
+ - type: precision_at_5
1323
+ value: 17.159
1324
+ - type: recall_at_1
1325
+ value: 54.54
1326
+ - type: recall_at_10
1327
+ value: 83.67999999999999
1328
+ - type: recall_at_100
1329
+ value: 92.099
1330
+ - type: recall_at_1000
1331
+ value: 95.532
1332
+ - type: recall_at_3
1333
+ value: 73.478
1334
+ - type: recall_at_5
1335
+ value: 78.424
1336
+ - task:
1337
+ type: Retrieval
1338
+ dataset:
1339
+ type: fiqa
1340
+ name: MTEB FiQA2018
1341
+ config: default
1342
+ split: test
1343
+ revision: None
1344
+ metrics:
1345
+ - type: map_at_1
1346
+ value: 17.601
1347
+ - type: map_at_10
1348
+ value: 28.676000000000002
1349
+ - type: map_at_100
1350
+ value: 30.463
1351
+ - type: map_at_1000
1352
+ value: 30.666
1353
+ - type: map_at_3
1354
+ value: 24.734
1355
+ - type: map_at_5
1356
+ value: 27.026
1357
+ - type: mrr_at_1
1358
+ value: 34.259
1359
+ - type: mrr_at_10
1360
+ value: 43.613
1361
+ - type: mrr_at_100
1362
+ value: 44.535000000000004
1363
+ - type: mrr_at_1000
1364
+ value: 44.583
1365
+ - type: mrr_at_3
1366
+ value: 41.307
1367
+ - type: mrr_at_5
1368
+ value: 42.626
1369
+ - type: ndcg_at_1
1370
+ value: 34.259
1371
+ - type: ndcg_at_10
1372
+ value: 36.097
1373
+ - type: ndcg_at_100
1374
+ value: 43.039
1375
+ - type: ndcg_at_1000
1376
+ value: 46.498
1377
+ - type: ndcg_at_3
1378
+ value: 32.244
1379
+ - type: ndcg_at_5
1380
+ value: 33.711999999999996
1381
+ - type: precision_at_1
1382
+ value: 34.259
1383
+ - type: precision_at_10
1384
+ value: 10.030999999999999
1385
+ - type: precision_at_100
1386
+ value: 1.7239999999999998
1387
+ - type: precision_at_1000
1388
+ value: 0.234
1389
+ - type: precision_at_3
1390
+ value: 21.193
1391
+ - type: precision_at_5
1392
+ value: 15.956999999999999
1393
+ - type: recall_at_1
1394
+ value: 17.601
1395
+ - type: recall_at_10
1396
+ value: 42.807
1397
+ - type: recall_at_100
1398
+ value: 68.571
1399
+ - type: recall_at_1000
1400
+ value: 89.237
1401
+ - type: recall_at_3
1402
+ value: 29.301
1403
+ - type: recall_at_5
1404
+ value: 35.528999999999996
1405
+ - task:
1406
+ type: Retrieval
1407
+ dataset:
1408
+ type: hotpotqa
1409
+ name: MTEB HotpotQA
1410
+ config: default
1411
+ split: test
1412
+ revision: None
1413
+ metrics:
1414
+ - type: map_at_1
1415
+ value: 31.182
1416
+ - type: map_at_10
1417
+ value: 42.631
1418
+ - type: map_at_100
1419
+ value: 43.577
1420
+ - type: map_at_1000
1421
+ value: 43.661
1422
+ - type: map_at_3
1423
+ value: 40.06
1424
+ - type: map_at_5
1425
+ value: 41.591
1426
+ - type: mrr_at_1
1427
+ value: 62.363
1428
+ - type: mrr_at_10
1429
+ value: 69.047
1430
+ - type: mrr_at_100
1431
+ value: 69.46
1432
+ - type: mrr_at_1000
1433
+ value: 69.48100000000001
1434
+ - type: mrr_at_3
1435
+ value: 67.574
1436
+ - type: mrr_at_5
1437
+ value: 68.487
1438
+ - type: ndcg_at_1
1439
+ value: 62.363
1440
+ - type: ndcg_at_10
1441
+ value: 51.629999999999995
1442
+ - type: ndcg_at_100
1443
+ value: 55.301
1444
+ - type: ndcg_at_1000
1445
+ value: 57.071000000000005
1446
+ - type: ndcg_at_3
1447
+ value: 47.496
1448
+ - type: ndcg_at_5
1449
+ value: 49.687
1450
+ - type: precision_at_1
1451
+ value: 62.363
1452
+ - type: precision_at_10
1453
+ value: 10.628
1454
+ - type: precision_at_100
1455
+ value: 1.352
1456
+ - type: precision_at_1000
1457
+ value: 0.159
1458
+ - type: precision_at_3
1459
+ value: 29.296
1460
+ - type: precision_at_5
1461
+ value: 19.309
1462
+ - type: recall_at_1
1463
+ value: 31.182
1464
+ - type: recall_at_10
1465
+ value: 53.14
1466
+ - type: recall_at_100
1467
+ value: 67.596
1468
+ - type: recall_at_1000
1469
+ value: 79.372
1470
+ - type: recall_at_3
1471
+ value: 43.943
1472
+ - type: recall_at_5
1473
+ value: 48.271
1474
+ - task:
1475
+ type: Classification
1476
+ dataset:
1477
+ type: mteb/imdb
1478
+ name: MTEB ImdbClassification
1479
+ config: default
1480
+ split: test
1481
+ revision: 3d86128a09e091d6018b6d26cad27f2739fc2db7
1482
+ metrics:
1483
+ - type: accuracy
1484
+ value: 71.55319999999999
1485
+ - type: ap
1486
+ value: 65.44170899953346
1487
+ - type: f1
1488
+ value: 71.33420141354401
1489
+ - task:
1490
+ type: Retrieval
1491
+ dataset:
1492
+ type: msmarco
1493
+ name: MTEB MSMARCO
1494
+ config: default
1495
+ split: dev
1496
+ revision: None
1497
+ metrics:
1498
+ - type: map_at_1
1499
+ value: 18.89
1500
+ - type: map_at_10
1501
+ value: 30.076999999999998
1502
+ - type: map_at_100
1503
+ value: 31.281
1504
+ - type: map_at_1000
1505
+ value: 31.341
1506
+ - type: map_at_3
1507
+ value: 26.391
1508
+ - type: map_at_5
1509
+ value: 28.557
1510
+ - type: mrr_at_1
1511
+ value: 19.312
1512
+ - type: mrr_at_10
1513
+ value: 30.566
1514
+ - type: mrr_at_100
1515
+ value: 31.728
1516
+ - type: mrr_at_1000
1517
+ value: 31.781
1518
+ - type: mrr_at_3
1519
+ value: 26.901000000000003
1520
+ - type: mrr_at_5
1521
+ value: 29.072
1522
+ - type: ndcg_at_1
1523
+ value: 19.326999999999998
1524
+ - type: ndcg_at_10
1525
+ value: 36.516999999999996
1526
+ - type: ndcg_at_100
1527
+ value: 42.458
1528
+ - type: ndcg_at_1000
1529
+ value: 43.99
1530
+ - type: ndcg_at_3
1531
+ value: 29.005
1532
+ - type: ndcg_at_5
1533
+ value: 32.889
1534
+ - type: precision_at_1
1535
+ value: 19.326999999999998
1536
+ - type: precision_at_10
1537
+ value: 5.868
1538
+ - type: precision_at_100
1539
+ value: 0.8880000000000001
1540
+ - type: precision_at_1000
1541
+ value: 0.10200000000000001
1542
+ - type: precision_at_3
1543
+ value: 12.388
1544
+ - type: precision_at_5
1545
+ value: 9.401
1546
+ - type: recall_at_1
1547
+ value: 18.89
1548
+ - type: recall_at_10
1549
+ value: 56.442
1550
+ - type: recall_at_100
1551
+ value: 84.16
1552
+ - type: recall_at_1000
1553
+ value: 95.97099999999999
1554
+ - type: recall_at_3
1555
+ value: 36.077999999999996
1556
+ - type: recall_at_5
1557
+ value: 45.395
1558
+ - task:
1559
+ type: Classification
1560
+ dataset:
1561
+ type: mteb/mtop_domain
1562
+ name: MTEB MTOPDomainClassification (en)
1563
+ config: en
1564
+ split: test
1565
+ revision: d80d48c1eb48d3562165c59d59d0034df9fff0bf
1566
+ metrics:
1567
+ - type: accuracy
1568
+ value: 93.69585043319653
1569
+ - type: f1
1570
+ value: 93.27706251110098
1571
+ - task:
1572
+ type: Classification
1573
+ dataset:
1574
+ type: mteb/mtop_intent
1575
+ name: MTEB MTOPIntentClassification (en)
1576
+ config: en
1577
+ split: test
1578
+ revision: ae001d0e6b1228650b7bd1c2c65fb50ad11a8aba
1579
+ metrics:
1580
+ - type: accuracy
1581
+ value: 74.62836297309622
1582
+ - type: f1
1583
+ value: 56.21163652384411
1584
+ - task:
1585
+ type: Classification
1586
+ dataset:
1587
+ type: mteb/amazon_massive_intent
1588
+ name: MTEB MassiveIntentClassification (en)
1589
+ config: en
1590
+ split: test
1591
+ revision: 31efe3c427b0bae9c22cbb560b8f15491cc6bed7
1592
+ metrics:
1593
+ - type: accuracy
1594
+ value: 71.37861466039006
1595
+ - type: f1
1596
+ value: 69.85338860172736
1597
+ - task:
1598
+ type: Classification
1599
+ dataset:
1600
+ type: mteb/amazon_massive_scenario
1601
+ name: MTEB MassiveScenarioClassification (en)
1602
+ config: en
1603
+ split: test
1604
+ revision: 7d571f92784cd94a019292a1f45445077d0ef634
1605
+ metrics:
1606
+ - type: accuracy
1607
+ value: 75.58170813718897
1608
+ - type: f1
1609
+ value: 75.77358464349743
1610
+ - task:
1611
+ type: Clustering
1612
+ dataset:
1613
+ type: mteb/medrxiv-clustering-p2p
1614
+ name: MTEB MedrxivClusteringP2P
1615
+ config: default
1616
+ split: test
1617
+ revision: e7a26af6f3ae46b30dde8737f02c07b1505bcc73
1618
+ metrics:
1619
+ - type: v_measure
1620
+ value: 33.29659845527655
1621
+ - task:
1622
+ type: Clustering
1623
+ dataset:
1624
+ type: mteb/medrxiv-clustering-s2s
1625
+ name: MTEB MedrxivClusteringS2S
1626
+ config: default
1627
+ split: test
1628
+ revision: 35191c8c0dca72d8ff3efcd72aa802307d469663
1629
+ metrics:
1630
+ - type: v_measure
1631
+ value: 29.97507851301835
1632
+ - task:
1633
+ type: Reranking
1634
+ dataset:
1635
+ type: mteb/mind_small
1636
+ name: MTEB MindSmallReranking
1637
+ config: default
1638
+ split: test
1639
+ revision: 3bdac13927fdc888b903db93b2ffdbd90b295a69
1640
+ metrics:
1641
+ - type: map
1642
+ value: 31.158968289313076
1643
+ - type: mrr
1644
+ value: 32.27027446726339
1645
+ - task:
1646
+ type: Retrieval
1647
+ dataset:
1648
+ type: nfcorpus
1649
+ name: MTEB NFCorpus
1650
+ config: default
1651
+ split: test
1652
+ revision: None
1653
+ metrics:
1654
+ - type: map_at_1
1655
+ value: 5.021
1656
+ - type: map_at_10
1657
+ value: 11.346
1658
+ - type: map_at_100
1659
+ value: 14.457
1660
+ - type: map_at_1000
1661
+ value: 15.875
1662
+ - type: map_at_3
1663
+ value: 8.376999999999999
1664
+ - type: map_at_5
1665
+ value: 9.793000000000001
1666
+ - type: mrr_at_1
1667
+ value: 43.344
1668
+ - type: mrr_at_10
1669
+ value: 51.266
1670
+ - type: mrr_at_100
1671
+ value: 51.871
1672
+ - type: mrr_at_1000
1673
+ value: 51.915
1674
+ - type: mrr_at_3
1675
+ value: 49.174
1676
+ - type: mrr_at_5
1677
+ value: 50.475
1678
+ - type: ndcg_at_1
1679
+ value: 41.331
1680
+ - type: ndcg_at_10
1681
+ value: 31.257
1682
+ - type: ndcg_at_100
1683
+ value: 29.264000000000003
1684
+ - type: ndcg_at_1000
1685
+ value: 38.024
1686
+ - type: ndcg_at_3
1687
+ value: 36.643
1688
+ - type: ndcg_at_5
1689
+ value: 34.808
1690
+ - type: precision_at_1
1691
+ value: 43.034
1692
+ - type: precision_at_10
1693
+ value: 22.972
1694
+ - type: precision_at_100
1695
+ value: 7.576
1696
+ - type: precision_at_1000
1697
+ value: 2.0629999999999997
1698
+ - type: precision_at_3
1699
+ value: 34.572
1700
+ - type: precision_at_5
1701
+ value: 30.341
1702
+ - type: recall_at_1
1703
+ value: 5.021
1704
+ - type: recall_at_10
1705
+ value: 15.197
1706
+ - type: recall_at_100
1707
+ value: 30.874000000000002
1708
+ - type: recall_at_1000
1709
+ value: 61.934
1710
+ - type: recall_at_3
1711
+ value: 9.467
1712
+ - type: recall_at_5
1713
+ value: 11.904
1714
+ - task:
1715
+ type: Retrieval
1716
+ dataset:
1717
+ type: nq
1718
+ name: MTEB NQ
1719
+ config: default
1720
+ split: test
1721
+ revision: None
1722
+ metrics:
1723
+ - type: map_at_1
1724
+ value: 24.468999999999998
1725
+ - type: map_at_10
1726
+ value: 38.885999999999996
1727
+ - type: map_at_100
1728
+ value: 40.154
1729
+ - type: map_at_1000
1730
+ value: 40.195
1731
+ - type: map_at_3
1732
+ value: 34.565
1733
+ - type: map_at_5
1734
+ value: 37.069
1735
+ - type: mrr_at_1
1736
+ value: 27.578000000000003
1737
+ - type: mrr_at_10
1738
+ value: 41.079
1739
+ - type: mrr_at_100
1740
+ value: 42.081
1741
+ - type: mrr_at_1000
1742
+ value: 42.109
1743
+ - type: mrr_at_3
1744
+ value: 37.278
1745
+ - type: mrr_at_5
1746
+ value: 39.585
1747
+ - type: ndcg_at_1
1748
+ value: 27.549
1749
+ - type: ndcg_at_10
1750
+ value: 46.506
1751
+ - type: ndcg_at_100
1752
+ value: 51.92400000000001
1753
+ - type: ndcg_at_1000
1754
+ value: 52.833
1755
+ - type: ndcg_at_3
1756
+ value: 38.214999999999996
1757
+ - type: ndcg_at_5
1758
+ value: 42.498000000000005
1759
+ - type: precision_at_1
1760
+ value: 27.549
1761
+ - type: precision_at_10
1762
+ value: 8.019
1763
+ - type: precision_at_100
1764
+ value: 1.103
1765
+ - type: precision_at_1000
1766
+ value: 0.11900000000000001
1767
+ - type: precision_at_3
1768
+ value: 17.806
1769
+ - type: precision_at_5
1770
+ value: 13.100000000000001
1771
+ - type: recall_at_1
1772
+ value: 24.468999999999998
1773
+ - type: recall_at_10
1774
+ value: 67.632
1775
+ - type: recall_at_100
1776
+ value: 91.169
1777
+ - type: recall_at_1000
1778
+ value: 97.851
1779
+ - type: recall_at_3
1780
+ value: 46.043
1781
+ - type: recall_at_5
1782
+ value: 55.962999999999994
1783
+ - task:
1784
+ type: Retrieval
1785
+ dataset:
1786
+ type: quora
1787
+ name: MTEB QuoraRetrieval
1788
+ config: default
1789
+ split: test
1790
+ revision: None
1791
+ metrics:
1792
+ - type: map_at_1
1793
+ value: 70.44
1794
+ - type: map_at_10
1795
+ value: 84.209
1796
+ - type: map_at_100
1797
+ value: 84.868
1798
+ - type: map_at_1000
1799
+ value: 84.884
1800
+ - type: map_at_3
1801
+ value: 81.192
1802
+ - type: map_at_5
1803
+ value: 83.06099999999999
1804
+ - type: mrr_at_1
1805
+ value: 81.12
1806
+ - type: mrr_at_10
1807
+ value: 87.30499999999999
1808
+ - type: mrr_at_100
1809
+ value: 87.413
1810
+ - type: mrr_at_1000
1811
+ value: 87.414
1812
+ - type: mrr_at_3
1813
+ value: 86.337
1814
+ - type: mrr_at_5
1815
+ value: 86.985
1816
+ - type: ndcg_at_1
1817
+ value: 81.15
1818
+ - type: ndcg_at_10
1819
+ value: 88.032
1820
+ - type: ndcg_at_100
1821
+ value: 89.292
1822
+ - type: ndcg_at_1000
1823
+ value: 89.393
1824
+ - type: ndcg_at_3
1825
+ value: 85.098
1826
+ - type: ndcg_at_5
1827
+ value: 86.691
1828
+ - type: precision_at_1
1829
+ value: 81.15
1830
+ - type: precision_at_10
1831
+ value: 13.395999999999999
1832
+ - type: precision_at_100
1833
+ value: 1.5310000000000001
1834
+ - type: precision_at_1000
1835
+ value: 0.157
1836
+ - type: precision_at_3
1837
+ value: 37.16
1838
+ - type: precision_at_5
1839
+ value: 24.458
1840
+ - type: recall_at_1
1841
+ value: 70.44
1842
+ - type: recall_at_10
1843
+ value: 95.204
1844
+ - type: recall_at_100
1845
+ value: 99.506
1846
+ - type: recall_at_1000
1847
+ value: 99.978
1848
+ - type: recall_at_3
1849
+ value: 86.83999999999999
1850
+ - type: recall_at_5
1851
+ value: 91.328
1852
+ - task:
1853
+ type: Clustering
1854
+ dataset:
1855
+ type: mteb/reddit-clustering
1856
+ name: MTEB RedditClustering
1857
+ config: default
1858
+ split: test
1859
+ revision: 24640382cdbf8abc73003fb0fa6d111a705499eb
1860
+ metrics:
1861
+ - type: v_measure
1862
+ value: 44.091918771223966
1863
+ - task:
1864
+ type: Clustering
1865
+ dataset:
1866
+ type: mteb/reddit-clustering-p2p
1867
+ name: MTEB RedditClusteringP2P
1868
+ config: default
1869
+ split: test
1870
+ revision: 282350215ef01743dc01b456c7f5241fa8937f16
1871
+ metrics:
1872
+ - type: v_measure
1873
+ value: 49.3850718319815
1874
+ - task:
1875
+ type: Retrieval
1876
+ dataset:
1877
+ type: scidocs
1878
+ name: MTEB SCIDOCS
1879
+ config: default
1880
+ split: test
1881
+ revision: None
1882
+ metrics:
1883
+ - type: map_at_1
1884
+ value: 5.108
1885
+ - type: map_at_10
1886
+ value: 12.878
1887
+ - type: map_at_100
1888
+ value: 15.398
1889
+ - type: map_at_1000
1890
+ value: 15.762
1891
+ - type: map_at_3
1892
+ value: 9.028
1893
+ - type: map_at_5
1894
+ value: 10.886
1895
+ - type: mrr_at_1
1896
+ value: 25.2
1897
+ - type: mrr_at_10
1898
+ value: 36.051
1899
+ - type: mrr_at_100
1900
+ value: 37.198
1901
+ - type: mrr_at_1000
1902
+ value: 37.254
1903
+ - type: mrr_at_3
1904
+ value: 32.483000000000004
1905
+ - type: mrr_at_5
1906
+ value: 34.583000000000006
1907
+ - type: ndcg_at_1
1908
+ value: 25.2
1909
+ - type: ndcg_at_10
1910
+ value: 21.436
1911
+ - type: ndcg_at_100
1912
+ value: 30.758000000000003
1913
+ - type: ndcg_at_1000
1914
+ value: 36.774
1915
+ - type: ndcg_at_3
1916
+ value: 19.977
1917
+ - type: ndcg_at_5
1918
+ value: 17.634
1919
+ - type: precision_at_1
1920
+ value: 25.2
1921
+ - type: precision_at_10
1922
+ value: 11.16
1923
+ - type: precision_at_100
1924
+ value: 2.46
1925
+ - type: precision_at_1000
1926
+ value: 0.38999999999999996
1927
+ - type: precision_at_3
1928
+ value: 18.4
1929
+ - type: precision_at_5
1930
+ value: 15.440000000000001
1931
+ - type: recall_at_1
1932
+ value: 5.108
1933
+ - type: recall_at_10
1934
+ value: 22.615
1935
+ - type: recall_at_100
1936
+ value: 49.838
1937
+ - type: recall_at_1000
1938
+ value: 79.12700000000001
1939
+ - type: recall_at_3
1940
+ value: 11.203000000000001
1941
+ - type: recall_at_5
1942
+ value: 15.638
1943
+ - task:
1944
+ type: STS
1945
+ dataset:
1946
+ type: mteb/sickr-sts
1947
+ name: MTEB SICK-R
1948
+ config: default
1949
+ split: test
1950
+ revision: a6ea5a8cab320b040a23452cc28066d9beae2cee
1951
+ metrics:
1952
+ - type: cos_sim_pearson
1953
+ value: 84.87907802108278
1954
+ - type: cos_sim_spearman
1955
+ value: 78.47745630820519
1956
+ - type: euclidean_pearson
1957
+ value: 81.24598854050433
1958
+ - type: euclidean_spearman
1959
+ value: 76.49536405466311
1960
+ - type: manhattan_pearson
1961
+ value: 81.2143517198192
1962
+ - type: manhattan_spearman
1963
+ value: 76.41735187637899
1964
+ - task:
1965
+ type: STS
1966
+ dataset:
1967
+ type: mteb/sts12-sts
1968
+ name: MTEB STS12
1969
+ config: default
1970
+ split: test
1971
+ revision: a0d554a64d88156834ff5ae9920b964011b16384
1972
+ metrics:
1973
+ - type: cos_sim_pearson
1974
+ value: 84.72222146895906
1975
+ - type: cos_sim_spearman
1976
+ value: 75.78345138703104
1977
+ - type: euclidean_pearson
1978
+ value: 81.35072741369821
1979
+ - type: euclidean_spearman
1980
+ value: 71.44372390021385
1981
+ - type: manhattan_pearson
1982
+ value: 81.42777992212991
1983
+ - type: manhattan_spearman
1984
+ value: 71.50748732911025
1985
+ - task:
1986
+ type: STS
1987
+ dataset:
1988
+ type: mteb/sts13-sts
1989
+ name: MTEB STS13
1990
+ config: default
1991
+ split: test
1992
+ revision: 7e90230a92c190f1bf69ae9002b8cea547a64cca
1993
+ metrics:
1994
+ - type: cos_sim_pearson
1995
+ value: 82.46314178714782
1996
+ - type: cos_sim_spearman
1997
+ value: 83.30487501773337
1998
+ - type: euclidean_pearson
1999
+ value: 81.97496753880277
2000
+ - type: euclidean_spearman
2001
+ value: 83.26569157819903
2002
+ - type: manhattan_pearson
2003
+ value: 81.95087299528338
2004
+ - type: manhattan_spearman
2005
+ value: 83.25657383286989
2006
+ - task:
2007
+ type: STS
2008
+ dataset:
2009
+ type: mteb/sts14-sts
2010
+ name: MTEB STS14
2011
+ config: default
2012
+ split: test
2013
+ revision: 6031580fec1f6af667f0bd2da0a551cf4f0b2375
2014
+ metrics:
2015
+ - type: cos_sim_pearson
2016
+ value: 82.38192118423038
2017
+ - type: cos_sim_spearman
2018
+ value: 78.40410104736917
2019
+ - type: euclidean_pearson
2020
+ value: 79.48941144435967
2021
+ - type: euclidean_spearman
2022
+ value: 76.87243228899331
2023
+ - type: manhattan_pearson
2024
+ value: 79.37383745954276
2025
+ - type: manhattan_spearman
2026
+ value: 76.81624170740595
2027
+ - task:
2028
+ type: STS
2029
+ dataset:
2030
+ type: mteb/sts15-sts
2031
+ name: MTEB STS15
2032
+ config: default
2033
+ split: test
2034
+ revision: ae752c7c21bf194d8b67fd573edf7ae58183cbe3
2035
+ metrics:
2036
+ - type: cos_sim_pearson
2037
+ value: 84.89499997364136
2038
+ - type: cos_sim_spearman
2039
+ value: 86.49722400765071
2040
+ - type: euclidean_pearson
2041
+ value: 80.83327622391033
2042
+ - type: euclidean_spearman
2043
+ value: 81.77906221038033
2044
+ - type: manhattan_pearson
2045
+ value: 80.68927444298423
2046
+ - type: manhattan_spearman
2047
+ value: 81.67585996918764
2048
+ - task:
2049
+ type: STS
2050
+ dataset:
2051
+ type: mteb/sts16-sts
2052
+ name: MTEB STS16
2053
+ config: default
2054
+ split: test
2055
+ revision: 4d8694f8f0e0100860b497b999b3dbed754a0513
2056
+ metrics:
2057
+ - type: cos_sim_pearson
2058
+ value: 80.85434430333662
2059
+ - type: cos_sim_spearman
2060
+ value: 82.32641704038703
2061
+ - type: euclidean_pearson
2062
+ value: 78.92319495883405
2063
+ - type: euclidean_spearman
2064
+ value: 80.06748121443441
2065
+ - type: manhattan_pearson
2066
+ value: 78.68188267117745
2067
+ - type: manhattan_spearman
2068
+ value: 79.72019793896195
2069
+ - task:
2070
+ type: STS
2071
+ dataset:
2072
+ type: mteb/sts17-crosslingual-sts
2073
+ name: MTEB STS17 (en-en)
2074
+ config: en-en
2075
+ split: test
2076
+ revision: af5e6fb845001ecf41f4c1e033ce921939a2a68d
2077
+ metrics:
2078
+ - type: cos_sim_pearson
2079
+ value: 87.0896689258414
2080
+ - type: cos_sim_spearman
2081
+ value: 87.31114069713735
2082
+ - type: euclidean_pearson
2083
+ value: 83.93671908621272
2084
+ - type: euclidean_spearman
2085
+ value: 82.83918654090873
2086
+ - type: manhattan_pearson
2087
+ value: 83.5943550673816
2088
+ - type: manhattan_spearman
2089
+ value: 82.47327946394148
2090
+ - task:
2091
+ type: STS
2092
+ dataset:
2093
+ type: mteb/sts22-crosslingual-sts
2094
+ name: MTEB STS22 (en)
2095
+ config: en
2096
+ split: test
2097
+ revision: 6d1ba47164174a496b7fa5d3569dae26a6813b80
2098
+ metrics:
2099
+ - type: cos_sim_pearson
2100
+ value: 66.4799391480602
2101
+ - type: cos_sim_spearman
2102
+ value: 66.59141182659532
2103
+ - type: euclidean_pearson
2104
+ value: 45.85714541149068
2105
+ - type: euclidean_spearman
2106
+ value: 61.605252732946404
2107
+ - type: manhattan_pearson
2108
+ value: 46.69415667711241
2109
+ - type: manhattan_spearman
2110
+ value: 61.38490967409539
2111
+ - task:
2112
+ type: STS
2113
+ dataset:
2114
+ type: mteb/stsbenchmark-sts
2115
+ name: MTEB STSBenchmark
2116
+ config: default
2117
+ split: test
2118
+ revision: b0fddb56ed78048fa8b90373c8a3cfc37b684831
2119
+ metrics:
2120
+ - type: cos_sim_pearson
2121
+ value: 82.22064334651283
2122
+ - type: cos_sim_spearman
2123
+ value: 84.23556405551305
2124
+ - type: euclidean_pearson
2125
+ value: 80.64484589022672
2126
+ - type: euclidean_spearman
2127
+ value: 80.27585966983669
2128
+ - type: manhattan_pearson
2129
+ value: 80.44248540454653
2130
+ - type: manhattan_spearman
2131
+ value: 80.06071452831723
2132
+ - task:
2133
+ type: Reranking
2134
+ dataset:
2135
+ type: mteb/scidocs-reranking
2136
+ name: MTEB SciDocsRR
2137
+ config: default
2138
+ split: test
2139
+ revision: d3c5e1fc0b855ab6097bf1cda04dd73947d7caab
2140
+ metrics:
2141
+ - type: map
2142
+ value: 86.82632940766443
2143
+ - type: mrr
2144
+ value: 96.27367186190715
2145
+ - task:
2146
+ type: Retrieval
2147
+ dataset:
2148
+ type: scifact
2149
+ name: MTEB SciFact
2150
+ config: default
2151
+ split: test
2152
+ revision: None
2153
+ metrics:
2154
+ - type: map_at_1
2155
+ value: 48.443999999999996
2156
+ - type: map_at_10
2157
+ value: 58.309
2158
+ - type: map_at_100
2159
+ value: 59.116
2160
+ - type: map_at_1000
2161
+ value: 59.155
2162
+ - type: map_at_3
2163
+ value: 55.598000000000006
2164
+ - type: map_at_5
2165
+ value: 57.550999999999995
2166
+ - type: mrr_at_1
2167
+ value: 50.666999999999994
2168
+ - type: mrr_at_10
2169
+ value: 59.099000000000004
2170
+ - type: mrr_at_100
2171
+ value: 59.843
2172
+ - type: mrr_at_1000
2173
+ value: 59.879000000000005
2174
+ - type: mrr_at_3
2175
+ value: 57.167
2176
+ - type: mrr_at_5
2177
+ value: 58.5
2178
+ - type: ndcg_at_1
2179
+ value: 50.666999999999994
2180
+ - type: ndcg_at_10
2181
+ value: 62.483999999999995
2182
+ - type: ndcg_at_100
2183
+ value: 66.131
2184
+ - type: ndcg_at_1000
2185
+ value: 67.17
2186
+ - type: ndcg_at_3
2187
+ value: 58.07299999999999
2188
+ - type: ndcg_at_5
2189
+ value: 60.87200000000001
2190
+ - type: precision_at_1
2191
+ value: 50.666999999999994
2192
+ - type: precision_at_10
2193
+ value: 8.4
2194
+ - type: precision_at_100
2195
+ value: 1.0330000000000001
2196
+ - type: precision_at_1000
2197
+ value: 0.11199999999999999
2198
+ - type: precision_at_3
2199
+ value: 22.889
2200
+ - type: precision_at_5
2201
+ value: 15.467
2202
+ - type: recall_at_1
2203
+ value: 48.443999999999996
2204
+ - type: recall_at_10
2205
+ value: 74.26700000000001
2206
+ - type: recall_at_100
2207
+ value: 90.5
2208
+ - type: recall_at_1000
2209
+ value: 98.667
2210
+ - type: recall_at_3
2211
+ value: 63.039
2212
+ - type: recall_at_5
2213
+ value: 69.706
2214
+ - task:
2215
+ type: PairClassification
2216
+ dataset:
2217
+ type: mteb/sprintduplicatequestions-pairclassification
2218
+ name: MTEB SprintDuplicateQuestions
2219
+ config: default
2220
+ split: test
2221
+ revision: d66bd1f72af766a5cc4b0ca5e00c162f89e8cc46
2222
+ metrics:
2223
+ - type: cos_sim_accuracy
2224
+ value: 99.76336633663367
2225
+ - type: cos_sim_ap
2226
+ value: 94.05677361006421
2227
+ - type: cos_sim_f1
2228
+ value: 87.85894206549118
2229
+ - type: cos_sim_precision
2230
+ value: 88.52791878172589
2231
+ - type: cos_sim_recall
2232
+ value: 87.2
2233
+ - type: dot_accuracy
2234
+ value: 99.06732673267327
2235
+ - type: dot_ap
2236
+ value: 25.234902506145275
2237
+ - type: dot_f1
2238
+ value: 31.687715269804816
2239
+ - type: dot_precision
2240
+ value: 37.19676549865229
2241
+ - type: dot_recall
2242
+ value: 27.6
2243
+ - type: euclidean_accuracy
2244
+ value: 99.73861386138614
2245
+ - type: euclidean_ap
2246
+ value: 92.39504711224613
2247
+ - type: euclidean_f1
2248
+ value: 86.40576725025747
2249
+ - type: euclidean_precision
2250
+ value: 89.06581740976645
2251
+ - type: euclidean_recall
2252
+ value: 83.89999999999999
2253
+ - type: manhattan_accuracy
2254
+ value: 99.74455445544554
2255
+ - type: manhattan_ap
2256
+ value: 92.5050066340186
2257
+ - type: manhattan_f1
2258
+ value: 86.67355371900827
2259
+ - type: manhattan_precision
2260
+ value: 89.63675213675214
2261
+ - type: manhattan_recall
2262
+ value: 83.89999999999999
2263
+ - type: max_accuracy
2264
+ value: 99.76336633663367
2265
+ - type: max_ap
2266
+ value: 94.05677361006421
2267
+ - type: max_f1
2268
+ value: 87.85894206549118
2269
+ - task:
2270
+ type: Clustering
2271
+ dataset:
2272
+ type: mteb/stackexchange-clustering
2273
+ name: MTEB StackExchangeClustering
2274
+ config: default
2275
+ split: test
2276
+ revision: 6cbc1f7b2bc0622f2e39d2c77fa502909748c259
2277
+ metrics:
2278
+ - type: v_measure
2279
+ value: 52.66315650755836
2280
+ - task:
2281
+ type: Clustering
2282
+ dataset:
2283
+ type: mteb/stackexchange-clustering-p2p
2284
+ name: MTEB StackExchangeClusteringP2P
2285
+ config: default
2286
+ split: test
2287
+ revision: 815ca46b2622cec33ccafc3735d572c266efdb44
2288
+ metrics:
2289
+ - type: v_measure
2290
+ value: 32.36019149648443
2291
+ - task:
2292
+ type: Reranking
2293
+ dataset:
2294
+ type: mteb/stackoverflowdupquestions-reranking
2295
+ name: MTEB StackOverflowDupQuestions
2296
+ config: default
2297
+ split: test
2298
+ revision: e185fbe320c72810689fc5848eb6114e1ef5ec69
2299
+ metrics:
2300
+ - type: map
2301
+ value: 50.10933600138655
2302
+ - type: mrr
2303
+ value: 50.84273671589848
2304
+ - task:
2305
+ type: Summarization
2306
+ dataset:
2307
+ type: mteb/summeval
2308
+ name: MTEB SummEval
2309
+ config: default
2310
+ split: test
2311
+ revision: cda12ad7615edc362dbf25a00fdd61d3b1eaf93c
2312
+ metrics:
2313
+ - type: cos_sim_pearson
2314
+ value: 30.342194052503917
2315
+ - type: cos_sim_spearman
2316
+ value: 30.74326118928312
2317
+ - type: dot_pearson
2318
+ value: 12.329727800033176
2319
+ - type: dot_spearman
2320
+ value: 14.54557726626662
2321
+ - task:
2322
+ type: Retrieval
2323
+ dataset:
2324
+ type: trec-covid
2325
+ name: MTEB TRECCOVID
2326
+ config: default
2327
+ split: test
2328
+ revision: None
2329
+ metrics:
2330
+ - type: map_at_1
2331
+ value: 0.173
2332
+ - type: map_at_10
2333
+ value: 1.1320000000000001
2334
+ - type: map_at_100
2335
+ value: 5.885
2336
+ - type: map_at_1000
2337
+ value: 14.762
2338
+ - type: map_at_3
2339
+ value: 0.443
2340
+ - type: map_at_5
2341
+ value: 0.66
2342
+ - type: mrr_at_1
2343
+ value: 66.0
2344
+ - type: mrr_at_10
2345
+ value: 76.34100000000001
2346
+ - type: mrr_at_100
2347
+ value: 76.37
2348
+ - type: mrr_at_1000
2349
+ value: 76.376
2350
+ - type: mrr_at_3
2351
+ value: 74.667
2352
+ - type: mrr_at_5
2353
+ value: 74.667
2354
+ - type: ndcg_at_1
2355
+ value: 59.0
2356
+ - type: ndcg_at_10
2357
+ value: 50.047
2358
+ - type: ndcg_at_100
2359
+ value: 37.744
2360
+ - type: ndcg_at_1000
2361
+ value: 35.903
2362
+ - type: ndcg_at_3
2363
+ value: 55.95
2364
+ - type: ndcg_at_5
2365
+ value: 53.379
2366
+ - type: precision_at_1
2367
+ value: 66.0
2368
+ - type: precision_at_10
2369
+ value: 53.0
2370
+ - type: precision_at_100
2371
+ value: 38.78
2372
+ - type: precision_at_1000
2373
+ value: 16.24
2374
+ - type: precision_at_3
2375
+ value: 60.0
2376
+ - type: precision_at_5
2377
+ value: 56.39999999999999
2378
+ - type: recall_at_1
2379
+ value: 0.173
2380
+ - type: recall_at_10
2381
+ value: 1.379
2382
+ - type: recall_at_100
2383
+ value: 9.196
2384
+ - type: recall_at_1000
2385
+ value: 34.488
2386
+ - type: recall_at_3
2387
+ value: 0.475
2388
+ - type: recall_at_5
2389
+ value: 0.738
2390
+ - task:
2391
+ type: Retrieval
2392
+ dataset:
2393
+ type: webis-touche2020
2394
+ name: MTEB Touche2020
2395
+ config: default
2396
+ split: test
2397
+ revision: None
2398
+ metrics:
2399
+ - type: map_at_1
2400
+ value: 2.1260000000000003
2401
+ - type: map_at_10
2402
+ value: 7.216
2403
+ - type: map_at_100
2404
+ value: 12.732
2405
+ - type: map_at_1000
2406
+ value: 14.158999999999999
2407
+ - type: map_at_3
2408
+ value: 3.9530000000000003
2409
+ - type: map_at_5
2410
+ value: 5.252
2411
+ - type: mrr_at_1
2412
+ value: 24.490000000000002
2413
+ - type: mrr_at_10
2414
+ value: 36.949
2415
+ - type: mrr_at_100
2416
+ value: 38.0
2417
+ - type: mrr_at_1000
2418
+ value: 38.0
2419
+ - type: mrr_at_3
2420
+ value: 31.973000000000003
2421
+ - type: mrr_at_5
2422
+ value: 34.32
2423
+ - type: ndcg_at_1
2424
+ value: 19.387999999999998
2425
+ - type: ndcg_at_10
2426
+ value: 17.918
2427
+ - type: ndcg_at_100
2428
+ value: 30.558999999999997
2429
+ - type: ndcg_at_1000
2430
+ value: 42.028
2431
+ - type: ndcg_at_3
2432
+ value: 17.202
2433
+ - type: ndcg_at_5
2434
+ value: 17.788
2435
+ - type: precision_at_1
2436
+ value: 24.490000000000002
2437
+ - type: precision_at_10
2438
+ value: 17.347
2439
+ - type: precision_at_100
2440
+ value: 6.918
2441
+ - type: precision_at_1000
2442
+ value: 1.4569999999999999
2443
+ - type: precision_at_3
2444
+ value: 19.728
2445
+ - type: precision_at_5
2446
+ value: 19.592000000000002
2447
+ - type: recall_at_1
2448
+ value: 2.1260000000000003
2449
+ - type: recall_at_10
2450
+ value: 12.897
2451
+ - type: recall_at_100
2452
+ value: 42.632999999999996
2453
+ - type: recall_at_1000
2454
+ value: 77.783
2455
+ - type: recall_at_3
2456
+ value: 4.836
2457
+ - type: recall_at_5
2458
+ value: 7.331
2459
+ - task:
2460
+ type: Classification
2461
+ dataset:
2462
+ type: mteb/toxic_conversations_50k
2463
+ name: MTEB ToxicConversationsClassification
2464
+ config: default
2465
+ split: test
2466
+ revision: d7c0de2777da35d6aae2200a62c6e0e5af397c4c
2467
+ metrics:
2468
+ - type: accuracy
2469
+ value: 70.9516
2470
+ - type: ap
2471
+ value: 14.148097836321893
2472
+ - type: f1
2473
+ value: 54.52189833022899
2474
+ - task:
2475
+ type: Classification
2476
+ dataset:
2477
+ type: mteb/tweet_sentiment_extraction
2478
+ name: MTEB TweetSentimentExtractionClassification
2479
+ config: default
2480
+ split: test
2481
+ revision: d604517c81ca91fe16a244d1248fc021f9ecee7a
2482
+ metrics:
2483
+ - type: accuracy
2484
+ value: 58.33899264289756
2485
+ - type: f1
2486
+ value: 58.684516042056565
2487
+ - task:
2488
+ type: Clustering
2489
+ dataset:
2490
+ type: mteb/twentynewsgroups-clustering
2491
+ name: MTEB TwentyNewsgroupsClustering
2492
+ config: default
2493
+ split: test
2494
+ revision: 6125ec4e24fa026cec8a478383ee943acfbd5449
2495
+ metrics:
2496
+ - type: v_measure
2497
+ value: 41.45569187892743
2498
+ - task:
2499
+ type: PairClassification
2500
+ dataset:
2501
+ type: mteb/twittersemeval2015-pairclassification
2502
+ name: MTEB TwitterSemEval2015
2503
+ config: default
2504
+ split: test
2505
+ revision: 70970daeab8776df92f5ea462b6173c0b46fd2d1
2506
+ metrics:
2507
+ - type: cos_sim_accuracy
2508
+ value: 85.05692316862371
2509
+ - type: cos_sim_ap
2510
+ value: 70.54785019750204
2511
+ - type: cos_sim_f1
2512
+ value: 65.99060103883255
2513
+ - type: cos_sim_precision
2514
+ value: 62.10428305400373
2515
+ - type: cos_sim_recall
2516
+ value: 70.3957783641161
2517
+ - type: dot_accuracy
2518
+ value: 77.82678667222984
2519
+ - type: dot_ap
2520
+ value: 32.73452779849359
2521
+ - type: dot_f1
2522
+ value: 38.1269911832259
2523
+ - type: dot_precision
2524
+ value: 26.5066446893994
2525
+ - type: dot_recall
2526
+ value: 67.8891820580475
2527
+ - type: euclidean_accuracy
2528
+ value: 84.62180365977231
2529
+ - type: euclidean_ap
2530
+ value: 68.57434108453688
2531
+ - type: euclidean_f1
2532
+ value: 65.23069391751316
2533
+ - type: euclidean_precision
2534
+ value: 60.83086053412463
2535
+ - type: euclidean_recall
2536
+ value: 70.31662269129288
2537
+ - type: manhattan_accuracy
2538
+ value: 84.57411933003517
2539
+ - type: manhattan_ap
2540
+ value: 68.3530821550187
2541
+ - type: manhattan_f1
2542
+ value: 64.74820143884892
2543
+ - type: manhattan_precision
2544
+ value: 61.09550561797753
2545
+ - type: manhattan_recall
2546
+ value: 68.86543535620054
2547
+ - type: max_accuracy
2548
+ value: 85.05692316862371
2549
+ - type: max_ap
2550
+ value: 70.54785019750204
2551
+ - type: max_f1
2552
+ value: 65.99060103883255
2553
+ - task:
2554
+ type: PairClassification
2555
+ dataset:
2556
+ type: mteb/twitterurlcorpus-pairclassification
2557
+ name: MTEB TwitterURLCorpus
2558
+ config: default
2559
+ split: test
2560
+ revision: 8b6510b0b1fa4e4c4f879467980e9be563ec1cdf
2561
+ metrics:
2562
+ - type: cos_sim_accuracy
2563
+ value: 88.77440136608841
2564
+ - type: cos_sim_ap
2565
+ value: 85.6224854550336
2566
+ - type: cos_sim_f1
2567
+ value: 77.76333865518139
2568
+ - type: cos_sim_precision
2569
+ value: 75.09501613481535
2570
+ - type: cos_sim_recall
2571
+ value: 80.6282722513089
2572
+ - type: dot_accuracy
2573
+ value: 79.73570846431483
2574
+ - type: dot_ap
2575
+ value: 59.509855217305315
2576
+ - type: dot_f1
2577
+ value: 57.20318336852364
2578
+ - type: dot_precision
2579
+ value: 49.474630555711634
2580
+ - type: dot_recall
2581
+ value: 67.79334770557438
2582
+ - type: euclidean_accuracy
2583
+ value: 87.06096945705748
2584
+ - type: euclidean_ap
2585
+ value: 81.65241378370953
2586
+ - type: euclidean_f1
2587
+ value: 73.29885784441386
2588
+ - type: euclidean_precision
2589
+ value: 70.91642070405298
2590
+ - type: euclidean_recall
2591
+ value: 75.8469356328919
2592
+ - type: manhattan_accuracy
2593
+ value: 86.973648465091
2594
+ - type: manhattan_ap
2595
+ value: 81.57560533116907
2596
+ - type: manhattan_f1
2597
+ value: 73.2408287397833
2598
+ - type: manhattan_precision
2599
+ value: 72.33611173687767
2600
+ - type: manhattan_recall
2601
+ value: 74.16846319679703
2602
+ - type: max_accuracy
2603
+ value: 88.77440136608841
2604
+ - type: max_ap
2605
+ value: 85.6224854550336
2606
+ - type: max_f1
2607
+ value: 77.76333865518139
2608
  ---
2609
+ <h1 align="center">GIST Embedding v0 - all-MiniLM-L6-v2</h1>
2610
+
2611
+ *GIST Embedding: Guided In-sample Selection of Training Negatives for Text Embedding*
2612
+
2613
+ The model is fine-tuned on top of the [sentence-transformers/all-MiniLM-L6-v2](https://huggingface.co/sentence-transformers/all-MiniLM-L6-v2) using the [MEDI dataset](https://github.com/xlang-ai/instructor-embedding.git) augmented with mined triplets from the [MTEB Classification](https://huggingface.co/mteb) training dataset (excluding data from the Amazon Polarity Classification task).
2614
+
2615
+ The model does not require any instruction for generating embeddings. This means that queries for retrieval tasks can be directly encoded without crafting instructions.
2616
+
2617
+ Technical details of the model will be published shortly.
2618
+
2619
+ # Data
2620
+
2621
+ The dataset used is a compilation of the MEDI dataset and the MTEB Classification training dataset. Third-party datasets may be subject to additional terms and conditions under their associated licenses. A HuggingFace Dataset version of the compiled dataset, and the specific revision used to train the model, is available:
2622
+
2623
+ - Dataset: [avsolatorio/medi-data-mteb_avs_triplets](https://huggingface.co/datasets/avsolatorio/medi-data-mteb_avs_triplets)
2624
+ - Revision: 238a0499b6e6b690cc64ea56fde8461daa8341bb
2625
+
2626
+ The dataset contains a `task_type` key which can be used to select only the mteb classification tasks (prefixed with `mteb_`).
2627
+
2628
+ The **MEDI Dataset** is published in the following paper: [One Embedder, Any Task: Instruction-Finetuned Text Embeddings](https://arxiv.org/abs/2212.09741).
2629
+
2630
+ The MTEB Benchmark results of the GIST embedding model, compared with the base model, suggest that the fine-tuning dataset has perturbed the model considerably, which resulted in significant improvements in certain tasks while adversely degrading performance in some.
2631
+
2632
+ The retrieval performance for the TRECCOVID task is of note. The fine-tuning dataset does not contain significant knowledge about COVID, which could have caused the observed performance degradation. Further work is currently being undertaken to validate this hypothesis.
2633
+
2634
+ # Usage
2635
+
2636
+ The model can be easily loaded using the Sentence Transformers library.
2637
+
2638
+ ```Python
2639
+ import torch.nn.functional as F
2640
+ from sentence_transformers import SentenceTransformer
2641
+
2642
+ revision = None # Replace with the specific revision to ensure reproducibility in case the model is updated.
2643
+
2644
+ model = SentenceTransformer("avsolatorio/GIST-all-MiniLM-L6-v2", revision=revision)
2645
+
2646
+ texts = [
2647
+ "Illustration of the REaLTabFormer model. The left block shows the non-relational tabular data model using GPT-2 with a causal LM head. In contrast, the right block shows how a relational dataset's child table is modeled using a sequence-to-sequence (Seq2Seq) model. The Seq2Seq model uses the observations in the parent table to condition the generation of the observations in the child table. The trained GPT-2 model on the parent table, with weights frozen, is also used as the encoder in the Seq2Seq model.",
2648
+ "Predicting human mobility holds significant practical value, with applications ranging from enhancing disaster risk planning to simulating epidemic spread. In this paper, we present the GeoFormer, a decoder-only transformer model adapted from the GPT architecture to forecast human mobility.",
2649
+ "As the economies of Southeast Asia continue adopting digital technologies, policy makers increasingly ask how to prepare the workforce for emerging labor demands. However, little is known about the skills that workers need to adapt to these changes"
2650
+ ]
2651
+
2652
+ # Compute embeddings
2653
+ embeddings = model.encode(texts, convert_to_tensor=True)
2654
+
2655
+ # Compute cosine-similarity for each pair of sentences
2656
+ scores = F.cosine_similarity(embeddings.unsqueeze(1), embeddings.unsqueeze(0), dim=-1)
2657
+
2658
+ print(scores.cpu().numpy())
2659
+ ```
2660
+
2661
+ # Training Parameters
2662
+
2663
+ Below are the training parameters used to fine-tune the model:
2664
+
2665
+ ```
2666
+ Epochs = 40
2667
+ Warmup ratio = 0.1
2668
+ Learning rate = 5e-6
2669
+ Batch size = 16
2670
+ Checkpoint step = 102000
2671
+ Contrastive loss temperature = 0.01
2672
+ ```
2673
+
2674
+ Specific training details and strategies will be published shortly.
2675
+
2676
+ # Evaluation
2677
+
2678
+ The model was evaluated using the [MTEB Evaluation](https://huggingface.co/mteb) suite.
2679
+
2680
+
2681
+ # Acknowledgements
2682
+
2683
+ This work is supported by the "KCP IV - Exploring Data Use in the Development Economics Literature using Large Language Models (AI and LLMs)" project funded by the [Knowledge for Change Program (KCP)](https://www.worldbank.org/en/programs/knowledge-for-change) of the World Bank - RA-P503405-RESE-TF0C3444.
2684
+
2685
+ The findings, interpretations, and conclusions expressed in this material are entirely those of the authors. They do not necessarily represent the views of the International Bank for Reconstruction and Development/World Bank and its affiliated organizations, or those of the Executive Directors of the World Bank or the governments they represent.