zeroshot commited on
Commit
47911f8
1 Parent(s): 2c2c275

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +380 -2
README.md CHANGED
@@ -1,6 +1,385 @@
1
  ---
2
  tags:
3
  - sparse sparsity quantized onnx embeddings int8
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
4
  license: mit
5
  language:
6
  - en
@@ -48,5 +427,4 @@ For further details regarding DeepSparse & Sentence Transformers integration, re
48
 
49
  For general questions on these models and sparsification methods, reach out to the engineering team on our [community Slack](https://join.slack.com/t/discuss-neuralmagic/shared_invite/zt-q1a1cnvo-YBoICSIw3L1dmQpjBeDurQ).
50
 
51
- ![;)](https://media.giphy.com/media/bYg33GbNbNIVzSrr84/giphy-downsized-large.gif)
52
-
 
1
  ---
2
  tags:
3
  - sparse sparsity quantized onnx embeddings int8
4
+ - mteb
5
+ model-index:
6
+ - name: gte-large-sparse
7
+ results:
8
+ - task:
9
+ type: STS
10
+ dataset:
11
+ type: mteb/biosses-sts
12
+ name: MTEB BIOSSES
13
+ config: default
14
+ split: test
15
+ revision: d3fb88f8f02e40887cd149695127462bbcf29b4a
16
+ metrics:
17
+ - type: cos_sim_pearson
18
+ value: 88.64253410928214
19
+ - type: cos_sim_spearman
20
+ value: 85.83388349410652
21
+ - type: euclidean_pearson
22
+ value: 86.86126159318735
23
+ - type: euclidean_spearman
24
+ value: 85.61580623591163
25
+ - type: manhattan_pearson
26
+ value: 86.6901132883383
27
+ - type: manhattan_spearman
28
+ value: 85.60255292187769
29
+ - task:
30
+ type: STS
31
+ dataset:
32
+ type: mteb/sickr-sts
33
+ name: MTEB SICK-R
34
+ config: default
35
+ split: test
36
+ revision: a6ea5a8cab320b040a23452cc28066d9beae2cee
37
+ metrics:
38
+ - type: cos_sim_pearson
39
+ value: 85.23314640591607
40
+ - type: cos_sim_spearman
41
+ value: 79.00078545104338
42
+ - type: euclidean_pearson
43
+ value: 83.48009254500714
44
+ - type: euclidean_spearman
45
+ value: 78.95413001389939
46
+ - type: manhattan_pearson
47
+ value: 83.46945566025941
48
+ - type: manhattan_spearman
49
+ value: 78.9241707208135
50
+ - task:
51
+ type: STS
52
+ dataset:
53
+ type: mteb/sts12-sts
54
+ name: MTEB STS12
55
+ config: default
56
+ split: test
57
+ revision: a0d554a64d88156834ff5ae9920b964011b16384
58
+ metrics:
59
+ - type: cos_sim_pearson
60
+ value: 81.77526666043804
61
+ - type: cos_sim_spearman
62
+ value: 73.4849063285867
63
+ - type: euclidean_pearson
64
+ value: 78.04477932740524
65
+ - type: euclidean_spearman
66
+ value: 73.01394205771743
67
+ - type: manhattan_pearson
68
+ value: 78.08836684503294
69
+ - type: manhattan_spearman
70
+ value: 73.05074711098149
71
+ - task:
72
+ type: STS
73
+ dataset:
74
+ type: mteb/sts13-sts
75
+ name: MTEB STS13
76
+ config: default
77
+ split: test
78
+ revision: 7e90230a92c190f1bf69ae9002b8cea547a64cca
79
+ metrics:
80
+ - type: cos_sim_pearson
81
+ value: 84.57839215661352
82
+ - type: cos_sim_spearman
83
+ value: 86.13854767345153
84
+ - type: euclidean_pearson
85
+ value: 85.12712609946449
86
+ - type: euclidean_spearman
87
+ value: 85.52497994789026
88
+ - type: manhattan_pearson
89
+ value: 85.06833141611173
90
+ - type: manhattan_spearman
91
+ value: 85.45003068636466
92
+ - task:
93
+ type: STS
94
+ dataset:
95
+ type: mteb/sts14-sts
96
+ name: MTEB STS14
97
+ config: default
98
+ split: test
99
+ revision: 6031580fec1f6af667f0bd2da0a551cf4f0b2375
100
+ metrics:
101
+ - type: cos_sim_pearson
102
+ value: 83.30485126978374
103
+ - type: cos_sim_spearman
104
+ value: 80.36497172462357
105
+ - type: euclidean_pearson
106
+ value: 82.91977909424605
107
+ - type: euclidean_spearman
108
+ value: 80.16995106297438
109
+ - type: manhattan_pearson
110
+ value: 82.88200991402184
111
+ - type: manhattan_spearman
112
+ value: 80.14259757215227
113
+ - task:
114
+ type: STS
115
+ dataset:
116
+ type: mteb/sts15-sts
117
+ name: MTEB STS15
118
+ config: default
119
+ split: test
120
+ revision: ae752c7c21bf194d8b67fd573edf7ae58183cbe3
121
+ metrics:
122
+ - type: cos_sim_pearson
123
+ value: 86.99883111314007
124
+ - type: cos_sim_spearman
125
+ value: 88.531352572377
126
+ - type: euclidean_pearson
127
+ value: 87.96834578059067
128
+ - type: euclidean_spearman
129
+ value: 88.44800718542935
130
+ - type: manhattan_pearson
131
+ value: 87.94889391725033
132
+ - type: manhattan_spearman
133
+ value: 88.45467695837115
134
+ - task:
135
+ type: STS
136
+ dataset:
137
+ type: mteb/sts16-sts
138
+ name: MTEB STS16
139
+ config: default
140
+ split: test
141
+ revision: 4d8694f8f0e0100860b497b999b3dbed754a0513
142
+ metrics:
143
+ - type: cos_sim_pearson
144
+ value: 82.4636984892402
145
+ - type: cos_sim_spearman
146
+ value: 84.0808920789148
147
+ - type: euclidean_pearson
148
+ value: 83.70613486028309
149
+ - type: euclidean_spearman
150
+ value: 84.35941626905009
151
+ - type: manhattan_pearson
152
+ value: 83.70259457073782
153
+ - type: manhattan_spearman
154
+ value: 84.35496521501604
155
+ - task:
156
+ type: STS
157
+ dataset:
158
+ type: mteb/sts17-crosslingual-sts
159
+ name: MTEB STS17 (en-en)
160
+ config: en-en
161
+ split: test
162
+ revision: af5e6fb845001ecf41f4c1e033ce921939a2a68d
163
+ metrics:
164
+ - type: cos_sim_pearson
165
+ value: 88.76172944971023
166
+ - type: cos_sim_spearman
167
+ value: 89.4190945039165
168
+ - type: euclidean_pearson
169
+ value: 89.47263005347381
170
+ - type: euclidean_spearman
171
+ value: 89.49228360724095
172
+ - type: manhattan_pearson
173
+ value: 89.49959868816694
174
+ - type: manhattan_spearman
175
+ value: 89.5314536157954
176
+ - task:
177
+ type: STS
178
+ dataset:
179
+ type: mteb/sts22-crosslingual-sts
180
+ name: MTEB STS22 (en)
181
+ config: en
182
+ split: test
183
+ revision: 6d1ba47164174a496b7fa5d3569dae26a6813b80
184
+ metrics:
185
+ - type: cos_sim_pearson
186
+ value: 64.57158223787549
187
+ - type: cos_sim_spearman
188
+ value: 66.75053533168037
189
+ - type: euclidean_pearson
190
+ value: 66.45526604831747
191
+ - type: euclidean_spearman
192
+ value: 66.14567667353113
193
+ - type: manhattan_pearson
194
+ value: 66.47352000151176
195
+ - type: manhattan_spearman
196
+ value: 66.21099856852885
197
+ - task:
198
+ type: STS
199
+ dataset:
200
+ type: mteb/stsbenchmark-sts
201
+ name: MTEB STSBenchmark
202
+ config: default
203
+ split: test
204
+ revision: b0fddb56ed78048fa8b90373c8a3cfc37b684831
205
+ metrics:
206
+ - type: cos_sim_pearson
207
+ value: 85.055653571006
208
+ - type: cos_sim_spearman
209
+ value: 85.45387832634702
210
+ - type: euclidean_pearson
211
+ value: 86.31667154906651
212
+ - type: euclidean_spearman
213
+ value: 85.66079590537946
214
+ - type: manhattan_pearson
215
+ value: 86.2806853257308
216
+ - type: manhattan_spearman
217
+ value: 85.63700636713952
218
+ - task:
219
+ type: PairClassification
220
+ dataset:
221
+ type: mteb/sprintduplicatequestions-pairclassification
222
+ name: MTEB SprintDuplicateQuestions
223
+ config: default
224
+ split: test
225
+ revision: d66bd1f72af766a5cc4b0ca5e00c162f89e8cc46
226
+ metrics:
227
+ - type: cos_sim_accuracy
228
+ value: 99.78811881188119
229
+ - type: cos_sim_ap
230
+ value: 94.67027715905307
231
+ - type: cos_sim_f1
232
+ value: 89.33074684772066
233
+ - type: cos_sim_precision
234
+ value: 86.7231638418079
235
+ - type: cos_sim_recall
236
+ value: 92.10000000000001
237
+ - type: dot_accuracy
238
+ value: 99.47128712871287
239
+ - type: dot_ap
240
+ value: 78.41478815918727
241
+ - type: dot_f1
242
+ value: 73.30049261083744
243
+ - type: dot_precision
244
+ value: 72.23300970873787
245
+ - type: dot_recall
246
+ value: 74.4
247
+ - type: euclidean_accuracy
248
+ value: 99.78415841584159
249
+ - type: euclidean_ap
250
+ value: 94.60075930867181
251
+ - type: euclidean_f1
252
+ value: 89.12175648702593
253
+ - type: euclidean_precision
254
+ value: 88.94422310756973
255
+ - type: euclidean_recall
256
+ value: 89.3
257
+ - type: manhattan_accuracy
258
+ value: 99.78415841584159
259
+ - type: manhattan_ap
260
+ value: 94.62867439278095
261
+ - type: manhattan_f1
262
+ value: 89.2337536372454
263
+ - type: manhattan_precision
264
+ value: 86.62900188323917
265
+ - type: manhattan_recall
266
+ value: 92.0
267
+ - type: max_accuracy
268
+ value: 99.78811881188119
269
+ - type: max_ap
270
+ value: 94.67027715905307
271
+ - type: max_f1
272
+ value: 89.33074684772066
273
+ - task:
274
+ type: PairClassification
275
+ dataset:
276
+ type: mteb/twittersemeval2015-pairclassification
277
+ name: MTEB TwitterSemEval2015
278
+ config: default
279
+ split: test
280
+ revision: 70970daeab8776df92f5ea462b6173c0b46fd2d1
281
+ metrics:
282
+ - type: cos_sim_accuracy
283
+ value: 85.09864695714371
284
+ - type: cos_sim_ap
285
+ value: 70.33704198164713
286
+ - type: cos_sim_f1
287
+ value: 66.22893954410307
288
+ - type: cos_sim_precision
289
+ value: 62.42410088743577
290
+ - type: cos_sim_recall
291
+ value: 70.52770448548813
292
+ - type: dot_accuracy
293
+ value: 79.11426357513263
294
+ - type: dot_ap
295
+ value: 49.15484584572233
296
+ - type: dot_f1
297
+ value: 51.12580243364951
298
+ - type: dot_precision
299
+ value: 40.13840830449827
300
+ - type: dot_recall
301
+ value: 70.3957783641161
302
+ - type: euclidean_accuracy
303
+ value: 85.15825236931514
304
+ - type: euclidean_ap
305
+ value: 70.51017350854076
306
+ - type: euclidean_f1
307
+ value: 66.45416294785159
308
+ - type: euclidean_precision
309
+ value: 64.29805082654823
310
+ - type: euclidean_recall
311
+ value: 68.7598944591029
312
+ - type: manhattan_accuracy
313
+ value: 85.1403707456637
314
+ - type: manhattan_ap
315
+ value: 70.47587863399994
316
+ - type: manhattan_f1
317
+ value: 66.4576802507837
318
+ - type: manhattan_precision
319
+ value: 63.32138590203107
320
+ - type: manhattan_recall
321
+ value: 69.92084432717678
322
+ - type: max_accuracy
323
+ value: 85.15825236931514
324
+ - type: max_ap
325
+ value: 70.51017350854076
326
+ - type: max_f1
327
+ value: 66.4576802507837
328
+ - task:
329
+ type: PairClassification
330
+ dataset:
331
+ type: mteb/twitterurlcorpus-pairclassification
332
+ name: MTEB TwitterURLCorpus
333
+ config: default
334
+ split: test
335
+ revision: 8b6510b0b1fa4e4c4f879467980e9be563ec1cdf
336
+ metrics:
337
+ - type: cos_sim_accuracy
338
+ value: 88.8539604921023
339
+ - type: cos_sim_ap
340
+ value: 85.71869912577101
341
+ - type: cos_sim_f1
342
+ value: 78.00535626720983
343
+ - type: cos_sim_precision
344
+ value: 76.46232344893885
345
+ - type: cos_sim_recall
346
+ value: 79.61194949183862
347
+ - type: dot_accuracy
348
+ value: 84.57717235223348
349
+ - type: dot_ap
350
+ value: 74.89496650237145
351
+ - type: dot_f1
352
+ value: 69.05327823892932
353
+ - type: dot_precision
354
+ value: 65.75666829166377
355
+ - type: dot_recall
356
+ value: 72.69787496150293
357
+ - type: euclidean_accuracy
358
+ value: 88.89471028835332
359
+ - type: euclidean_ap
360
+ value: 85.75169460500409
361
+ - type: euclidean_f1
362
+ value: 78.17055393586006
363
+ - type: euclidean_precision
364
+ value: 74.21118184334348
365
+ - type: euclidean_recall
366
+ value: 82.57622420696026
367
+ - type: manhattan_accuracy
368
+ value: 88.92187681918733
369
+ - type: manhattan_ap
370
+ value: 85.7496679471825
371
+ - type: manhattan_f1
372
+ value: 78.11088295687884
373
+ - type: manhattan_precision
374
+ value: 75.82083061535117
375
+ - type: manhattan_recall
376
+ value: 80.5435786880197
377
+ - type: max_accuracy
378
+ value: 88.92187681918733
379
+ - type: max_ap
380
+ value: 85.75169460500409
381
+ - type: max_f1
382
+ value: 78.17055393586006
383
  license: mit
384
  language:
385
  - en
 
427
 
428
  For general questions on these models and sparsification methods, reach out to the engineering team on our [community Slack](https://join.slack.com/t/discuss-neuralmagic/shared_invite/zt-q1a1cnvo-YBoICSIw3L1dmQpjBeDurQ).
429
 
430
+ ![;)](https://media.giphy.com/media/bYg33GbNbNIVzSrr84/giphy-downsized-large.gif)