bnightning commited on
Commit
86ddf3c
·
verified ·
1 Parent(s): e8c7c31

Upload README.md with huggingface_hub

Browse files
Files changed (1) hide show
  1. README.md +1107 -0
README.md ADDED
@@ -0,0 +1,1107 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ base_model: aspire/acge_text_embedding
3
+ pipeline_tag: sentence-similarity
4
+ tags:
5
+ - mteb
6
+ - sentence-transformers
7
+ - feature-extraction
8
+ - sentence-similarity
9
+ - llama-cpp
10
+ - gguf-my-repo
11
+ model-index:
12
+ - name: acge_text_embedding
13
+ results:
14
+ - task:
15
+ type: STS
16
+ dataset:
17
+ name: MTEB AFQMC
18
+ type: C-MTEB/AFQMC
19
+ config: default
20
+ split: validation
21
+ revision: b44c3b011063adb25877c13823db83bb193913c4
22
+ metrics:
23
+ - type: cos_sim_pearson
24
+ value: 54.03434872650919
25
+ - type: cos_sim_spearman
26
+ value: 58.80730796688325
27
+ - type: euclidean_pearson
28
+ value: 57.47231387497989
29
+ - type: euclidean_spearman
30
+ value: 58.80775026351807
31
+ - type: manhattan_pearson
32
+ value: 57.46332720141574
33
+ - type: manhattan_spearman
34
+ value: 58.80196022940078
35
+ - task:
36
+ type: STS
37
+ dataset:
38
+ name: MTEB ATEC
39
+ type: C-MTEB/ATEC
40
+ config: default
41
+ split: test
42
+ revision: 0f319b1142f28d00e055a6770f3f726ae9b7d865
43
+ metrics:
44
+ - type: cos_sim_pearson
45
+ value: 53.52621290548175
46
+ - type: cos_sim_spearman
47
+ value: 57.945227768312144
48
+ - type: euclidean_pearson
49
+ value: 61.17041394151802
50
+ - type: euclidean_spearman
51
+ value: 57.94553287835657
52
+ - type: manhattan_pearson
53
+ value: 61.168327500057885
54
+ - type: manhattan_spearman
55
+ value: 57.94477516925043
56
+ - task:
57
+ type: Classification
58
+ dataset:
59
+ name: MTEB AmazonReviewsClassification (zh)
60
+ type: mteb/amazon_reviews_multi
61
+ config: zh
62
+ split: test
63
+ revision: 1399c76144fd37290681b995c656ef9b2e06e26d
64
+ metrics:
65
+ - type: accuracy
66
+ value: 48.538000000000004
67
+ - type: f1
68
+ value: 46.59920995594044
69
+ - task:
70
+ type: STS
71
+ dataset:
72
+ name: MTEB BQ
73
+ type: C-MTEB/BQ
74
+ config: default
75
+ split: test
76
+ revision: e3dda5e115e487b39ec7e618c0c6a29137052a55
77
+ metrics:
78
+ - type: cos_sim_pearson
79
+ value: 68.27529991817154
80
+ - type: cos_sim_spearman
81
+ value: 70.37095914176643
82
+ - type: euclidean_pearson
83
+ value: 69.42690712802727
84
+ - type: euclidean_spearman
85
+ value: 70.37017971889912
86
+ - type: manhattan_pearson
87
+ value: 69.40264877917839
88
+ - type: manhattan_spearman
89
+ value: 70.34786744049524
90
+ - task:
91
+ type: Clustering
92
+ dataset:
93
+ name: MTEB CLSClusteringP2P
94
+ type: C-MTEB/CLSClusteringP2P
95
+ config: default
96
+ split: test
97
+ revision: 4b6227591c6c1a73bc76b1055f3b7f3588e72476
98
+ metrics:
99
+ - type: v_measure
100
+ value: 47.08027536192709
101
+ - task:
102
+ type: Clustering
103
+ dataset:
104
+ name: MTEB CLSClusteringS2S
105
+ type: C-MTEB/CLSClusteringS2S
106
+ config: default
107
+ split: test
108
+ revision: e458b3f5414b62b7f9f83499ac1f5497ae2e869f
109
+ metrics:
110
+ - type: v_measure
111
+ value: 44.0526024940363
112
+ - task:
113
+ type: Reranking
114
+ dataset:
115
+ name: MTEB CMedQAv1
116
+ type: C-MTEB/CMedQAv1-reranking
117
+ config: default
118
+ split: test
119
+ revision: 8d7f1e942507dac42dc58017c1a001c3717da7df
120
+ metrics:
121
+ - type: map
122
+ value: 88.65974993133156
123
+ - type: mrr
124
+ value: 90.64761904761905
125
+ - task:
126
+ type: Reranking
127
+ dataset:
128
+ name: MTEB CMedQAv2
129
+ type: C-MTEB/CMedQAv2-reranking
130
+ config: default
131
+ split: test
132
+ revision: 23d186750531a14a0357ca22cd92d712fd512ea0
133
+ metrics:
134
+ - type: map
135
+ value: 88.90396838907245
136
+ - type: mrr
137
+ value: 90.90932539682541
138
+ - task:
139
+ type: Retrieval
140
+ dataset:
141
+ name: MTEB CmedqaRetrieval
142
+ type: C-MTEB/CmedqaRetrieval
143
+ config: default
144
+ split: dev
145
+ revision: cd540c506dae1cf9e9a59c3e06f42030d54e7301
146
+ metrics:
147
+ - type: map_at_1
148
+ value: 26.875
149
+ - type: map_at_10
150
+ value: 39.995999999999995
151
+ - type: map_at_100
152
+ value: 41.899
153
+ - type: map_at_1000
154
+ value: 42.0
155
+ - type: map_at_3
156
+ value: 35.414
157
+ - type: map_at_5
158
+ value: 38.019
159
+ - type: mrr_at_1
160
+ value: 40.635
161
+ - type: mrr_at_10
162
+ value: 48.827
163
+ - type: mrr_at_100
164
+ value: 49.805
165
+ - type: mrr_at_1000
166
+ value: 49.845
167
+ - type: mrr_at_3
168
+ value: 46.145
169
+ - type: mrr_at_5
170
+ value: 47.693999999999996
171
+ - type: ndcg_at_1
172
+ value: 40.635
173
+ - type: ndcg_at_10
174
+ value: 46.78
175
+ - type: ndcg_at_100
176
+ value: 53.986999999999995
177
+ - type: ndcg_at_1000
178
+ value: 55.684
179
+ - type: ndcg_at_3
180
+ value: 41.018
181
+ - type: ndcg_at_5
182
+ value: 43.559
183
+ - type: precision_at_1
184
+ value: 40.635
185
+ - type: precision_at_10
186
+ value: 10.427999999999999
187
+ - type: precision_at_100
188
+ value: 1.625
189
+ - type: precision_at_1000
190
+ value: 0.184
191
+ - type: precision_at_3
192
+ value: 23.139000000000003
193
+ - type: precision_at_5
194
+ value: 17.004
195
+ - type: recall_at_1
196
+ value: 26.875
197
+ - type: recall_at_10
198
+ value: 57.887
199
+ - type: recall_at_100
200
+ value: 87.408
201
+ - type: recall_at_1000
202
+ value: 98.721
203
+ - type: recall_at_3
204
+ value: 40.812
205
+ - type: recall_at_5
206
+ value: 48.397
207
+ - task:
208
+ type: PairClassification
209
+ dataset:
210
+ name: MTEB Cmnli
211
+ type: C-MTEB/CMNLI
212
+ config: default
213
+ split: validation
214
+ revision: 41bc36f332156f7adc9e38f53777c959b2ae9766
215
+ metrics:
216
+ - type: cos_sim_accuracy
217
+ value: 83.43956704750451
218
+ - type: cos_sim_ap
219
+ value: 90.49172854352659
220
+ - type: cos_sim_f1
221
+ value: 84.28475486903963
222
+ - type: cos_sim_precision
223
+ value: 80.84603822203135
224
+ - type: cos_sim_recall
225
+ value: 88.02899228431144
226
+ - type: dot_accuracy
227
+ value: 83.43956704750451
228
+ - type: dot_ap
229
+ value: 90.46317132695233
230
+ - type: dot_f1
231
+ value: 84.28794294628929
232
+ - type: dot_precision
233
+ value: 80.51948051948052
234
+ - type: dot_recall
235
+ value: 88.4264671498714
236
+ - type: euclidean_accuracy
237
+ value: 83.43956704750451
238
+ - type: euclidean_ap
239
+ value: 90.49171785256486
240
+ - type: euclidean_f1
241
+ value: 84.28235820561584
242
+ - type: euclidean_precision
243
+ value: 80.8022308022308
244
+ - type: euclidean_recall
245
+ value: 88.07575403320084
246
+ - type: manhattan_accuracy
247
+ value: 83.55983162958509
248
+ - type: manhattan_ap
249
+ value: 90.48046779812815
250
+ - type: manhattan_f1
251
+ value: 84.45354259069714
252
+ - type: manhattan_precision
253
+ value: 82.21877767936226
254
+ - type: manhattan_recall
255
+ value: 86.81318681318682
256
+ - type: max_accuracy
257
+ value: 83.55983162958509
258
+ - type: max_ap
259
+ value: 90.49172854352659
260
+ - type: max_f1
261
+ value: 84.45354259069714
262
+ - task:
263
+ type: Retrieval
264
+ dataset:
265
+ name: MTEB CovidRetrieval
266
+ type: C-MTEB/CovidRetrieval
267
+ config: default
268
+ split: dev
269
+ revision: 1271c7809071a13532e05f25fb53511ffce77117
270
+ metrics:
271
+ - type: map_at_1
272
+ value: 68.54599999999999
273
+ - type: map_at_10
274
+ value: 77.62400000000001
275
+ - type: map_at_100
276
+ value: 77.886
277
+ - type: map_at_1000
278
+ value: 77.89
279
+ - type: map_at_3
280
+ value: 75.966
281
+ - type: map_at_5
282
+ value: 76.995
283
+ - type: mrr_at_1
284
+ value: 68.915
285
+ - type: mrr_at_10
286
+ value: 77.703
287
+ - type: mrr_at_100
288
+ value: 77.958
289
+ - type: mrr_at_1000
290
+ value: 77.962
291
+ - type: mrr_at_3
292
+ value: 76.08
293
+ - type: mrr_at_5
294
+ value: 77.118
295
+ - type: ndcg_at_1
296
+ value: 68.809
297
+ - type: ndcg_at_10
298
+ value: 81.563
299
+ - type: ndcg_at_100
300
+ value: 82.758
301
+ - type: ndcg_at_1000
302
+ value: 82.864
303
+ - type: ndcg_at_3
304
+ value: 78.29
305
+ - type: ndcg_at_5
306
+ value: 80.113
307
+ - type: precision_at_1
308
+ value: 68.809
309
+ - type: precision_at_10
310
+ value: 9.463000000000001
311
+ - type: precision_at_100
312
+ value: 1.001
313
+ - type: precision_at_1000
314
+ value: 0.101
315
+ - type: precision_at_3
316
+ value: 28.486
317
+ - type: precision_at_5
318
+ value: 18.019
319
+ - type: recall_at_1
320
+ value: 68.54599999999999
321
+ - type: recall_at_10
322
+ value: 93.625
323
+ - type: recall_at_100
324
+ value: 99.05199999999999
325
+ - type: recall_at_1000
326
+ value: 99.895
327
+ - type: recall_at_3
328
+ value: 84.879
329
+ - type: recall_at_5
330
+ value: 89.252
331
+ - task:
332
+ type: Retrieval
333
+ dataset:
334
+ name: MTEB DuRetrieval
335
+ type: C-MTEB/DuRetrieval
336
+ config: default
337
+ split: dev
338
+ revision: a1a333e290fe30b10f3f56498e3a0d911a693ced
339
+ metrics:
340
+ - type: map_at_1
341
+ value: 25.653
342
+ - type: map_at_10
343
+ value: 79.105
344
+ - type: map_at_100
345
+ value: 81.902
346
+ - type: map_at_1000
347
+ value: 81.947
348
+ - type: map_at_3
349
+ value: 54.54599999999999
350
+ - type: map_at_5
351
+ value: 69.226
352
+ - type: mrr_at_1
353
+ value: 89.35
354
+ - type: mrr_at_10
355
+ value: 92.69
356
+ - type: mrr_at_100
357
+ value: 92.77
358
+ - type: mrr_at_1000
359
+ value: 92.774
360
+ - type: mrr_at_3
361
+ value: 92.425
362
+ - type: mrr_at_5
363
+ value: 92.575
364
+ - type: ndcg_at_1
365
+ value: 89.35
366
+ - type: ndcg_at_10
367
+ value: 86.55199999999999
368
+ - type: ndcg_at_100
369
+ value: 89.35300000000001
370
+ - type: ndcg_at_1000
371
+ value: 89.782
372
+ - type: ndcg_at_3
373
+ value: 85.392
374
+ - type: ndcg_at_5
375
+ value: 84.5
376
+ - type: precision_at_1
377
+ value: 89.35
378
+ - type: precision_at_10
379
+ value: 41.589999999999996
380
+ - type: precision_at_100
381
+ value: 4.781
382
+ - type: precision_at_1000
383
+ value: 0.488
384
+ - type: precision_at_3
385
+ value: 76.683
386
+ - type: precision_at_5
387
+ value: 65.06
388
+ - type: recall_at_1
389
+ value: 25.653
390
+ - type: recall_at_10
391
+ value: 87.64999999999999
392
+ - type: recall_at_100
393
+ value: 96.858
394
+ - type: recall_at_1000
395
+ value: 99.13300000000001
396
+ - type: recall_at_3
397
+ value: 56.869
398
+ - type: recall_at_5
399
+ value: 74.024
400
+ - task:
401
+ type: Retrieval
402
+ dataset:
403
+ name: MTEB EcomRetrieval
404
+ type: C-MTEB/EcomRetrieval
405
+ config: default
406
+ split: dev
407
+ revision: 687de13dc7294d6fd9be10c6945f9e8fec8166b9
408
+ metrics:
409
+ - type: map_at_1
410
+ value: 52.1
411
+ - type: map_at_10
412
+ value: 62.629999999999995
413
+ - type: map_at_100
414
+ value: 63.117000000000004
415
+ - type: map_at_1000
416
+ value: 63.134
417
+ - type: map_at_3
418
+ value: 60.267
419
+ - type: map_at_5
420
+ value: 61.777
421
+ - type: mrr_at_1
422
+ value: 52.1
423
+ - type: mrr_at_10
424
+ value: 62.629999999999995
425
+ - type: mrr_at_100
426
+ value: 63.117000000000004
427
+ - type: mrr_at_1000
428
+ value: 63.134
429
+ - type: mrr_at_3
430
+ value: 60.267
431
+ - type: mrr_at_5
432
+ value: 61.777
433
+ - type: ndcg_at_1
434
+ value: 52.1
435
+ - type: ndcg_at_10
436
+ value: 67.596
437
+ - type: ndcg_at_100
438
+ value: 69.95
439
+ - type: ndcg_at_1000
440
+ value: 70.33500000000001
441
+ - type: ndcg_at_3
442
+ value: 62.82600000000001
443
+ - type: ndcg_at_5
444
+ value: 65.546
445
+ - type: precision_at_1
446
+ value: 52.1
447
+ - type: precision_at_10
448
+ value: 8.309999999999999
449
+ - type: precision_at_100
450
+ value: 0.941
451
+ - type: precision_at_1000
452
+ value: 0.097
453
+ - type: precision_at_3
454
+ value: 23.400000000000002
455
+ - type: precision_at_5
456
+ value: 15.36
457
+ - type: recall_at_1
458
+ value: 52.1
459
+ - type: recall_at_10
460
+ value: 83.1
461
+ - type: recall_at_100
462
+ value: 94.1
463
+ - type: recall_at_1000
464
+ value: 97.0
465
+ - type: recall_at_3
466
+ value: 70.19999999999999
467
+ - type: recall_at_5
468
+ value: 76.8
469
+ - task:
470
+ type: Classification
471
+ dataset:
472
+ name: MTEB IFlyTek
473
+ type: C-MTEB/IFlyTek-classification
474
+ config: default
475
+ split: validation
476
+ revision: 421605374b29664c5fc098418fe20ada9bd55f8a
477
+ metrics:
478
+ - type: accuracy
479
+ value: 51.773759138130046
480
+ - type: f1
481
+ value: 40.341407912920054
482
+ - task:
483
+ type: Classification
484
+ dataset:
485
+ name: MTEB JDReview
486
+ type: C-MTEB/JDReview-classification
487
+ config: default
488
+ split: test
489
+ revision: b7c64bd89eb87f8ded463478346f76731f07bf8b
490
+ metrics:
491
+ - type: accuracy
492
+ value: 86.69793621013133
493
+ - type: ap
494
+ value: 55.46718958939327
495
+ - type: f1
496
+ value: 81.48228915952436
497
+ - task:
498
+ type: STS
499
+ dataset:
500
+ name: MTEB LCQMC
501
+ type: C-MTEB/LCQMC
502
+ config: default
503
+ split: test
504
+ revision: 17f9b096f80380fce5ed12a9be8be7784b337daf
505
+ metrics:
506
+ - type: cos_sim_pearson
507
+ value: 71.1397780205448
508
+ - type: cos_sim_spearman
509
+ value: 78.17368193033309
510
+ - type: euclidean_pearson
511
+ value: 77.4849177602368
512
+ - type: euclidean_spearman
513
+ value: 78.17369079663212
514
+ - type: manhattan_pearson
515
+ value: 77.47344305182406
516
+ - type: manhattan_spearman
517
+ value: 78.16454335155387
518
+ - task:
519
+ type: Reranking
520
+ dataset:
521
+ name: MTEB MMarcoReranking
522
+ type: C-MTEB/Mmarco-reranking
523
+ config: default
524
+ split: dev
525
+ revision: 8e0c766dbe9e16e1d221116a3f36795fbade07f6
526
+ metrics:
527
+ - type: map
528
+ value: 27.76160559006673
529
+ - type: mrr
530
+ value: 28.02420634920635
531
+ - task:
532
+ type: Retrieval
533
+ dataset:
534
+ name: MTEB MMarcoRetrieval
535
+ type: C-MTEB/MMarcoRetrieval
536
+ config: default
537
+ split: dev
538
+ revision: 539bbde593d947e2a124ba72651aafc09eb33fc2
539
+ metrics:
540
+ - type: map_at_1
541
+ value: 65.661
542
+ - type: map_at_10
543
+ value: 74.752
544
+ - type: map_at_100
545
+ value: 75.091
546
+ - type: map_at_1000
547
+ value: 75.104
548
+ - type: map_at_3
549
+ value: 72.997
550
+ - type: map_at_5
551
+ value: 74.119
552
+ - type: mrr_at_1
553
+ value: 67.923
554
+ - type: mrr_at_10
555
+ value: 75.376
556
+ - type: mrr_at_100
557
+ value: 75.673
558
+ - type: mrr_at_1000
559
+ value: 75.685
560
+ - type: mrr_at_3
561
+ value: 73.856
562
+ - type: mrr_at_5
563
+ value: 74.82799999999999
564
+ - type: ndcg_at_1
565
+ value: 67.923
566
+ - type: ndcg_at_10
567
+ value: 78.424
568
+ - type: ndcg_at_100
569
+ value: 79.95100000000001
570
+ - type: ndcg_at_1000
571
+ value: 80.265
572
+ - type: ndcg_at_3
573
+ value: 75.101
574
+ - type: ndcg_at_5
575
+ value: 76.992
576
+ - type: precision_at_1
577
+ value: 67.923
578
+ - type: precision_at_10
579
+ value: 9.474
580
+ - type: precision_at_100
581
+ value: 1.023
582
+ - type: precision_at_1000
583
+ value: 0.105
584
+ - type: precision_at_3
585
+ value: 28.319
586
+ - type: precision_at_5
587
+ value: 17.986
588
+ - type: recall_at_1
589
+ value: 65.661
590
+ - type: recall_at_10
591
+ value: 89.09899999999999
592
+ - type: recall_at_100
593
+ value: 96.023
594
+ - type: recall_at_1000
595
+ value: 98.455
596
+ - type: recall_at_3
597
+ value: 80.314
598
+ - type: recall_at_5
599
+ value: 84.81
600
+ - task:
601
+ type: Classification
602
+ dataset:
603
+ name: MTEB MassiveIntentClassification (zh-CN)
604
+ type: mteb/amazon_massive_intent
605
+ config: zh-CN
606
+ split: test
607
+ revision: 31efe3c427b0bae9c22cbb560b8f15491cc6bed7
608
+ metrics:
609
+ - type: accuracy
610
+ value: 75.86751849361131
611
+ - type: f1
612
+ value: 73.04918450508
613
+ - task:
614
+ type: Classification
615
+ dataset:
616
+ name: MTEB MassiveScenarioClassification (zh-CN)
617
+ type: mteb/amazon_massive_scenario
618
+ config: zh-CN
619
+ split: test
620
+ revision: 7d571f92784cd94a019292a1f45445077d0ef634
621
+ metrics:
622
+ - type: accuracy
623
+ value: 78.4364492266308
624
+ - type: f1
625
+ value: 78.120686034844
626
+ - task:
627
+ type: Retrieval
628
+ dataset:
629
+ name: MTEB MedicalRetrieval
630
+ type: C-MTEB/MedicalRetrieval
631
+ config: default
632
+ split: dev
633
+ revision: 2039188fb5800a9803ba5048df7b76e6fb151fc6
634
+ metrics:
635
+ - type: map_at_1
636
+ value: 55.00000000000001
637
+ - type: map_at_10
638
+ value: 61.06399999999999
639
+ - type: map_at_100
640
+ value: 61.622
641
+ - type: map_at_1000
642
+ value: 61.663000000000004
643
+ - type: map_at_3
644
+ value: 59.583
645
+ - type: map_at_5
646
+ value: 60.373
647
+ - type: mrr_at_1
648
+ value: 55.2
649
+ - type: mrr_at_10
650
+ value: 61.168
651
+ - type: mrr_at_100
652
+ value: 61.726000000000006
653
+ - type: mrr_at_1000
654
+ value: 61.767
655
+ - type: mrr_at_3
656
+ value: 59.683
657
+ - type: mrr_at_5
658
+ value: 60.492999999999995
659
+ - type: ndcg_at_1
660
+ value: 55.00000000000001
661
+ - type: ndcg_at_10
662
+ value: 64.098
663
+ - type: ndcg_at_100
664
+ value: 67.05
665
+ - type: ndcg_at_1000
666
+ value: 68.262
667
+ - type: ndcg_at_3
668
+ value: 61.00600000000001
669
+ - type: ndcg_at_5
670
+ value: 62.439
671
+ - type: precision_at_1
672
+ value: 55.00000000000001
673
+ - type: precision_at_10
674
+ value: 7.37
675
+ - type: precision_at_100
676
+ value: 0.881
677
+ - type: precision_at_1000
678
+ value: 0.098
679
+ - type: precision_at_3
680
+ value: 21.7
681
+ - type: precision_at_5
682
+ value: 13.719999999999999
683
+ - type: recall_at_1
684
+ value: 55.00000000000001
685
+ - type: recall_at_10
686
+ value: 73.7
687
+ - type: recall_at_100
688
+ value: 88.1
689
+ - type: recall_at_1000
690
+ value: 97.8
691
+ - type: recall_at_3
692
+ value: 65.10000000000001
693
+ - type: recall_at_5
694
+ value: 68.60000000000001
695
+ - task:
696
+ type: Classification
697
+ dataset:
698
+ name: MTEB MultilingualSentiment
699
+ type: C-MTEB/MultilingualSentiment-classification
700
+ config: default
701
+ split: validation
702
+ revision: 46958b007a63fdbf239b7672c25d0bea67b5ea1a
703
+ metrics:
704
+ - type: accuracy
705
+ value: 77.52666666666667
706
+ - type: f1
707
+ value: 77.49784731367215
708
+ - task:
709
+ type: PairClassification
710
+ dataset:
711
+ name: MTEB Ocnli
712
+ type: C-MTEB/OCNLI
713
+ config: default
714
+ split: validation
715
+ revision: 66e76a618a34d6d565d5538088562851e6daa7ec
716
+ metrics:
717
+ - type: cos_sim_accuracy
718
+ value: 81.10449377368705
719
+ - type: cos_sim_ap
720
+ value: 85.17742765935606
721
+ - type: cos_sim_f1
722
+ value: 83.00094966761633
723
+ - type: cos_sim_precision
724
+ value: 75.40983606557377
725
+ - type: cos_sim_recall
726
+ value: 92.29144667370645
727
+ - type: dot_accuracy
728
+ value: 81.10449377368705
729
+ - type: dot_ap
730
+ value: 85.17143850809614
731
+ - type: dot_f1
732
+ value: 83.01707779886148
733
+ - type: dot_precision
734
+ value: 75.36606373815677
735
+ - type: dot_recall
736
+ value: 92.39704329461456
737
+ - type: euclidean_accuracy
738
+ value: 81.10449377368705
739
+ - type: euclidean_ap
740
+ value: 85.17856775343333
741
+ - type: euclidean_f1
742
+ value: 83.00094966761633
743
+ - type: euclidean_precision
744
+ value: 75.40983606557377
745
+ - type: euclidean_recall
746
+ value: 92.29144667370645
747
+ - type: manhattan_accuracy
748
+ value: 81.05035192203573
749
+ - type: manhattan_ap
750
+ value: 85.14464459395809
751
+ - type: manhattan_f1
752
+ value: 82.96155671570953
753
+ - type: manhattan_precision
754
+ value: 75.3448275862069
755
+ - type: manhattan_recall
756
+ value: 92.29144667370645
757
+ - type: max_accuracy
758
+ value: 81.10449377368705
759
+ - type: max_ap
760
+ value: 85.17856775343333
761
+ - type: max_f1
762
+ value: 83.01707779886148
763
+ - task:
764
+ type: Classification
765
+ dataset:
766
+ name: MTEB OnlineShopping
767
+ type: C-MTEB/OnlineShopping-classification
768
+ config: default
769
+ split: test
770
+ revision: e610f2ebd179a8fda30ae534c3878750a96db120
771
+ metrics:
772
+ - type: accuracy
773
+ value: 93.71000000000001
774
+ - type: ap
775
+ value: 91.83202232349356
776
+ - type: f1
777
+ value: 93.69900560334331
778
+ - task:
779
+ type: STS
780
+ dataset:
781
+ name: MTEB PAWSX
782
+ type: C-MTEB/PAWSX
783
+ config: default
784
+ split: test
785
+ revision: 9c6a90e430ac22b5779fb019a23e820b11a8b5e1
786
+ metrics:
787
+ - type: cos_sim_pearson
788
+ value: 39.175047651512415
789
+ - type: cos_sim_spearman
790
+ value: 45.51434675777896
791
+ - type: euclidean_pearson
792
+ value: 44.864110004132286
793
+ - type: euclidean_spearman
794
+ value: 45.516433048896076
795
+ - type: manhattan_pearson
796
+ value: 44.87153627706517
797
+ - type: manhattan_spearman
798
+ value: 45.52862617925012
799
+ - task:
800
+ type: STS
801
+ dataset:
802
+ name: MTEB QBQTC
803
+ type: C-MTEB/QBQTC
804
+ config: default
805
+ split: test
806
+ revision: 790b0510dc52b1553e8c49f3d2afb48c0e5c48b7
807
+ metrics:
808
+ - type: cos_sim_pearson
809
+ value: 34.249579701429084
810
+ - type: cos_sim_spearman
811
+ value: 37.30903127368978
812
+ - type: euclidean_pearson
813
+ value: 35.129438425253355
814
+ - type: euclidean_spearman
815
+ value: 37.308544018709085
816
+ - type: manhattan_pearson
817
+ value: 35.08936153503652
818
+ - type: manhattan_spearman
819
+ value: 37.25582901077839
820
+ - task:
821
+ type: STS
822
+ dataset:
823
+ name: MTEB STS22 (zh)
824
+ type: mteb/sts22-crosslingual-sts
825
+ config: zh
826
+ split: test
827
+ revision: eea2b4fe26a775864c896887d910b76a8098ad3f
828
+ metrics:
829
+ - type: cos_sim_pearson
830
+ value: 61.29309637460004
831
+ - type: cos_sim_spearman
832
+ value: 65.85136090376717
833
+ - type: euclidean_pearson
834
+ value: 64.04783990953557
835
+ - type: euclidean_spearman
836
+ value: 65.85036859610366
837
+ - type: manhattan_pearson
838
+ value: 63.995852552712186
839
+ - type: manhattan_spearman
840
+ value: 65.86508416749417
841
+ - task:
842
+ type: STS
843
+ dataset:
844
+ name: MTEB STSB
845
+ type: C-MTEB/STSB
846
+ config: default
847
+ split: test
848
+ revision: 0cde68302b3541bb8b3c340dc0644b0b745b3dc0
849
+ metrics:
850
+ - type: cos_sim_pearson
851
+ value: 81.5595940455587
852
+ - type: cos_sim_spearman
853
+ value: 82.72654634579749
854
+ - type: euclidean_pearson
855
+ value: 82.4892721061365
856
+ - type: euclidean_spearman
857
+ value: 82.72678504228253
858
+ - type: manhattan_pearson
859
+ value: 82.4770861422454
860
+ - type: manhattan_spearman
861
+ value: 82.71137469783162
862
+ - task:
863
+ type: Reranking
864
+ dataset:
865
+ name: MTEB T2Reranking
866
+ type: C-MTEB/T2Reranking
867
+ config: default
868
+ split: dev
869
+ revision: 76631901a18387f85eaa53e5450019b87ad58ef9
870
+ metrics:
871
+ - type: map
872
+ value: 66.6159547610527
873
+ - type: mrr
874
+ value: 76.35739406347057
875
+ - task:
876
+ type: Retrieval
877
+ dataset:
878
+ name: MTEB T2Retrieval
879
+ type: C-MTEB/T2Retrieval
880
+ config: default
881
+ split: dev
882
+ revision: 8731a845f1bf500a4f111cf1070785c793d10e64
883
+ metrics:
884
+ - type: map_at_1
885
+ value: 27.878999999999998
886
+ - type: map_at_10
887
+ value: 77.517
888
+ - type: map_at_100
889
+ value: 81.139
890
+ - type: map_at_1000
891
+ value: 81.204
892
+ - type: map_at_3
893
+ value: 54.728
894
+ - type: map_at_5
895
+ value: 67.128
896
+ - type: mrr_at_1
897
+ value: 90.509
898
+ - type: mrr_at_10
899
+ value: 92.964
900
+ - type: mrr_at_100
901
+ value: 93.045
902
+ - type: mrr_at_1000
903
+ value: 93.048
904
+ - type: mrr_at_3
905
+ value: 92.551
906
+ - type: mrr_at_5
907
+ value: 92.81099999999999
908
+ - type: ndcg_at_1
909
+ value: 90.509
910
+ - type: ndcg_at_10
911
+ value: 85.075
912
+ - type: ndcg_at_100
913
+ value: 88.656
914
+ - type: ndcg_at_1000
915
+ value: 89.25699999999999
916
+ - type: ndcg_at_3
917
+ value: 86.58200000000001
918
+ - type: ndcg_at_5
919
+ value: 85.138
920
+ - type: precision_at_1
921
+ value: 90.509
922
+ - type: precision_at_10
923
+ value: 42.05
924
+ - type: precision_at_100
925
+ value: 5.013999999999999
926
+ - type: precision_at_1000
927
+ value: 0.516
928
+ - type: precision_at_3
929
+ value: 75.551
930
+ - type: precision_at_5
931
+ value: 63.239999999999995
932
+ - type: recall_at_1
933
+ value: 27.878999999999998
934
+ - type: recall_at_10
935
+ value: 83.941
936
+ - type: recall_at_100
937
+ value: 95.568
938
+ - type: recall_at_1000
939
+ value: 98.55000000000001
940
+ - type: recall_at_3
941
+ value: 56.374
942
+ - type: recall_at_5
943
+ value: 70.435
944
+ - task:
945
+ type: Classification
946
+ dataset:
947
+ name: MTEB TNews
948
+ type: C-MTEB/TNews-classification
949
+ config: default
950
+ split: validation
951
+ revision: 317f262bf1e6126357bbe89e875451e4b0938fe4
952
+ metrics:
953
+ - type: accuracy
954
+ value: 53.687
955
+ - type: f1
956
+ value: 51.86911933364655
957
+ - task:
958
+ type: Clustering
959
+ dataset:
960
+ name: MTEB ThuNewsClusteringP2P
961
+ type: C-MTEB/ThuNewsClusteringP2P
962
+ config: default
963
+ split: test
964
+ revision: 5798586b105c0434e4f0fe5e767abe619442cf93
965
+ metrics:
966
+ - type: v_measure
967
+ value: 74.65887489872564
968
+ - task:
969
+ type: Clustering
970
+ dataset:
971
+ name: MTEB ThuNewsClusteringS2S
972
+ type: C-MTEB/ThuNewsClusteringS2S
973
+ config: default
974
+ split: test
975
+ revision: 8a8b2caeda43f39e13c4bc5bea0f8a667896e10d
976
+ metrics:
977
+ - type: v_measure
978
+ value: 69.00410995984436
979
+ - task:
980
+ type: Retrieval
981
+ dataset:
982
+ name: MTEB VideoRetrieval
983
+ type: C-MTEB/VideoRetrieval
984
+ config: default
985
+ split: dev
986
+ revision: 58c2597a5943a2ba48f4668c3b90d796283c5639
987
+ metrics:
988
+ - type: map_at_1
989
+ value: 59.4
990
+ - type: map_at_10
991
+ value: 69.214
992
+ - type: map_at_100
993
+ value: 69.72699999999999
994
+ - type: map_at_1000
995
+ value: 69.743
996
+ - type: map_at_3
997
+ value: 67.717
998
+ - type: map_at_5
999
+ value: 68.782
1000
+ - type: mrr_at_1
1001
+ value: 59.4
1002
+ - type: mrr_at_10
1003
+ value: 69.214
1004
+ - type: mrr_at_100
1005
+ value: 69.72699999999999
1006
+ - type: mrr_at_1000
1007
+ value: 69.743
1008
+ - type: mrr_at_3
1009
+ value: 67.717
1010
+ - type: mrr_at_5
1011
+ value: 68.782
1012
+ - type: ndcg_at_1
1013
+ value: 59.4
1014
+ - type: ndcg_at_10
1015
+ value: 73.32300000000001
1016
+ - type: ndcg_at_100
1017
+ value: 75.591
1018
+ - type: ndcg_at_1000
1019
+ value: 75.98700000000001
1020
+ - type: ndcg_at_3
1021
+ value: 70.339
1022
+ - type: ndcg_at_5
1023
+ value: 72.246
1024
+ - type: precision_at_1
1025
+ value: 59.4
1026
+ - type: precision_at_10
1027
+ value: 8.59
1028
+ - type: precision_at_100
1029
+ value: 0.96
1030
+ - type: precision_at_1000
1031
+ value: 0.099
1032
+ - type: precision_at_3
1033
+ value: 25.967000000000002
1034
+ - type: precision_at_5
1035
+ value: 16.5
1036
+ - type: recall_at_1
1037
+ value: 59.4
1038
+ - type: recall_at_10
1039
+ value: 85.9
1040
+ - type: recall_at_100
1041
+ value: 96.0
1042
+ - type: recall_at_1000
1043
+ value: 99.1
1044
+ - type: recall_at_3
1045
+ value: 77.9
1046
+ - type: recall_at_5
1047
+ value: 82.5
1048
+ - task:
1049
+ type: Classification
1050
+ dataset:
1051
+ name: MTEB Waimai
1052
+ type: C-MTEB/waimai-classification
1053
+ config: default
1054
+ split: test
1055
+ revision: 339287def212450dcaa9df8c22bf93e9980c7023
1056
+ metrics:
1057
+ - type: accuracy
1058
+ value: 88.53
1059
+ - type: ap
1060
+ value: 73.56216166534062
1061
+ - type: f1
1062
+ value: 87.06093694294485
1063
+ ---
1064
+
1065
+ # bnightning/acge_text_embedding-Q4_K_M-GGUF
1066
+ This model was converted to GGUF format from [`aspire/acge_text_embedding`](https://huggingface.co/aspire/acge_text_embedding) using llama.cpp via the ggml.ai's [GGUF-my-repo](https://huggingface.co/spaces/ggml-org/gguf-my-repo) space.
1067
+ Refer to the [original model card](https://huggingface.co/aspire/acge_text_embedding) for more details on the model.
1068
+
1069
+ ## Use with llama.cpp
1070
+ Install llama.cpp through brew (works on Mac and Linux)
1071
+
1072
+ ```bash
1073
+ brew install llama.cpp
1074
+
1075
+ ```
1076
+ Invoke the llama.cpp server or the CLI.
1077
+
1078
+ ### CLI:
1079
+ ```bash
1080
+ llama-cli --hf-repo bnightning/acge_text_embedding-Q4_K_M-GGUF --hf-file acge_text_embedding-q4_k_m.gguf -p "The meaning to life and the universe is"
1081
+ ```
1082
+
1083
+ ### Server:
1084
+ ```bash
1085
+ llama-server --hf-repo bnightning/acge_text_embedding-Q4_K_M-GGUF --hf-file acge_text_embedding-q4_k_m.gguf -c 2048
1086
+ ```
1087
+
1088
+ Note: You can also use this checkpoint directly through the [usage steps](https://github.com/ggerganov/llama.cpp?tab=readme-ov-file#usage) listed in the Llama.cpp repo as well.
1089
+
1090
+ Step 1: Clone llama.cpp from GitHub.
1091
+ ```
1092
+ git clone https://github.com/ggerganov/llama.cpp
1093
+ ```
1094
+
1095
+ Step 2: Move into the llama.cpp folder and build it with `LLAMA_CURL=1` flag along with other hardware-specific flags (for ex: LLAMA_CUDA=1 for Nvidia GPUs on Linux).
1096
+ ```
1097
+ cd llama.cpp && LLAMA_CURL=1 make
1098
+ ```
1099
+
1100
+ Step 3: Run inference through the main binary.
1101
+ ```
1102
+ ./llama-cli --hf-repo bnightning/acge_text_embedding-Q4_K_M-GGUF --hf-file acge_text_embedding-q4_k_m.gguf -p "The meaning to life and the universe is"
1103
+ ```
1104
+ or
1105
+ ```
1106
+ ./llama-server --hf-repo bnightning/acge_text_embedding-Q4_K_M-GGUF --hf-file acge_text_embedding-q4_k_m.gguf -c 2048
1107
+ ```