annofung commited on
Commit
2bfb3a8
·
verified ·
1 Parent(s): f0de533

Upload README.md with huggingface_hub

Browse files
Files changed (1) hide show
  1. README.md +1112 -0
README.md ADDED
@@ -0,0 +1,1112 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ tags:
3
+ - sentence-transformers
4
+ - feature-extraction
5
+ - sentence-similarity
6
+ - mteb
7
+ - RAG
8
+ - llama-cpp
9
+ - gguf-my-repo
10
+ license: apache-2.0
11
+ language:
12
+ - zh
13
+ - en
14
+ pipeline_tag: feature-extraction
15
+ base_model: DMetaSoul/Dmeta-embedding-zh
16
+ model-index:
17
+ - name: Dmeta-embedding
18
+ results:
19
+ - task:
20
+ type: STS
21
+ dataset:
22
+ name: MTEB AFQMC
23
+ type: C-MTEB/AFQMC
24
+ config: default
25
+ split: validation
26
+ revision: None
27
+ metrics:
28
+ - type: cos_sim_pearson
29
+ value: 65.60825224706932
30
+ - type: cos_sim_spearman
31
+ value: 71.12862586297193
32
+ - type: euclidean_pearson
33
+ value: 70.18130275750404
34
+ - type: euclidean_spearman
35
+ value: 71.12862586297193
36
+ - type: manhattan_pearson
37
+ value: 70.14470398075396
38
+ - type: manhattan_spearman
39
+ value: 71.05226975911737
40
+ - task:
41
+ type: STS
42
+ dataset:
43
+ name: MTEB ATEC
44
+ type: C-MTEB/ATEC
45
+ config: default
46
+ split: test
47
+ revision: None
48
+ metrics:
49
+ - type: cos_sim_pearson
50
+ value: 65.52386345655479
51
+ - type: cos_sim_spearman
52
+ value: 64.64245253181382
53
+ - type: euclidean_pearson
54
+ value: 73.20157662981914
55
+ - type: euclidean_spearman
56
+ value: 64.64245253178956
57
+ - type: manhattan_pearson
58
+ value: 73.22837571756348
59
+ - type: manhattan_spearman
60
+ value: 64.62632334391418
61
+ - task:
62
+ type: Classification
63
+ dataset:
64
+ name: MTEB AmazonReviewsClassification (zh)
65
+ type: mteb/amazon_reviews_multi
66
+ config: zh
67
+ split: test
68
+ revision: 1399c76144fd37290681b995c656ef9b2e06e26d
69
+ metrics:
70
+ - type: accuracy
71
+ value: 44.925999999999995
72
+ - type: f1
73
+ value: 42.82555191308971
74
+ - task:
75
+ type: STS
76
+ dataset:
77
+ name: MTEB BQ
78
+ type: C-MTEB/BQ
79
+ config: default
80
+ split: test
81
+ revision: None
82
+ metrics:
83
+ - type: cos_sim_pearson
84
+ value: 71.35236446393156
85
+ - type: cos_sim_spearman
86
+ value: 72.29629643702184
87
+ - type: euclidean_pearson
88
+ value: 70.94570179874498
89
+ - type: euclidean_spearman
90
+ value: 72.29629297226953
91
+ - type: manhattan_pearson
92
+ value: 70.84463025501125
93
+ - type: manhattan_spearman
94
+ value: 72.24527021975821
95
+ - task:
96
+ type: Clustering
97
+ dataset:
98
+ name: MTEB CLSClusteringP2P
99
+ type: C-MTEB/CLSClusteringP2P
100
+ config: default
101
+ split: test
102
+ revision: None
103
+ metrics:
104
+ - type: v_measure
105
+ value: 40.24232916894152
106
+ - task:
107
+ type: Clustering
108
+ dataset:
109
+ name: MTEB CLSClusteringS2S
110
+ type: C-MTEB/CLSClusteringS2S
111
+ config: default
112
+ split: test
113
+ revision: None
114
+ metrics:
115
+ - type: v_measure
116
+ value: 39.167806226929706
117
+ - task:
118
+ type: Reranking
119
+ dataset:
120
+ name: MTEB CMedQAv1
121
+ type: C-MTEB/CMedQAv1-reranking
122
+ config: default
123
+ split: test
124
+ revision: None
125
+ metrics:
126
+ - type: map
127
+ value: 88.48837920106357
128
+ - type: mrr
129
+ value: 90.36861111111111
130
+ - task:
131
+ type: Reranking
132
+ dataset:
133
+ name: MTEB CMedQAv2
134
+ type: C-MTEB/CMedQAv2-reranking
135
+ config: default
136
+ split: test
137
+ revision: None
138
+ metrics:
139
+ - type: map
140
+ value: 89.17878171657071
141
+ - type: mrr
142
+ value: 91.35805555555555
143
+ - task:
144
+ type: Retrieval
145
+ dataset:
146
+ name: MTEB CmedqaRetrieval
147
+ type: C-MTEB/CmedqaRetrieval
148
+ config: default
149
+ split: dev
150
+ revision: None
151
+ metrics:
152
+ - type: map_at_1
153
+ value: 25.751
154
+ - type: map_at_10
155
+ value: 38.946
156
+ - type: map_at_100
157
+ value: 40.855000000000004
158
+ - type: map_at_1000
159
+ value: 40.953
160
+ - type: map_at_3
161
+ value: 34.533
162
+ - type: map_at_5
163
+ value: 36.905
164
+ - type: mrr_at_1
165
+ value: 39.235
166
+ - type: mrr_at_10
167
+ value: 47.713
168
+ - type: mrr_at_100
169
+ value: 48.71
170
+ - type: mrr_at_1000
171
+ value: 48.747
172
+ - type: mrr_at_3
173
+ value: 45.086
174
+ - type: mrr_at_5
175
+ value: 46.498
176
+ - type: ndcg_at_1
177
+ value: 39.235
178
+ - type: ndcg_at_10
179
+ value: 45.831
180
+ - type: ndcg_at_100
181
+ value: 53.162
182
+ - type: ndcg_at_1000
183
+ value: 54.800000000000004
184
+ - type: ndcg_at_3
185
+ value: 40.188
186
+ - type: ndcg_at_5
187
+ value: 42.387
188
+ - type: precision_at_1
189
+ value: 39.235
190
+ - type: precision_at_10
191
+ value: 10.273
192
+ - type: precision_at_100
193
+ value: 1.627
194
+ - type: precision_at_1000
195
+ value: 0.183
196
+ - type: precision_at_3
197
+ value: 22.772000000000002
198
+ - type: precision_at_5
199
+ value: 16.524
200
+ - type: recall_at_1
201
+ value: 25.751
202
+ - type: recall_at_10
203
+ value: 57.411
204
+ - type: recall_at_100
205
+ value: 87.44
206
+ - type: recall_at_1000
207
+ value: 98.386
208
+ - type: recall_at_3
209
+ value: 40.416000000000004
210
+ - type: recall_at_5
211
+ value: 47.238
212
+ - task:
213
+ type: PairClassification
214
+ dataset:
215
+ name: MTEB Cmnli
216
+ type: C-MTEB/CMNLI
217
+ config: default
218
+ split: validation
219
+ revision: None
220
+ metrics:
221
+ - type: cos_sim_accuracy
222
+ value: 83.59591100420926
223
+ - type: cos_sim_ap
224
+ value: 90.65538153970263
225
+ - type: cos_sim_f1
226
+ value: 84.76466651795673
227
+ - type: cos_sim_precision
228
+ value: 81.04073363190446
229
+ - type: cos_sim_recall
230
+ value: 88.84732288987608
231
+ - type: dot_accuracy
232
+ value: 83.59591100420926
233
+ - type: dot_ap
234
+ value: 90.64355541781003
235
+ - type: dot_f1
236
+ value: 84.76466651795673
237
+ - type: dot_precision
238
+ value: 81.04073363190446
239
+ - type: dot_recall
240
+ value: 88.84732288987608
241
+ - type: euclidean_accuracy
242
+ value: 83.59591100420926
243
+ - type: euclidean_ap
244
+ value: 90.6547878194287
245
+ - type: euclidean_f1
246
+ value: 84.76466651795673
247
+ - type: euclidean_precision
248
+ value: 81.04073363190446
249
+ - type: euclidean_recall
250
+ value: 88.84732288987608
251
+ - type: manhattan_accuracy
252
+ value: 83.51172579675286
253
+ - type: manhattan_ap
254
+ value: 90.59941589844144
255
+ - type: manhattan_f1
256
+ value: 84.51827242524917
257
+ - type: manhattan_precision
258
+ value: 80.28613507258574
259
+ - type: manhattan_recall
260
+ value: 89.22141688099134
261
+ - type: max_accuracy
262
+ value: 83.59591100420926
263
+ - type: max_ap
264
+ value: 90.65538153970263
265
+ - type: max_f1
266
+ value: 84.76466651795673
267
+ - task:
268
+ type: Retrieval
269
+ dataset:
270
+ name: MTEB CovidRetrieval
271
+ type: C-MTEB/CovidRetrieval
272
+ config: default
273
+ split: dev
274
+ revision: None
275
+ metrics:
276
+ - type: map_at_1
277
+ value: 63.251000000000005
278
+ - type: map_at_10
279
+ value: 72.442
280
+ - type: map_at_100
281
+ value: 72.79299999999999
282
+ - type: map_at_1000
283
+ value: 72.80499999999999
284
+ - type: map_at_3
285
+ value: 70.293
286
+ - type: map_at_5
287
+ value: 71.571
288
+ - type: mrr_at_1
289
+ value: 63.541000000000004
290
+ - type: mrr_at_10
291
+ value: 72.502
292
+ - type: mrr_at_100
293
+ value: 72.846
294
+ - type: mrr_at_1000
295
+ value: 72.858
296
+ - type: mrr_at_3
297
+ value: 70.39
298
+ - type: mrr_at_5
299
+ value: 71.654
300
+ - type: ndcg_at_1
301
+ value: 63.541000000000004
302
+ - type: ndcg_at_10
303
+ value: 76.774
304
+ - type: ndcg_at_100
305
+ value: 78.389
306
+ - type: ndcg_at_1000
307
+ value: 78.678
308
+ - type: ndcg_at_3
309
+ value: 72.47
310
+ - type: ndcg_at_5
311
+ value: 74.748
312
+ - type: precision_at_1
313
+ value: 63.541000000000004
314
+ - type: precision_at_10
315
+ value: 9.115
316
+ - type: precision_at_100
317
+ value: 0.9860000000000001
318
+ - type: precision_at_1000
319
+ value: 0.101
320
+ - type: precision_at_3
321
+ value: 26.379
322
+ - type: precision_at_5
323
+ value: 16.965
324
+ - type: recall_at_1
325
+ value: 63.251000000000005
326
+ - type: recall_at_10
327
+ value: 90.253
328
+ - type: recall_at_100
329
+ value: 97.576
330
+ - type: recall_at_1000
331
+ value: 99.789
332
+ - type: recall_at_3
333
+ value: 78.635
334
+ - type: recall_at_5
335
+ value: 84.141
336
+ - task:
337
+ type: Retrieval
338
+ dataset:
339
+ name: MTEB DuRetrieval
340
+ type: C-MTEB/DuRetrieval
341
+ config: default
342
+ split: dev
343
+ revision: None
344
+ metrics:
345
+ - type: map_at_1
346
+ value: 23.597
347
+ - type: map_at_10
348
+ value: 72.411
349
+ - type: map_at_100
350
+ value: 75.58500000000001
351
+ - type: map_at_1000
352
+ value: 75.64800000000001
353
+ - type: map_at_3
354
+ value: 49.61
355
+ - type: map_at_5
356
+ value: 62.527
357
+ - type: mrr_at_1
358
+ value: 84.65
359
+ - type: mrr_at_10
360
+ value: 89.43900000000001
361
+ - type: mrr_at_100
362
+ value: 89.525
363
+ - type: mrr_at_1000
364
+ value: 89.529
365
+ - type: mrr_at_3
366
+ value: 89
367
+ - type: mrr_at_5
368
+ value: 89.297
369
+ - type: ndcg_at_1
370
+ value: 84.65
371
+ - type: ndcg_at_10
372
+ value: 81.47
373
+ - type: ndcg_at_100
374
+ value: 85.198
375
+ - type: ndcg_at_1000
376
+ value: 85.828
377
+ - type: ndcg_at_3
378
+ value: 79.809
379
+ - type: ndcg_at_5
380
+ value: 78.55
381
+ - type: precision_at_1
382
+ value: 84.65
383
+ - type: precision_at_10
384
+ value: 39.595
385
+ - type: precision_at_100
386
+ value: 4.707
387
+ - type: precision_at_1000
388
+ value: 0.485
389
+ - type: precision_at_3
390
+ value: 71.61699999999999
391
+ - type: precision_at_5
392
+ value: 60.45
393
+ - type: recall_at_1
394
+ value: 23.597
395
+ - type: recall_at_10
396
+ value: 83.34
397
+ - type: recall_at_100
398
+ value: 95.19800000000001
399
+ - type: recall_at_1000
400
+ value: 98.509
401
+ - type: recall_at_3
402
+ value: 52.744
403
+ - type: recall_at_5
404
+ value: 68.411
405
+ - task:
406
+ type: Retrieval
407
+ dataset:
408
+ name: MTEB EcomRetrieval
409
+ type: C-MTEB/EcomRetrieval
410
+ config: default
411
+ split: dev
412
+ revision: None
413
+ metrics:
414
+ - type: map_at_1
415
+ value: 53.1
416
+ - type: map_at_10
417
+ value: 63.359
418
+ - type: map_at_100
419
+ value: 63.9
420
+ - type: map_at_1000
421
+ value: 63.909000000000006
422
+ - type: map_at_3
423
+ value: 60.95
424
+ - type: map_at_5
425
+ value: 62.305
426
+ - type: mrr_at_1
427
+ value: 53.1
428
+ - type: mrr_at_10
429
+ value: 63.359
430
+ - type: mrr_at_100
431
+ value: 63.9
432
+ - type: mrr_at_1000
433
+ value: 63.909000000000006
434
+ - type: mrr_at_3
435
+ value: 60.95
436
+ - type: mrr_at_5
437
+ value: 62.305
438
+ - type: ndcg_at_1
439
+ value: 53.1
440
+ - type: ndcg_at_10
441
+ value: 68.418
442
+ - type: ndcg_at_100
443
+ value: 70.88499999999999
444
+ - type: ndcg_at_1000
445
+ value: 71.135
446
+ - type: ndcg_at_3
447
+ value: 63.50599999999999
448
+ - type: ndcg_at_5
449
+ value: 65.92
450
+ - type: precision_at_1
451
+ value: 53.1
452
+ - type: precision_at_10
453
+ value: 8.43
454
+ - type: precision_at_100
455
+ value: 0.955
456
+ - type: precision_at_1000
457
+ value: 0.098
458
+ - type: precision_at_3
459
+ value: 23.633000000000003
460
+ - type: precision_at_5
461
+ value: 15.340000000000002
462
+ - type: recall_at_1
463
+ value: 53.1
464
+ - type: recall_at_10
465
+ value: 84.3
466
+ - type: recall_at_100
467
+ value: 95.5
468
+ - type: recall_at_1000
469
+ value: 97.5
470
+ - type: recall_at_3
471
+ value: 70.89999999999999
472
+ - type: recall_at_5
473
+ value: 76.7
474
+ - task:
475
+ type: Classification
476
+ dataset:
477
+ name: MTEB IFlyTek
478
+ type: C-MTEB/IFlyTek-classification
479
+ config: default
480
+ split: validation
481
+ revision: None
482
+ metrics:
483
+ - type: accuracy
484
+ value: 48.303193535975375
485
+ - type: f1
486
+ value: 35.96559358693866
487
+ - task:
488
+ type: Classification
489
+ dataset:
490
+ name: MTEB JDReview
491
+ type: C-MTEB/JDReview-classification
492
+ config: default
493
+ split: test
494
+ revision: None
495
+ metrics:
496
+ - type: accuracy
497
+ value: 85.06566604127579
498
+ - type: ap
499
+ value: 52.0596483757231
500
+ - type: f1
501
+ value: 79.5196835127668
502
+ - task:
503
+ type: STS
504
+ dataset:
505
+ name: MTEB LCQMC
506
+ type: C-MTEB/LCQMC
507
+ config: default
508
+ split: test
509
+ revision: None
510
+ metrics:
511
+ - type: cos_sim_pearson
512
+ value: 74.48499423626059
513
+ - type: cos_sim_spearman
514
+ value: 78.75806756061169
515
+ - type: euclidean_pearson
516
+ value: 78.47917601852879
517
+ - type: euclidean_spearman
518
+ value: 78.75807199272622
519
+ - type: manhattan_pearson
520
+ value: 78.40207586289772
521
+ - type: manhattan_spearman
522
+ value: 78.6911776964119
523
+ - task:
524
+ type: Reranking
525
+ dataset:
526
+ name: MTEB MMarcoReranking
527
+ type: C-MTEB/Mmarco-reranking
528
+ config: default
529
+ split: dev
530
+ revision: None
531
+ metrics:
532
+ - type: map
533
+ value: 24.75987466552363
534
+ - type: mrr
535
+ value: 23.40515873015873
536
+ - task:
537
+ type: Retrieval
538
+ dataset:
539
+ name: MTEB MMarcoRetrieval
540
+ type: C-MTEB/MMarcoRetrieval
541
+ config: default
542
+ split: dev
543
+ revision: None
544
+ metrics:
545
+ - type: map_at_1
546
+ value: 58.026999999999994
547
+ - type: map_at_10
548
+ value: 67.50699999999999
549
+ - type: map_at_100
550
+ value: 67.946
551
+ - type: map_at_1000
552
+ value: 67.96600000000001
553
+ - type: map_at_3
554
+ value: 65.503
555
+ - type: map_at_5
556
+ value: 66.649
557
+ - type: mrr_at_1
558
+ value: 60.20100000000001
559
+ - type: mrr_at_10
560
+ value: 68.271
561
+ - type: mrr_at_100
562
+ value: 68.664
563
+ - type: mrr_at_1000
564
+ value: 68.682
565
+ - type: mrr_at_3
566
+ value: 66.47800000000001
567
+ - type: mrr_at_5
568
+ value: 67.499
569
+ - type: ndcg_at_1
570
+ value: 60.20100000000001
571
+ - type: ndcg_at_10
572
+ value: 71.697
573
+ - type: ndcg_at_100
574
+ value: 73.736
575
+ - type: ndcg_at_1000
576
+ value: 74.259
577
+ - type: ndcg_at_3
578
+ value: 67.768
579
+ - type: ndcg_at_5
580
+ value: 69.72
581
+ - type: precision_at_1
582
+ value: 60.20100000000001
583
+ - type: precision_at_10
584
+ value: 8.927999999999999
585
+ - type: precision_at_100
586
+ value: 0.9950000000000001
587
+ - type: precision_at_1000
588
+ value: 0.104
589
+ - type: precision_at_3
590
+ value: 25.883
591
+ - type: precision_at_5
592
+ value: 16.55
593
+ - type: recall_at_1
594
+ value: 58.026999999999994
595
+ - type: recall_at_10
596
+ value: 83.966
597
+ - type: recall_at_100
598
+ value: 93.313
599
+ - type: recall_at_1000
600
+ value: 97.426
601
+ - type: recall_at_3
602
+ value: 73.342
603
+ - type: recall_at_5
604
+ value: 77.997
605
+ - task:
606
+ type: Classification
607
+ dataset:
608
+ name: MTEB MassiveIntentClassification (zh-CN)
609
+ type: mteb/amazon_massive_intent
610
+ config: zh-CN
611
+ split: test
612
+ revision: 31efe3c427b0bae9c22cbb560b8f15491cc6bed7
613
+ metrics:
614
+ - type: accuracy
615
+ value: 71.1600537995965
616
+ - type: f1
617
+ value: 68.8126216609964
618
+ - task:
619
+ type: Classification
620
+ dataset:
621
+ name: MTEB MassiveScenarioClassification (zh-CN)
622
+ type: mteb/amazon_massive_scenario
623
+ config: zh-CN
624
+ split: test
625
+ revision: 7d571f92784cd94a019292a1f45445077d0ef634
626
+ metrics:
627
+ - type: accuracy
628
+ value: 73.54068594485541
629
+ - type: f1
630
+ value: 73.46845879869848
631
+ - task:
632
+ type: Retrieval
633
+ dataset:
634
+ name: MTEB MedicalRetrieval
635
+ type: C-MTEB/MedicalRetrieval
636
+ config: default
637
+ split: dev
638
+ revision: None
639
+ metrics:
640
+ - type: map_at_1
641
+ value: 54.900000000000006
642
+ - type: map_at_10
643
+ value: 61.363
644
+ - type: map_at_100
645
+ value: 61.924
646
+ - type: map_at_1000
647
+ value: 61.967000000000006
648
+ - type: map_at_3
649
+ value: 59.767
650
+ - type: map_at_5
651
+ value: 60.802
652
+ - type: mrr_at_1
653
+ value: 55.1
654
+ - type: mrr_at_10
655
+ value: 61.454
656
+ - type: mrr_at_100
657
+ value: 62.016000000000005
658
+ - type: mrr_at_1000
659
+ value: 62.059
660
+ - type: mrr_at_3
661
+ value: 59.882999999999996
662
+ - type: mrr_at_5
663
+ value: 60.893
664
+ - type: ndcg_at_1
665
+ value: 54.900000000000006
666
+ - type: ndcg_at_10
667
+ value: 64.423
668
+ - type: ndcg_at_100
669
+ value: 67.35900000000001
670
+ - type: ndcg_at_1000
671
+ value: 68.512
672
+ - type: ndcg_at_3
673
+ value: 61.224000000000004
674
+ - type: ndcg_at_5
675
+ value: 63.083
676
+ - type: precision_at_1
677
+ value: 54.900000000000006
678
+ - type: precision_at_10
679
+ value: 7.3999999999999995
680
+ - type: precision_at_100
681
+ value: 0.882
682
+ - type: precision_at_1000
683
+ value: 0.097
684
+ - type: precision_at_3
685
+ value: 21.8
686
+ - type: precision_at_5
687
+ value: 13.98
688
+ - type: recall_at_1
689
+ value: 54.900000000000006
690
+ - type: recall_at_10
691
+ value: 74
692
+ - type: recall_at_100
693
+ value: 88.2
694
+ - type: recall_at_1000
695
+ value: 97.3
696
+ - type: recall_at_3
697
+ value: 65.4
698
+ - type: recall_at_5
699
+ value: 69.89999999999999
700
+ - task:
701
+ type: Classification
702
+ dataset:
703
+ name: MTEB MultilingualSentiment
704
+ type: C-MTEB/MultilingualSentiment-classification
705
+ config: default
706
+ split: validation
707
+ revision: None
708
+ metrics:
709
+ - type: accuracy
710
+ value: 75.15666666666667
711
+ - type: f1
712
+ value: 74.8306375354435
713
+ - task:
714
+ type: PairClassification
715
+ dataset:
716
+ name: MTEB Ocnli
717
+ type: C-MTEB/OCNLI
718
+ config: default
719
+ split: validation
720
+ revision: None
721
+ metrics:
722
+ - type: cos_sim_accuracy
723
+ value: 83.10774228478614
724
+ - type: cos_sim_ap
725
+ value: 87.17679348388666
726
+ - type: cos_sim_f1
727
+ value: 84.59302325581395
728
+ - type: cos_sim_precision
729
+ value: 78.15577439570276
730
+ - type: cos_sim_recall
731
+ value: 92.18585005279832
732
+ - type: dot_accuracy
733
+ value: 83.10774228478614
734
+ - type: dot_ap
735
+ value: 87.17679348388666
736
+ - type: dot_f1
737
+ value: 84.59302325581395
738
+ - type: dot_precision
739
+ value: 78.15577439570276
740
+ - type: dot_recall
741
+ value: 92.18585005279832
742
+ - type: euclidean_accuracy
743
+ value: 83.10774228478614
744
+ - type: euclidean_ap
745
+ value: 87.17679348388666
746
+ - type: euclidean_f1
747
+ value: 84.59302325581395
748
+ - type: euclidean_precision
749
+ value: 78.15577439570276
750
+ - type: euclidean_recall
751
+ value: 92.18585005279832
752
+ - type: manhattan_accuracy
753
+ value: 82.67460747157553
754
+ - type: manhattan_ap
755
+ value: 86.94296334435238
756
+ - type: manhattan_f1
757
+ value: 84.32327166504382
758
+ - type: manhattan_precision
759
+ value: 78.22944896115628
760
+ - type: manhattan_recall
761
+ value: 91.4466737064414
762
+ - type: max_accuracy
763
+ value: 83.10774228478614
764
+ - type: max_ap
765
+ value: 87.17679348388666
766
+ - type: max_f1
767
+ value: 84.59302325581395
768
+ - task:
769
+ type: Classification
770
+ dataset:
771
+ name: MTEB OnlineShopping
772
+ type: C-MTEB/OnlineShopping-classification
773
+ config: default
774
+ split: test
775
+ revision: None
776
+ metrics:
777
+ - type: accuracy
778
+ value: 93.24999999999999
779
+ - type: ap
780
+ value: 90.98617641063584
781
+ - type: f1
782
+ value: 93.23447883650289
783
+ - task:
784
+ type: STS
785
+ dataset:
786
+ name: MTEB PAWSX
787
+ type: C-MTEB/PAWSX
788
+ config: default
789
+ split: test
790
+ revision: None
791
+ metrics:
792
+ - type: cos_sim_pearson
793
+ value: 41.071417937737856
794
+ - type: cos_sim_spearman
795
+ value: 45.049199344455424
796
+ - type: euclidean_pearson
797
+ value: 44.913450096830786
798
+ - type: euclidean_spearman
799
+ value: 45.05733424275291
800
+ - type: manhattan_pearson
801
+ value: 44.881623825912065
802
+ - type: manhattan_spearman
803
+ value: 44.989923561416596
804
+ - task:
805
+ type: STS
806
+ dataset:
807
+ name: MTEB QBQTC
808
+ type: C-MTEB/QBQTC
809
+ config: default
810
+ split: test
811
+ revision: None
812
+ metrics:
813
+ - type: cos_sim_pearson
814
+ value: 41.38238052689359
815
+ - type: cos_sim_spearman
816
+ value: 42.61949690594399
817
+ - type: euclidean_pearson
818
+ value: 40.61261500356766
819
+ - type: euclidean_spearman
820
+ value: 42.619626605620724
821
+ - type: manhattan_pearson
822
+ value: 40.8886109204474
823
+ - type: manhattan_spearman
824
+ value: 42.75791523010463
825
+ - task:
826
+ type: STS
827
+ dataset:
828
+ name: MTEB STS22 (zh)
829
+ type: mteb/sts22-crosslingual-sts
830
+ config: zh
831
+ split: test
832
+ revision: 6d1ba47164174a496b7fa5d3569dae26a6813b80
833
+ metrics:
834
+ - type: cos_sim_pearson
835
+ value: 62.10977863727196
836
+ - type: cos_sim_spearman
837
+ value: 63.843727112473225
838
+ - type: euclidean_pearson
839
+ value: 63.25133487817196
840
+ - type: euclidean_spearman
841
+ value: 63.843727112473225
842
+ - type: manhattan_pearson
843
+ value: 63.58749018644103
844
+ - type: manhattan_spearman
845
+ value: 63.83820575456674
846
+ - task:
847
+ type: STS
848
+ dataset:
849
+ name: MTEB STSB
850
+ type: C-MTEB/STSB
851
+ config: default
852
+ split: test
853
+ revision: None
854
+ metrics:
855
+ - type: cos_sim_pearson
856
+ value: 79.30616496720054
857
+ - type: cos_sim_spearman
858
+ value: 80.767935782436
859
+ - type: euclidean_pearson
860
+ value: 80.4160642670106
861
+ - type: euclidean_spearman
862
+ value: 80.76820284024356
863
+ - type: manhattan_pearson
864
+ value: 80.27318714580251
865
+ - type: manhattan_spearman
866
+ value: 80.61030164164964
867
+ - task:
868
+ type: Reranking
869
+ dataset:
870
+ name: MTEB T2Reranking
871
+ type: C-MTEB/T2Reranking
872
+ config: default
873
+ split: dev
874
+ revision: None
875
+ metrics:
876
+ - type: map
877
+ value: 66.26242871142425
878
+ - type: mrr
879
+ value: 76.20689863623174
880
+ - task:
881
+ type: Retrieval
882
+ dataset:
883
+ name: MTEB T2Retrieval
884
+ type: C-MTEB/T2Retrieval
885
+ config: default
886
+ split: dev
887
+ revision: None
888
+ metrics:
889
+ - type: map_at_1
890
+ value: 26.240999999999996
891
+ - type: map_at_10
892
+ value: 73.009
893
+ - type: map_at_100
894
+ value: 76.893
895
+ - type: map_at_1000
896
+ value: 76.973
897
+ - type: map_at_3
898
+ value: 51.339
899
+ - type: map_at_5
900
+ value: 63.003
901
+ - type: mrr_at_1
902
+ value: 87.458
903
+ - type: mrr_at_10
904
+ value: 90.44
905
+ - type: mrr_at_100
906
+ value: 90.558
907
+ - type: mrr_at_1000
908
+ value: 90.562
909
+ - type: mrr_at_3
910
+ value: 89.89
911
+ - type: mrr_at_5
912
+ value: 90.231
913
+ - type: ndcg_at_1
914
+ value: 87.458
915
+ - type: ndcg_at_10
916
+ value: 81.325
917
+ - type: ndcg_at_100
918
+ value: 85.61999999999999
919
+ - type: ndcg_at_1000
920
+ value: 86.394
921
+ - type: ndcg_at_3
922
+ value: 82.796
923
+ - type: ndcg_at_5
924
+ value: 81.219
925
+ - type: precision_at_1
926
+ value: 87.458
927
+ - type: precision_at_10
928
+ value: 40.534
929
+ - type: precision_at_100
930
+ value: 4.96
931
+ - type: precision_at_1000
932
+ value: 0.514
933
+ - type: precision_at_3
934
+ value: 72.444
935
+ - type: precision_at_5
936
+ value: 60.601000000000006
937
+ - type: recall_at_1
938
+ value: 26.240999999999996
939
+ - type: recall_at_10
940
+ value: 80.42
941
+ - type: recall_at_100
942
+ value: 94.118
943
+ - type: recall_at_1000
944
+ value: 98.02199999999999
945
+ - type: recall_at_3
946
+ value: 53.174
947
+ - type: recall_at_5
948
+ value: 66.739
949
+ - task:
950
+ type: Classification
951
+ dataset:
952
+ name: MTEB TNews
953
+ type: C-MTEB/TNews-classification
954
+ config: default
955
+ split: validation
956
+ revision: None
957
+ metrics:
958
+ - type: accuracy
959
+ value: 52.40899999999999
960
+ - type: f1
961
+ value: 50.68532128056062
962
+ - task:
963
+ type: Clustering
964
+ dataset:
965
+ name: MTEB ThuNewsClusteringP2P
966
+ type: C-MTEB/ThuNewsClusteringP2P
967
+ config: default
968
+ split: test
969
+ revision: None
970
+ metrics:
971
+ - type: v_measure
972
+ value: 65.57616085176686
973
+ - task:
974
+ type: Clustering
975
+ dataset:
976
+ name: MTEB ThuNewsClusteringS2S
977
+ type: C-MTEB/ThuNewsClusteringS2S
978
+ config: default
979
+ split: test
980
+ revision: None
981
+ metrics:
982
+ - type: v_measure
983
+ value: 58.844999922904925
984
+ - task:
985
+ type: Retrieval
986
+ dataset:
987
+ name: MTEB VideoRetrieval
988
+ type: C-MTEB/VideoRetrieval
989
+ config: default
990
+ split: dev
991
+ revision: None
992
+ metrics:
993
+ - type: map_at_1
994
+ value: 58.4
995
+ - type: map_at_10
996
+ value: 68.64
997
+ - type: map_at_100
998
+ value: 69.062
999
+ - type: map_at_1000
1000
+ value: 69.073
1001
+ - type: map_at_3
1002
+ value: 66.567
1003
+ - type: map_at_5
1004
+ value: 67.89699999999999
1005
+ - type: mrr_at_1
1006
+ value: 58.4
1007
+ - type: mrr_at_10
1008
+ value: 68.64
1009
+ - type: mrr_at_100
1010
+ value: 69.062
1011
+ - type: mrr_at_1000
1012
+ value: 69.073
1013
+ - type: mrr_at_3
1014
+ value: 66.567
1015
+ - type: mrr_at_5
1016
+ value: 67.89699999999999
1017
+ - type: ndcg_at_1
1018
+ value: 58.4
1019
+ - type: ndcg_at_10
1020
+ value: 73.30600000000001
1021
+ - type: ndcg_at_100
1022
+ value: 75.276
1023
+ - type: ndcg_at_1000
1024
+ value: 75.553
1025
+ - type: ndcg_at_3
1026
+ value: 69.126
1027
+ - type: ndcg_at_5
1028
+ value: 71.519
1029
+ - type: precision_at_1
1030
+ value: 58.4
1031
+ - type: precision_at_10
1032
+ value: 8.780000000000001
1033
+ - type: precision_at_100
1034
+ value: 0.968
1035
+ - type: precision_at_1000
1036
+ value: 0.099
1037
+ - type: precision_at_3
1038
+ value: 25.5
1039
+ - type: precision_at_5
1040
+ value: 16.46
1041
+ - type: recall_at_1
1042
+ value: 58.4
1043
+ - type: recall_at_10
1044
+ value: 87.8
1045
+ - type: recall_at_100
1046
+ value: 96.8
1047
+ - type: recall_at_1000
1048
+ value: 99
1049
+ - type: recall_at_3
1050
+ value: 76.5
1051
+ - type: recall_at_5
1052
+ value: 82.3
1053
+ - task:
1054
+ type: Classification
1055
+ dataset:
1056
+ name: MTEB Waimai
1057
+ type: C-MTEB/waimai-classification
1058
+ config: default
1059
+ split: test
1060
+ revision: None
1061
+ metrics:
1062
+ - type: accuracy
1063
+ value: 86.21000000000001
1064
+ - type: ap
1065
+ value: 69.17460264576461
1066
+ - type: f1
1067
+ value: 84.68032984659226
1068
+ ---
1069
+
1070
+ # annofung/Dmeta-embedding-zh-Q5_K_M-GGUF
1071
+ This model was converted to GGUF format from [`DMetaSoul/Dmeta-embedding-zh`](https://huggingface.co/DMetaSoul/Dmeta-embedding-zh) using llama.cpp via the ggml.ai's [GGUF-my-repo](https://huggingface.co/spaces/ggml-org/gguf-my-repo) space.
1072
+ Refer to the [original model card](https://huggingface.co/DMetaSoul/Dmeta-embedding-zh) for more details on the model.
1073
+
1074
+ ## Use with llama.cpp
1075
+ Install llama.cpp through brew (works on Mac and Linux)
1076
+
1077
+ ```bash
1078
+ brew install llama.cpp
1079
+
1080
+ ```
1081
+ Invoke the llama.cpp server or the CLI.
1082
+
1083
+ ### CLI:
1084
+ ```bash
1085
+ llama-cli --hf-repo annofung/Dmeta-embedding-zh-Q5_K_M-GGUF --hf-file dmeta-embedding-zh-q5_k_m.gguf -p "The meaning to life and the universe is"
1086
+ ```
1087
+
1088
+ ### Server:
1089
+ ```bash
1090
+ llama-server --hf-repo annofung/Dmeta-embedding-zh-Q5_K_M-GGUF --hf-file dmeta-embedding-zh-q5_k_m.gguf -c 2048
1091
+ ```
1092
+
1093
+ Note: You can also use this checkpoint directly through the [usage steps](https://github.com/ggerganov/llama.cpp?tab=readme-ov-file#usage) listed in the Llama.cpp repo as well.
1094
+
1095
+ Step 1: Clone llama.cpp from GitHub.
1096
+ ```
1097
+ git clone https://github.com/ggerganov/llama.cpp
1098
+ ```
1099
+
1100
+ Step 2: Move into the llama.cpp folder and build it with `LLAMA_CURL=1` flag along with other hardware-specific flags (for ex: LLAMA_CUDA=1 for Nvidia GPUs on Linux).
1101
+ ```
1102
+ cd llama.cpp && LLAMA_CURL=1 make
1103
+ ```
1104
+
1105
+ Step 3: Run inference through the main binary.
1106
+ ```
1107
+ ./llama-cli --hf-repo annofung/Dmeta-embedding-zh-Q5_K_M-GGUF --hf-file dmeta-embedding-zh-q5_k_m.gguf -p "The meaning to life and the universe is"
1108
+ ```
1109
+ or
1110
+ ```
1111
+ ./llama-server --hf-repo annofung/Dmeta-embedding-zh-Q5_K_M-GGUF --hf-file dmeta-embedding-zh-q5_k_m.gguf -c 2048
1112
+ ```