|
--- |
|
pipeline_tag: text-classification |
|
tags: |
|
- sentence-transformers |
|
- feature-extraction |
|
- sentence-similarity |
|
- mteb |
|
model-index: |
|
- name: KE Sieve_model |
|
results: |
|
- task: |
|
type: Classification |
|
dataset: |
|
type: mteb/amazon_counterfactual |
|
name: MTEB AmazonCounterfactualClassification (en) |
|
config: en |
|
split: test |
|
revision: e8379541af4e31359cca9fbcf4b00f2671dba205 |
|
metrics: |
|
- type: accuracy |
|
value: 79.05970149253731 |
|
- type: ap |
|
value: 42.7075359884682 |
|
- type: f1 |
|
value: 72.99649470402085 |
|
- task: |
|
type: Classification |
|
dataset: |
|
type: mteb/amazon_polarity |
|
name: MTEB AmazonPolarityClassification |
|
config: default |
|
split: test |
|
revision: e2d317d38cd51312af73b3d32a06d1a08b442046 |
|
metrics: |
|
- type: accuracy |
|
value: 70.193 |
|
- type: ap |
|
value: 64.37171698026376 |
|
- type: f1 |
|
value: 69.99260638185035 |
|
- task: |
|
type: Classification |
|
dataset: |
|
type: mteb/amazon_reviews_multi |
|
name: MTEB AmazonReviewsClassification (en) |
|
config: en |
|
split: test |
|
revision: 1399c76144fd37290681b995c656ef9b2e06e26d |
|
metrics: |
|
- type: accuracy |
|
value: 34.288000000000004 |
|
- type: f1 |
|
value: 34.00390576721439 |
|
- task: |
|
type: STS |
|
dataset: |
|
type: mteb/biosses-sts |
|
name: MTEB BIOSSES |
|
config: default |
|
split: test |
|
revision: d3fb88f8f02e40887cd149695127462bbcf29b4a |
|
metrics: |
|
- type: cos_sim_pearson |
|
value: 70.37283775714532 |
|
- type: cos_sim_spearman |
|
value: 65.28702977793742 |
|
- type: euclidean_pearson |
|
value: 68.81678452970543 |
|
- type: euclidean_spearman |
|
value: 66.10212250382912 |
|
- type: manhattan_pearson |
|
value: 70.06439132928513 |
|
- type: manhattan_spearman |
|
value: 66.10212250382912 |
|
- task: |
|
type: Classification |
|
dataset: |
|
type: mteb/banking77 |
|
name: MTEB Banking77Classification |
|
config: default |
|
split: test |
|
revision: 0fd18e25b25c072e09e0d92ab615fda904d66300 |
|
metrics: |
|
- type: accuracy |
|
value: 75.88961038961038 |
|
- type: f1 |
|
value: 75.71295362599926 |
|
- task: |
|
type: Classification |
|
dataset: |
|
type: mteb/emotion |
|
name: MTEB EmotionClassification |
|
config: default |
|
split: test |
|
revision: 4f58c6b202a23cf9a4da393831edf4f9183cad37 |
|
metrics: |
|
- type: accuracy |
|
value: 40.26 |
|
- type: f1 |
|
value: 35.91571484611428 |
|
- task: |
|
type: Classification |
|
dataset: |
|
type: mteb/imdb |
|
name: MTEB ImdbClassification |
|
config: default |
|
split: test |
|
revision: 3d86128a09e091d6018b6d26cad27f2739fc2db7 |
|
metrics: |
|
- type: accuracy |
|
value: 61.1396 |
|
- type: ap |
|
value: 57.0336104684589 |
|
- type: f1 |
|
value: 60.711055351249385 |
|
- task: |
|
type: Classification |
|
dataset: |
|
type: mteb/mtop_domain |
|
name: MTEB MTOPDomainClassification (en) |
|
config: en |
|
split: test |
|
revision: d80d48c1eb48d3562165c59d59d0034df9fff0bf |
|
metrics: |
|
- type: accuracy |
|
value: 87.21842225262198 |
|
- type: f1 |
|
value: 86.60570158294514 |
|
- task: |
|
type: Classification |
|
dataset: |
|
type: mteb/mtop_intent |
|
name: MTEB MTOPIntentClassification (en) |
|
config: en |
|
split: test |
|
revision: ae001d0e6b1228650b7bd1c2c65fb50ad11a8aba |
|
metrics: |
|
- type: accuracy |
|
value: 69.44824441404468 |
|
- type: f1 |
|
value: 51.1702284173121 |
|
- task: |
|
type: Classification |
|
dataset: |
|
type: mteb/amazon_massive_intent |
|
name: MTEB MassiveIntentClassification (en) |
|
config: en |
|
split: test |
|
revision: 31efe3c427b0bae9c22cbb560b8f15491cc6bed7 |
|
metrics: |
|
- type: accuracy |
|
value: 65.60188298587761 |
|
- type: f1 |
|
value: 64.57658770410065 |
|
- task: |
|
type: Classification |
|
dataset: |
|
type: mteb/amazon_massive_scenario |
|
name: MTEB MassiveScenarioClassification (en) |
|
config: en |
|
split: test |
|
revision: 7d571f92784cd94a019292a1f45445077d0ef634 |
|
metrics: |
|
- type: accuracy |
|
value: 70.36987222595829 |
|
- type: f1 |
|
value: 70.34853403058946 |
|
- task: |
|
type: STS |
|
dataset: |
|
type: mteb/sickr-sts |
|
name: MTEB SICK-R |
|
config: default |
|
split: test |
|
revision: a6ea5a8cab320b040a23452cc28066d9beae2cee |
|
metrics: |
|
- type: cos_sim_pearson |
|
value: 78.1402991982508 |
|
- type: cos_sim_spearman |
|
value: 76.01438891892613 |
|
- type: euclidean_pearson |
|
value: 76.07791972310307 |
|
- type: euclidean_spearman |
|
value: 76.4750927224088 |
|
- type: manhattan_pearson |
|
value: 78.7022742184064 |
|
- type: manhattan_spearman |
|
value: 76.4750927224088 |
|
- task: |
|
type: STS |
|
dataset: |
|
type: mteb/sts12-sts |
|
name: MTEB STS12 |
|
config: default |
|
split: test |
|
revision: a0d554a64d88156834ff5ae9920b964011b16384 |
|
metrics: |
|
- type: cos_sim_pearson |
|
value: 77.41946856528065 |
|
- type: cos_sim_spearman |
|
value: 71.2452368975646 |
|
- type: euclidean_pearson |
|
value: 68.76119955717198 |
|
- type: euclidean_spearman |
|
value: 70.40762520824568 |
|
- type: manhattan_pearson |
|
value: 76.1638570991111 |
|
- type: manhattan_spearman |
|
value: 70.40762520824568 |
|
- task: |
|
type: STS |
|
dataset: |
|
type: mteb/sts13-sts |
|
name: MTEB STS13 |
|
config: default |
|
split: test |
|
revision: 7e90230a92c190f1bf69ae9002b8cea547a64cca |
|
metrics: |
|
- type: cos_sim_pearson |
|
value: 77.86983630535461 |
|
- type: cos_sim_spearman |
|
value: 78.39885607110992 |
|
- type: euclidean_pearson |
|
value: 75.81565277674996 |
|
- type: euclidean_spearman |
|
value: 78.70053430302474 |
|
- type: manhattan_pearson |
|
value: 78.14484348028292 |
|
- type: manhattan_spearman |
|
value: 78.70053430302474 |
|
- task: |
|
type: STS |
|
dataset: |
|
type: mteb/sts14-sts |
|
name: MTEB STS14 |
|
config: default |
|
split: test |
|
revision: 6031580fec1f6af667f0bd2da0a551cf4f0b2375 |
|
metrics: |
|
- type: cos_sim_pearson |
|
value: 76.52542250553228 |
|
- type: cos_sim_spearman |
|
value: 74.23425444398934 |
|
- type: euclidean_pearson |
|
value: 73.63790688920109 |
|
- type: euclidean_spearman |
|
value: 74.14127580980806 |
|
- type: manhattan_pearson |
|
value: 76.76724842158396 |
|
- type: manhattan_spearman |
|
value: 74.14127580980806 |
|
- task: |
|
type: STS |
|
dataset: |
|
type: mteb/sts15-sts |
|
name: MTEB STS15 |
|
config: default |
|
split: test |
|
revision: ae752c7c21bf194d8b67fd573edf7ae58183cbe3 |
|
metrics: |
|
- type: cos_sim_pearson |
|
value: 80.9319282262523 |
|
- type: cos_sim_spearman |
|
value: 81.40861373830771 |
|
- type: euclidean_pearson |
|
value: 79.61339072348075 |
|
- type: euclidean_spearman |
|
value: 82.1601716091385 |
|
- type: manhattan_pearson |
|
value: 81.76770515821788 |
|
- type: manhattan_spearman |
|
value: 82.1601716091385 |
|
- task: |
|
type: STS |
|
dataset: |
|
type: mteb/sts16-sts |
|
name: MTEB STS16 |
|
config: default |
|
split: test |
|
revision: 4d8694f8f0e0100860b497b999b3dbed754a0513 |
|
metrics: |
|
- type: cos_sim_pearson |
|
value: 78.83953330477087 |
|
- type: cos_sim_spearman |
|
value: 79.1312883671738 |
|
- type: euclidean_pearson |
|
value: 77.02068269010785 |
|
- type: euclidean_spearman |
|
value: 78.85332564873545 |
|
- type: manhattan_pearson |
|
value: 78.66151014252961 |
|
- type: manhattan_spearman |
|
value: 78.85332564873545 |
|
- task: |
|
type: STS |
|
dataset: |
|
type: mteb/sts17-crosslingual-sts |
|
name: MTEB STS17 (ko-ko) |
|
config: ko-ko |
|
split: test |
|
revision: af5e6fb845001ecf41f4c1e033ce921939a2a68d |
|
metrics: |
|
- type: cos_sim_pearson |
|
value: 77.06164373590121 |
|
- type: cos_sim_spearman |
|
value: 76.99890844656588 |
|
- type: euclidean_pearson |
|
value: 73.39118839457844 |
|
- type: euclidean_spearman |
|
value: 77.11144988540109 |
|
- type: manhattan_pearson |
|
value: 77.20681515013695 |
|
- type: manhattan_spearman |
|
value: 77.11144988540109 |
|
- task: |
|
type: STS |
|
dataset: |
|
type: mteb/sts17-crosslingual-sts |
|
name: MTEB STS17 (ar-ar) |
|
config: ar-ar |
|
split: test |
|
revision: af5e6fb845001ecf41f4c1e033ce921939a2a68d |
|
metrics: |
|
- type: cos_sim_pearson |
|
value: 77.60555084043324 |
|
- type: cos_sim_spearman |
|
value: 76.04445852887906 |
|
- type: euclidean_pearson |
|
value: 72.71133101639413 |
|
- type: euclidean_spearman |
|
value: 75.91338695530828 |
|
- type: manhattan_pearson |
|
value: 77.35612564470868 |
|
- type: manhattan_spearman |
|
value: 75.91338695530828 |
|
- task: |
|
type: STS |
|
dataset: |
|
type: mteb/sts17-crosslingual-sts |
|
name: MTEB STS17 (en-ar) |
|
config: en-ar |
|
split: test |
|
revision: af5e6fb845001ecf41f4c1e033ce921939a2a68d |
|
metrics: |
|
- type: cos_sim_pearson |
|
value: 78.41618617815928 |
|
- type: cos_sim_spearman |
|
value: 77.60195378076503 |
|
- type: euclidean_pearson |
|
value: 78.16168735305624 |
|
- type: euclidean_spearman |
|
value: 77.67819163961478 |
|
- type: manhattan_pearson |
|
value: 78.40140865643386 |
|
- type: manhattan_spearman |
|
value: 77.67819163961478 |
|
- task: |
|
type: STS |
|
dataset: |
|
type: mteb/sts17-crosslingual-sts |
|
name: MTEB STS17 (en-de) |
|
config: en-de |
|
split: test |
|
revision: af5e6fb845001ecf41f4c1e033ce921939a2a68d |
|
metrics: |
|
- type: cos_sim_pearson |
|
value: 71.44561691901534 |
|
- type: cos_sim_spearman |
|
value: 70.39834592402187 |
|
- type: euclidean_pearson |
|
value: 71.5559771884868 |
|
- type: euclidean_spearman |
|
value: 70.11301222833383 |
|
- type: manhattan_pearson |
|
value: 71.51922693185502 |
|
- type: manhattan_spearman |
|
value: 70.11301222833383 |
|
- task: |
|
type: STS |
|
dataset: |
|
type: mteb/sts17-crosslingual-sts |
|
name: MTEB STS17 (en-en) |
|
config: en-en |
|
split: test |
|
revision: af5e6fb845001ecf41f4c1e033ce921939a2a68d |
|
metrics: |
|
- type: cos_sim_pearson |
|
value: 86.7214978664316 |
|
- type: cos_sim_spearman |
|
value: 85.4010906321244 |
|
- type: euclidean_pearson |
|
value: 84.6346870837772 |
|
- type: euclidean_spearman |
|
value: 85.72569452807713 |
|
- type: manhattan_pearson |
|
value: 86.96159961830801 |
|
- type: manhattan_spearman |
|
value: 85.72569452807713 |
|
- task: |
|
type: STS |
|
dataset: |
|
type: mteb/sts17-crosslingual-sts |
|
name: MTEB STS17 (en-tr) |
|
config: en-tr |
|
split: test |
|
revision: af5e6fb845001ecf41f4c1e033ce921939a2a68d |
|
metrics: |
|
- type: cos_sim_pearson |
|
value: 72.09730265741813 |
|
- type: cos_sim_spearman |
|
value: 71.0352138913937 |
|
- type: euclidean_pearson |
|
value: 72.55713973075069 |
|
- type: euclidean_spearman |
|
value: 71.41534122613018 |
|
- type: manhattan_pearson |
|
value: 72.55966082460004 |
|
- type: manhattan_spearman |
|
value: 71.41534122613018 |
|
- task: |
|
type: STS |
|
dataset: |
|
type: mteb/sts17-crosslingual-sts |
|
name: MTEB STS17 (es-en) |
|
config: es-en |
|
split: test |
|
revision: af5e6fb845001ecf41f4c1e033ce921939a2a68d |
|
metrics: |
|
- type: cos_sim_pearson |
|
value: 82.03153344804768 |
|
- type: cos_sim_spearman |
|
value: 81.58711344537957 |
|
- type: euclidean_pearson |
|
value: 81.23021018553894 |
|
- type: euclidean_spearman |
|
value: 81.92757732356259 |
|
- type: manhattan_pearson |
|
value: 82.15831176471193 |
|
- type: manhattan_spearman |
|
value: 81.92757732356259 |
|
- task: |
|
type: STS |
|
dataset: |
|
type: mteb/sts17-crosslingual-sts |
|
name: MTEB STS17 (es-es) |
|
config: es-es |
|
split: test |
|
revision: af5e6fb845001ecf41f4c1e033ce921939a2a68d |
|
metrics: |
|
- type: cos_sim_pearson |
|
value: 83.82880794136425 |
|
- type: cos_sim_spearman |
|
value: 82.77088436337785 |
|
- type: euclidean_pearson |
|
value: 81.25832734044387 |
|
- type: euclidean_spearman |
|
value: 83.62944563056716 |
|
- type: manhattan_pearson |
|
value: 84.53726605538859 |
|
- type: manhattan_spearman |
|
value: 83.62944563056716 |
|
- task: |
|
type: STS |
|
dataset: |
|
type: mteb/sts17-crosslingual-sts |
|
name: MTEB STS17 (fr-en) |
|
config: fr-en |
|
split: test |
|
revision: af5e6fb845001ecf41f4c1e033ce921939a2a68d |
|
metrics: |
|
- type: cos_sim_pearson |
|
value: 78.4156098242599 |
|
- type: cos_sim_spearman |
|
value: 77.15842055051796 |
|
- type: euclidean_pearson |
|
value: 78.9792127917851 |
|
- type: euclidean_spearman |
|
value: 78.09974898801255 |
|
- type: manhattan_pearson |
|
value: 79.0925556678293 |
|
- type: manhattan_spearman |
|
value: 78.09974898801255 |
|
- task: |
|
type: STS |
|
dataset: |
|
type: mteb/sts17-crosslingual-sts |
|
name: MTEB STS17 (it-en) |
|
config: it-en |
|
split: test |
|
revision: af5e6fb845001ecf41f4c1e033ce921939a2a68d |
|
metrics: |
|
- type: cos_sim_pearson |
|
value: 82.90712716373704 |
|
- type: cos_sim_spearman |
|
value: 81.519207224176 |
|
- type: euclidean_pearson |
|
value: 82.74512409664257 |
|
- type: euclidean_spearman |
|
value: 81.99923052819682 |
|
- type: manhattan_pearson |
|
value: 83.32430067509108 |
|
- type: manhattan_spearman |
|
value: 81.99923052819682 |
|
- task: |
|
type: STS |
|
dataset: |
|
type: mteb/sts17-crosslingual-sts |
|
name: MTEB STS17 (nl-en) |
|
config: nl-en |
|
split: test |
|
revision: af5e6fb845001ecf41f4c1e033ce921939a2a68d |
|
metrics: |
|
- type: cos_sim_pearson |
|
value: 81.93681389517745 |
|
- type: cos_sim_spearman |
|
value: 80.70090384624984 |
|
- type: euclidean_pearson |
|
value: 82.04806027549073 |
|
- type: euclidean_spearman |
|
value: 81.45677948183294 |
|
- type: manhattan_pearson |
|
value: 82.62825908719917 |
|
- type: manhattan_spearman |
|
value: 81.45677948183294 |
|
- task: |
|
type: STS |
|
dataset: |
|
type: mteb/sts22-crosslingual-sts |
|
name: MTEB STS22 (en) |
|
config: en |
|
split: test |
|
revision: 6d1ba47164174a496b7fa5d3569dae26a6813b80 |
|
metrics: |
|
- type: cos_sim_pearson |
|
value: 57.8307489054962 |
|
- type: cos_sim_spearman |
|
value: 58.62505961044144 |
|
- type: euclidean_pearson |
|
value: 55.77564028818849 |
|
- type: euclidean_spearman |
|
value: 58.03263946623424 |
|
- type: manhattan_pearson |
|
value: 57.934500833835756 |
|
- type: manhattan_spearman |
|
value: 58.03263946623424 |
|
- task: |
|
type: STS |
|
dataset: |
|
type: mteb/sts22-crosslingual-sts |
|
name: MTEB STS22 (de) |
|
config: de |
|
split: test |
|
revision: 6d1ba47164174a496b7fa5d3569dae26a6813b80 |
|
metrics: |
|
- type: cos_sim_pearson |
|
value: 34.274519281072244 |
|
- type: cos_sim_spearman |
|
value: 41.84134494905925 |
|
- type: euclidean_pearson |
|
value: 24.113418166636 |
|
- type: euclidean_spearman |
|
value: 42.55202188864813 |
|
- type: manhattan_pearson |
|
value: 34.64265468569397 |
|
- type: manhattan_spearman |
|
value: 42.55202188864813 |
|
- task: |
|
type: STS |
|
dataset: |
|
type: mteb/sts22-crosslingual-sts |
|
name: MTEB STS22 (es) |
|
config: es |
|
split: test |
|
revision: 6d1ba47164174a496b7fa5d3569dae26a6813b80 |
|
metrics: |
|
- type: cos_sim_pearson |
|
value: 55.477886702880916 |
|
- type: cos_sim_spearman |
|
value: 57.226736875881365 |
|
- type: euclidean_pearson |
|
value: 51.58883207688278 |
|
- type: euclidean_spearman |
|
value: 57.86581420207087 |
|
- type: manhattan_pearson |
|
value: 55.6341174643668 |
|
- type: manhattan_spearman |
|
value: 57.86581420207087 |
|
- task: |
|
type: STS |
|
dataset: |
|
type: mteb/sts22-crosslingual-sts |
|
name: MTEB STS22 (pl) |
|
config: pl |
|
split: test |
|
revision: 6d1ba47164174a496b7fa5d3569dae26a6813b80 |
|
metrics: |
|
- type: cos_sim_pearson |
|
value: 20.336503083893273 |
|
- type: cos_sim_spearman |
|
value: 36.367365959741676 |
|
- type: euclidean_pearson |
|
value: 3.9896117703332306 |
|
- type: euclidean_spearman |
|
value: 35.58006670036499 |
|
- type: manhattan_pearson |
|
value: 19.472741193199475 |
|
- type: manhattan_spearman |
|
value: 35.58006670036499 |
|
- task: |
|
type: STS |
|
dataset: |
|
type: mteb/sts22-crosslingual-sts |
|
name: MTEB STS22 (tr) |
|
config: tr |
|
split: test |
|
revision: 6d1ba47164174a496b7fa5d3569dae26a6813b80 |
|
metrics: |
|
- type: cos_sim_pearson |
|
value: 52.55051438010185 |
|
- type: cos_sim_spearman |
|
value: 52.71302742082575 |
|
- type: euclidean_pearson |
|
value: 51.51870956964007 |
|
- type: euclidean_spearman |
|
value: 53.81785040820307 |
|
- type: manhattan_pearson |
|
value: 52.83864930315768 |
|
- type: manhattan_spearman |
|
value: 53.81785040820307 |
|
- task: |
|
type: STS |
|
dataset: |
|
type: mteb/sts22-crosslingual-sts |
|
name: MTEB STS22 (ar) |
|
config: ar |
|
split: test |
|
revision: 6d1ba47164174a496b7fa5d3569dae26a6813b80 |
|
metrics: |
|
- type: cos_sim_pearson |
|
value: 50.058410116717056 |
|
- type: cos_sim_spearman |
|
value: 52.60613795295755 |
|
- type: euclidean_pearson |
|
value: 44.34171068199546 |
|
- type: euclidean_spearman |
|
value: 50.972497500185995 |
|
- type: manhattan_pearson |
|
value: 48.47153098268435 |
|
- type: manhattan_spearman |
|
value: 50.972497500185995 |
|
- task: |
|
type: STS |
|
dataset: |
|
type: mteb/sts22-crosslingual-sts |
|
name: MTEB STS22 (ru) |
|
config: ru |
|
split: test |
|
revision: 6d1ba47164174a496b7fa5d3569dae26a6813b80 |
|
metrics: |
|
- type: cos_sim_pearson |
|
value: 48.18132407899186 |
|
- type: cos_sim_spearman |
|
value: 53.35340508300852 |
|
- type: euclidean_pearson |
|
value: 39.82149695080574 |
|
- type: euclidean_spearman |
|
value: 52.682446757364744 |
|
- type: manhattan_pearson |
|
value: 47.28762038747965 |
|
- type: manhattan_spearman |
|
value: 52.682446757364744 |
|
- task: |
|
type: STS |
|
dataset: |
|
type: mteb/sts22-crosslingual-sts |
|
name: MTEB STS22 (zh) |
|
config: zh |
|
split: test |
|
revision: 6d1ba47164174a496b7fa5d3569dae26a6813b80 |
|
metrics: |
|
- type: cos_sim_pearson |
|
value: 56.658087211796015 |
|
- type: cos_sim_spearman |
|
value: 60.00152778866955 |
|
- type: euclidean_pearson |
|
value: 49.64087381385087 |
|
- type: euclidean_spearman |
|
value: 60.15322267559951 |
|
- type: manhattan_pearson |
|
value: 56.343272070378504 |
|
- type: manhattan_spearman |
|
value: 60.15322267559951 |
|
- task: |
|
type: STS |
|
dataset: |
|
type: mteb/sts22-crosslingual-sts |
|
name: MTEB STS22 (fr) |
|
config: fr |
|
split: test |
|
revision: 6d1ba47164174a496b7fa5d3569dae26a6813b80 |
|
metrics: |
|
- type: cos_sim_pearson |
|
value: 70.45337327084312 |
|
- type: cos_sim_spearman |
|
value: 72.79410290057697 |
|
- type: euclidean_pearson |
|
value: 65.79888764581077 |
|
- type: euclidean_spearman |
|
value: 71.95723099514818 |
|
- type: manhattan_pearson |
|
value: 69.39143945386915 |
|
- type: manhattan_spearman |
|
value: 71.95723099514818 |
|
- task: |
|
type: STS |
|
dataset: |
|
type: mteb/sts22-crosslingual-sts |
|
name: MTEB STS22 (de-en) |
|
config: de-en |
|
split: test |
|
revision: 6d1ba47164174a496b7fa5d3569dae26a6813b80 |
|
metrics: |
|
- type: cos_sim_pearson |
|
value: 54.250555833893486 |
|
- type: cos_sim_spearman |
|
value: 49.08853609665319 |
|
- type: euclidean_pearson |
|
value: 56.41903104763859 |
|
- type: euclidean_spearman |
|
value: 48.5360965015595 |
|
- type: manhattan_pearson |
|
value: 55.42445266426144 |
|
- type: manhattan_spearman |
|
value: 48.5360965015595 |
|
- task: |
|
type: STS |
|
dataset: |
|
type: mteb/sts22-crosslingual-sts |
|
name: MTEB STS22 (es-en) |
|
config: es-en |
|
split: test |
|
revision: 6d1ba47164174a496b7fa5d3569dae26a6813b80 |
|
metrics: |
|
- type: cos_sim_pearson |
|
value: 66.77771892182398 |
|
- type: cos_sim_spearman |
|
value: 67.29191603287435 |
|
- type: euclidean_pearson |
|
value: 67.17511110245552 |
|
- type: euclidean_spearman |
|
value: 68.48737613290533 |
|
- type: manhattan_pearson |
|
value: 67.84988405103397 |
|
- type: manhattan_spearman |
|
value: 68.48737613290533 |
|
- task: |
|
type: STS |
|
dataset: |
|
type: mteb/sts22-crosslingual-sts |
|
name: MTEB STS22 (it) |
|
config: it |
|
split: test |
|
revision: 6d1ba47164174a496b7fa5d3569dae26a6813b80 |
|
metrics: |
|
- type: cos_sim_pearson |
|
value: 62.28155325846798 |
|
- type: cos_sim_spearman |
|
value: 64.16669097648895 |
|
- type: euclidean_pearson |
|
value: 59.403028984978356 |
|
- type: euclidean_spearman |
|
value: 64.53234398252941 |
|
- type: manhattan_pearson |
|
value: 62.71911466592815 |
|
- type: manhattan_spearman |
|
value: 64.53234398252941 |
|
- task: |
|
type: STS |
|
dataset: |
|
type: mteb/sts22-crosslingual-sts |
|
name: MTEB STS22 (pl-en) |
|
config: pl-en |
|
split: test |
|
revision: 6d1ba47164174a496b7fa5d3569dae26a6813b80 |
|
metrics: |
|
- type: cos_sim_pearson |
|
value: 66.52507293566482 |
|
- type: cos_sim_spearman |
|
value: 67.7160213688307 |
|
- type: euclidean_pearson |
|
value: 67.20401581128685 |
|
- type: euclidean_spearman |
|
value: 73.5516139257937 |
|
- type: manhattan_pearson |
|
value: 69.31380011990255 |
|
- type: manhattan_spearman |
|
value: 73.5516139257937 |
|
- task: |
|
type: STS |
|
dataset: |
|
type: mteb/sts22-crosslingual-sts |
|
name: MTEB STS22 (zh-en) |
|
config: zh-en |
|
split: test |
|
revision: 6d1ba47164174a496b7fa5d3569dae26a6813b80 |
|
metrics: |
|
- type: cos_sim_pearson |
|
value: 66.00687646805075 |
|
- type: cos_sim_spearman |
|
value: 64.45259281540577 |
|
- type: euclidean_pearson |
|
value: 67.27796918266225 |
|
- type: euclidean_spearman |
|
value: 63.85338920706559 |
|
- type: manhattan_pearson |
|
value: 67.1156006669401 |
|
- type: manhattan_spearman |
|
value: 63.85338920706559 |
|
- task: |
|
type: STS |
|
dataset: |
|
type: mteb/sts22-crosslingual-sts |
|
name: MTEB STS22 (es-it) |
|
config: es-it |
|
split: test |
|
revision: 6d1ba47164174a496b7fa5d3569dae26a6813b80 |
|
metrics: |
|
- type: cos_sim_pearson |
|
value: 58.377177955731966 |
|
- type: cos_sim_spearman |
|
value: 57.93025327632129 |
|
- type: euclidean_pearson |
|
value: 59.93402849184793 |
|
- type: euclidean_spearman |
|
value: 60.01820523185587 |
|
- type: manhattan_pearson |
|
value: 60.315338046091725 |
|
- type: manhattan_spearman |
|
value: 60.01820523185587 |
|
- task: |
|
type: STS |
|
dataset: |
|
type: mteb/sts22-crosslingual-sts |
|
name: MTEB STS22 (de-fr) |
|
config: de-fr |
|
split: test |
|
revision: 6d1ba47164174a496b7fa5d3569dae26a6813b80 |
|
metrics: |
|
- type: cos_sim_pearson |
|
value: 53.82667440921093 |
|
- type: cos_sim_spearman |
|
value: 50.5954961502418 |
|
- type: euclidean_pearson |
|
value: 55.73092376619234 |
|
- type: euclidean_spearman |
|
value: 55.313175399483484 |
|
- type: manhattan_pearson |
|
value: 56.81790111656754 |
|
- type: manhattan_spearman |
|
value: 55.313175399483484 |
|
- task: |
|
type: STS |
|
dataset: |
|
type: mteb/sts22-crosslingual-sts |
|
name: MTEB STS22 (de-pl) |
|
config: de-pl |
|
split: test |
|
revision: 6d1ba47164174a496b7fa5d3569dae26a6813b80 |
|
metrics: |
|
- type: cos_sim_pearson |
|
value: 37.23788982242752 |
|
- type: cos_sim_spearman |
|
value: 50.44074153238998 |
|
- type: euclidean_pearson |
|
value: 41.25620114235842 |
|
- type: euclidean_spearman |
|
value: 50.817224893459255 |
|
- type: manhattan_pearson |
|
value: 40.20839143792603 |
|
- type: manhattan_spearman |
|
value: 50.817224893459255 |
|
- task: |
|
type: STS |
|
dataset: |
|
type: mteb/sts22-crosslingual-sts |
|
name: MTEB STS22 (fr-pl) |
|
config: fr-pl |
|
split: test |
|
revision: 6d1ba47164174a496b7fa5d3569dae26a6813b80 |
|
metrics: |
|
- type: cos_sim_pearson |
|
value: 58.03829696246709 |
|
- type: cos_sim_spearman |
|
value: 73.24670207647144 |
|
- type: euclidean_pearson |
|
value: 55.854312917676864 |
|
- type: euclidean_spearman |
|
value: 73.24670207647144 |
|
- type: manhattan_pearson |
|
value: 58.529125221260614 |
|
- type: manhattan_spearman |
|
value: 73.24670207647144 |
|
- task: |
|
type: STS |
|
dataset: |
|
type: mteb/stsbenchmark-sts |
|
name: MTEB STSBenchmark |
|
config: default |
|
split: test |
|
revision: b0fddb56ed78048fa8b90373c8a3cfc37b684831 |
|
metrics: |
|
- type: cos_sim_pearson |
|
value: 82.10559795910007 |
|
- type: cos_sim_spearman |
|
value: 81.33502456405203 |
|
- type: euclidean_pearson |
|
value: 80.71725031531976 |
|
- type: euclidean_spearman |
|
value: 81.48140012027567 |
|
- type: manhattan_pearson |
|
value: 82.33088191846421 |
|
- type: manhattan_spearman |
|
value: 81.48140012027567 |
|
- task: |
|
type: PairClassification |
|
dataset: |
|
type: mteb/sprintduplicatequestions-pairclassification |
|
name: MTEB SprintDuplicateQuestions |
|
config: default |
|
split: test |
|
revision: d66bd1f72af766a5cc4b0ca5e00c162f89e8cc46 |
|
metrics: |
|
- type: cos_sim_accuracy |
|
value: 99.47227722772277 |
|
- type: cos_sim_ap |
|
value: 77.36042895972905 |
|
- type: cos_sim_f1 |
|
value: 72.23880597014924 |
|
- type: cos_sim_precision |
|
value: 71.88118811881188 |
|
- type: cos_sim_recall |
|
value: 72.6 |
|
- type: dot_accuracy |
|
value: 99.409900990099 |
|
- type: dot_ap |
|
value: 68.42835773716114 |
|
- type: dot_f1 |
|
value: 65.83783783783784 |
|
- type: dot_precision |
|
value: 71.6470588235294 |
|
- type: dot_recall |
|
value: 60.9 |
|
- type: euclidean_accuracy |
|
value: 99.48019801980197 |
|
- type: euclidean_ap |
|
value: 76.69004973047716 |
|
- type: euclidean_f1 |
|
value: 72.51638930912759 |
|
- type: euclidean_precision |
|
value: 73.14343845371313 |
|
- type: euclidean_recall |
|
value: 71.89999999999999 |
|
- type: manhattan_accuracy |
|
value: 99.48019801980197 |
|
- type: manhattan_ap |
|
value: 76.69004973047716 |
|
- type: manhattan_f1 |
|
value: 72.51638930912759 |
|
- type: manhattan_precision |
|
value: 73.14343845371313 |
|
- type: manhattan_recall |
|
value: 71.89999999999999 |
|
- type: max_accuracy |
|
value: 99.48019801980197 |
|
- type: max_ap |
|
value: 77.36042895972905 |
|
- type: max_f1 |
|
value: 72.51638930912759 |
|
- task: |
|
type: Classification |
|
dataset: |
|
type: mteb/toxic_conversations_50k |
|
name: MTEB ToxicConversationsClassification |
|
config: default |
|
split: test |
|
revision: d7c0de2777da35d6aae2200a62c6e0e5af397c4c |
|
metrics: |
|
- type: accuracy |
|
value: 70.2614 |
|
- type: ap |
|
value: 13.421228681716107 |
|
- type: f1 |
|
value: 53.71534671651974 |
|
- task: |
|
type: Classification |
|
dataset: |
|
type: mteb/tweet_sentiment_extraction |
|
name: MTEB TweetSentimentExtractionClassification |
|
config: default |
|
split: test |
|
revision: d604517c81ca91fe16a244d1248fc021f9ecee7a |
|
metrics: |
|
- type: accuracy |
|
value: 54.48783248443689 |
|
- type: f1 |
|
value: 54.7405015752634 |
|
- task: |
|
type: PairClassification |
|
dataset: |
|
type: mteb/twittersemeval2015-pairclassification |
|
name: MTEB TwitterSemEval2015 |
|
config: default |
|
split: test |
|
revision: 70970daeab8776df92f5ea462b6173c0b46fd2d1 |
|
metrics: |
|
- type: cos_sim_accuracy |
|
value: 83.22703701496096 |
|
- type: cos_sim_ap |
|
value: 63.58031791834936 |
|
- type: cos_sim_f1 |
|
value: 59.3132854578097 |
|
- type: cos_sim_precision |
|
value: 51.60093713393206 |
|
- type: cos_sim_recall |
|
value: 69.73614775725594 |
|
- type: dot_accuracy |
|
value: 81.96936281814389 |
|
- type: dot_ap |
|
value: 59.07547966241098 |
|
- type: dot_f1 |
|
value: 56.032535020334386 |
|
- type: dot_precision |
|
value: 48.99249308573686 |
|
- type: dot_recall |
|
value: 65.4353562005277 |
|
- type: euclidean_accuracy |
|
value: 83.26280026226381 |
|
- type: euclidean_ap |
|
value: 63.64817520735364 |
|
- type: euclidean_f1 |
|
value: 59.91221653255303 |
|
- type: euclidean_precision |
|
value: 55.68902991840435 |
|
- type: euclidean_recall |
|
value: 64.82849604221636 |
|
- type: manhattan_accuracy |
|
value: 83.26280026226381 |
|
- type: manhattan_ap |
|
value: 63.64817520735364 |
|
- type: manhattan_f1 |
|
value: 59.91221653255303 |
|
- type: manhattan_precision |
|
value: 55.68902991840435 |
|
- type: manhattan_recall |
|
value: 64.82849604221636 |
|
- type: max_accuracy |
|
value: 83.26280026226381 |
|
- type: max_ap |
|
value: 63.64817520735364 |
|
- type: max_f1 |
|
value: 59.91221653255303 |
|
- task: |
|
type: PairClassification |
|
dataset: |
|
type: mteb/twitterurlcorpus-pairclassification |
|
name: MTEB TwitterURLCorpus |
|
config: default |
|
split: test |
|
revision: 8b6510b0b1fa4e4c4f879467980e9be563ec1cdf |
|
metrics: |
|
- type: cos_sim_accuracy |
|
value: 87.49563395040167 |
|
- type: cos_sim_ap |
|
value: 82.6398035947217 |
|
- type: cos_sim_f1 |
|
value: 74.74134990715125 |
|
- type: cos_sim_precision |
|
value: 73.59504440629898 |
|
- type: cos_sim_recall |
|
value: 75.92392978133662 |
|
- type: dot_accuracy |
|
value: 85.70264291535685 |
|
- type: dot_ap |
|
value: 76.35175453791561 |
|
- type: dot_f1 |
|
value: 70.42039872869113 |
|
- type: dot_precision |
|
value: 66.31972789115646 |
|
- type: dot_recall |
|
value: 75.06159531875576 |
|
- type: euclidean_accuracy |
|
value: 87.51503861528312 |
|
- type: euclidean_ap |
|
value: 82.74416973508781 |
|
- type: euclidean_f1 |
|
value: 75.0812647754137 |
|
- type: euclidean_precision |
|
value: 72.15989775631922 |
|
- type: euclidean_recall |
|
value: 78.2491530643671 |
|
- type: manhattan_accuracy |
|
value: 87.51503861528312 |
|
- type: manhattan_ap |
|
value: 82.74416973508781 |
|
- type: manhattan_f1 |
|
value: 75.0812647754137 |
|
- type: manhattan_precision |
|
value: 72.15989775631922 |
|
- type: manhattan_recall |
|
value: 78.2491530643671 |
|
- type: max_accuracy |
|
value: 87.51503861528312 |
|
- type: max_ap |
|
value: 82.74416973508781 |
|
- type: max_f1 |
|
value: 75.0812647754137 |
|
--- |
|
# paraphrase-multilingual-mpnet-base-v2-KE_Sieve |
|
|
|
<!-- Provide a quick summary of what the model is/does. --> |
|
|
|
This modelcard aims to be a base template for new models. It has been generated using [this raw template](https://github.com/huggingface/huggingface_hub/blob/main/src/huggingface_hub/templates/modelcard_template.md?plain=1). |
|
|
|
## Model Details |
|
|
|
### Model Description |
|
|
|
<!-- Provide a longer summary of what this model is. --> |
|
|
|
|
|
|
|
- **Developed by:** [More Information Needed] |
|
- **Shared by [optional]:** [More Information Needed] |
|
- **Model type:** [More Information Needed] |
|
- **Language(s) (NLP):** [More Information Needed] |
|
- **License:** [More Information Needed] |
|
- **Finetuned from model [optional]:** [More Information Needed] |
|
|
|
### Model Sources [optional] |
|
|
|
<!-- Provide the basic links for the model. --> |
|
|
|
- **Repository:** [More Information Needed] |
|
- **Paper [optional]:** [More Information Needed] |
|
- **Demo [optional]:** [More Information Needed] |
|
|
|
## Uses |
|
|
|
<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. --> |
|
|
|
### Direct Use |
|
|
|
<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. --> |
|
|
|
[More Information Needed] |
|
|
|
### Downstream Use [optional] |
|
|
|
<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app --> |
|
|
|
[More Information Needed] |
|
|
|
### Out-of-Scope Use |
|
|
|
<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. --> |
|
|
|
[More Information Needed] |
|
|
|
## Bias, Risks, and Limitations |
|
|
|
<!-- This section is meant to convey both technical and sociotechnical limitations. --> |
|
|
|
[More Information Needed] |
|
|
|
### Recommendations |
|
|
|
<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. --> |
|
|
|
Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations. |
|
|
|
## How to Get Started with the Model |
|
|
|
Use the code below to get started with the model. |
|
|
|
[More Information Needed] |
|
|
|
## Training Details |
|
|
|
### Training Data |
|
|
|
<!-- This should link to a Data Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. --> |
|
|
|
[More Information Needed] |
|
|
|
### Training Procedure |
|
|
|
<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. --> |
|
|
|
#### Preprocessing [optional] |
|
|
|
[More Information Needed] |
|
|
|
|
|
#### Training Hyperparameters |
|
|
|
- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision --> |
|
|
|
#### Speeds, Sizes, Times [optional] |
|
|
|
<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. --> |
|
|
|
[More Information Needed] |
|
|
|
## Evaluation |
|
|
|
<!-- This section describes the evaluation protocols and provides the results. --> |
|
|
|
### Testing Data, Factors & Metrics |
|
|
|
#### Testing Data |
|
|
|
<!-- This should link to a Data Card if possible. --> |
|
|
|
[More Information Needed] |
|
|
|
#### Factors |
|
|
|
<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. --> |
|
|
|
[More Information Needed] |
|
|
|
#### Metrics |
|
|
|
<!-- These are the evaluation metrics being used, ideally with a description of why. --> |
|
|
|
[More Information Needed] |
|
|
|
### Results |
|
|
|
[More Information Needed] |
|
|
|
#### Summary |
|
|
|
|
|
|
|
## Model Examination [optional] |
|
|
|
<!-- Relevant interpretability work for the model goes here --> |
|
|
|
[More Information Needed] |
|
|
|
## Environmental Impact |
|
|
|
<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly --> |
|
|
|
Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700). |
|
|
|
- **Hardware Type:** [More Information Needed] |
|
- **Hours used:** [More Information Needed] |
|
- **Cloud Provider:** [More Information Needed] |
|
- **Compute Region:** [More Information Needed] |
|
- **Carbon Emitted:** [More Information Needed] |
|
|
|
## Technical Specifications [optional] |
|
|
|
### Model Architecture and Objective |
|
|
|
[More Information Needed] |
|
|
|
### Compute Infrastructure |
|
|
|
[More Information Needed] |
|
|
|
#### Hardware |
|
|
|
[More Information Needed] |
|
|
|
#### Software |
|
|
|
[More Information Needed] |
|
|
|
## Citation [optional] |
|
|
|
<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. --> |
|
|
|
**BibTeX:** |
|
|
|
[More Information Needed] |
|
|
|
**APA:** |
|
|
|
[More Information Needed] |
|
|
|
## Glossary [optional] |
|
|
|
<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. --> |
|
|
|
[More Information Needed] |
|
|
|
## More Information [optional] |
|
|
|
[More Information Needed] |
|
|
|
## Model Card Authors [optional] |
|
|
|
[More Information Needed] |
|
|
|
## Model Card Contact |
|
|
|
[More Information Needed] |
|
|
|
|
|
|
