--- pipeline_tag: sentence-similarity tags: - sentence-transformers - feature-extraction - sentence-similarity - transformers - semantic-search - chinese - mteb model-index: - name: sbert-chinese-general-v1 results: - task: type: STS dataset: type: C-MTEB/AFQMC name: MTEB AFQMC config: default split: validation revision: None metrics: - type: cos_sim_pearson value: 22.293919432958074 - type: cos_sim_spearman value: 22.56718923553609 - type: euclidean_pearson value: 22.525656322797026 - type: euclidean_spearman value: 22.56718923553609 - type: manhattan_pearson value: 22.501773028824065 - type: manhattan_spearman value: 22.536992587828397 - task: type: STS dataset: type: C-MTEB/ATEC name: MTEB ATEC config: default split: test revision: None metrics: - type: cos_sim_pearson value: 30.33575274463879 - type: cos_sim_spearman value: 30.298708742167772 - type: euclidean_pearson value: 32.33094743729218 - type: euclidean_spearman value: 30.298710993858734 - type: manhattan_pearson value: 32.31155376195945 - type: manhattan_spearman value: 30.267669681690744 - task: type: Classification dataset: type: mteb/amazon_reviews_multi name: MTEB AmazonReviewsClassification (zh) config: zh split: test revision: 1399c76144fd37290681b995c656ef9b2e06e26d metrics: - type: accuracy value: 37.507999999999996 - type: f1 value: 36.436808400753286 - task: type: STS dataset: type: C-MTEB/BQ name: MTEB BQ config: default split: test revision: None metrics: - type: cos_sim_pearson value: 41.493256724214255 - type: cos_sim_spearman value: 40.98395961967895 - type: euclidean_pearson value: 41.12345737966565 - type: euclidean_spearman value: 40.983959619555996 - type: manhattan_pearson value: 41.02584539471014 - type: manhattan_spearman value: 40.87549513383032 - task: type: BitextMining dataset: type: mteb/bucc-bitext-mining name: MTEB BUCC (zh-en) config: zh-en split: test revision: d51519689f32196a32af33b075a01d0e7c51e252 metrics: - type: accuracy value: 9.794628751974724 - type: f1 value: 9.350535369492716 - type: precision value: 9.179392662804986 - type: recall value: 9.794628751974724 - task: type: Clustering dataset: type: C-MTEB/CLSClusteringP2P name: MTEB CLSClusteringP2P config: default split: test revision: None metrics: - type: v_measure value: 34.984726547788284 - task: type: Clustering dataset: type: C-MTEB/CLSClusteringS2S name: MTEB CLSClusteringS2S config: default split: test revision: None metrics: - type: v_measure value: 27.81945732281589 - task: type: Reranking dataset: type: C-MTEB/CMedQAv1-reranking name: MTEB CMedQAv1 config: default split: test revision: None metrics: - type: map value: 53.06586280826805 - type: mrr value: 59.58781746031746 - task: type: Reranking dataset: type: C-MTEB/CMedQAv2-reranking name: MTEB CMedQAv2 config: default split: test revision: None metrics: - type: map value: 52.83635946154306 - type: mrr value: 59.315079365079356 - task: type: Retrieval dataset: type: C-MTEB/CmedqaRetrieval name: MTEB CmedqaRetrieval config: default split: dev revision: None metrics: - type: map_at_1 value: 5.721 - type: map_at_10 value: 8.645 - type: map_at_100 value: 9.434 - type: map_at_1000 value: 9.586 - type: map_at_3 value: 7.413 - type: map_at_5 value: 8.05 - type: mrr_at_1 value: 9.626999999999999 - type: mrr_at_10 value: 13.094 - type: mrr_at_100 value: 13.854 - type: mrr_at_1000 value: 13.958 - type: mrr_at_3 value: 11.724 - type: mrr_at_5 value: 12.409 - type: ndcg_at_1 value: 9.626999999999999 - type: ndcg_at_10 value: 11.35 - type: ndcg_at_100 value: 15.593000000000002 - type: ndcg_at_1000 value: 19.619 - type: ndcg_at_3 value: 9.317 - type: ndcg_at_5 value: 10.049 - type: precision_at_1 value: 9.626999999999999 - type: precision_at_10 value: 2.796 - type: precision_at_100 value: 0.629 - type: precision_at_1000 value: 0.11800000000000001 - type: precision_at_3 value: 5.476 - type: precision_at_5 value: 4.1209999999999996 - type: recall_at_1 value: 5.721 - type: recall_at_10 value: 15.190000000000001 - type: recall_at_100 value: 33.633 - type: recall_at_1000 value: 62.019999999999996 - type: recall_at_3 value: 9.099 - type: recall_at_5 value: 11.423 - task: type: PairClassification dataset: type: C-MTEB/CMNLI name: MTEB Cmnli config: default split: validation revision: None metrics: - type: cos_sim_accuracy value: 77.36620565243535 - type: cos_sim_ap value: 85.92291866877001 - type: cos_sim_f1 value: 78.19390231037029 - type: cos_sim_precision value: 71.24183006535948 - type: cos_sim_recall value: 86.64952069207388 - type: dot_accuracy value: 77.36620565243535 - type: dot_ap value: 85.94113738490068 - type: dot_f1 value: 78.19390231037029 - type: dot_precision value: 71.24183006535948 - type: dot_recall value: 86.64952069207388 - type: euclidean_accuracy value: 77.36620565243535 - type: euclidean_ap value: 85.92291893444687 - type: euclidean_f1 value: 78.19390231037029 - type: euclidean_precision value: 71.24183006535948 - type: euclidean_recall value: 86.64952069207388 - type: manhattan_accuracy value: 77.29404690318701 - type: manhattan_ap value: 85.88284362100919 - type: manhattan_f1 value: 78.17836812144213 - type: manhattan_precision value: 71.18448838548666 - type: manhattan_recall value: 86.69628244096329 - type: max_accuracy value: 77.36620565243535 - type: max_ap value: 85.94113738490068 - type: max_f1 value: 78.19390231037029 - task: type: Retrieval dataset: type: C-MTEB/CovidRetrieval name: MTEB CovidRetrieval config: default split: dev revision: None metrics: - type: map_at_1 value: 26.976 - type: map_at_10 value: 35.18 - type: map_at_100 value: 35.921 - type: map_at_1000 value: 35.998999999999995 - type: map_at_3 value: 32.763 - type: map_at_5 value: 34.165 - type: mrr_at_1 value: 26.976 - type: mrr_at_10 value: 35.234 - type: mrr_at_100 value: 35.939 - type: mrr_at_1000 value: 36.016 - type: mrr_at_3 value: 32.771 - type: mrr_at_5 value: 34.172999999999995 - type: ndcg_at_1 value: 26.976 - type: ndcg_at_10 value: 39.635 - type: ndcg_at_100 value: 43.54 - type: ndcg_at_1000 value: 45.723 - type: ndcg_at_3 value: 34.652 - type: ndcg_at_5 value: 37.186 - type: precision_at_1 value: 26.976 - type: precision_at_10 value: 5.406 - type: precision_at_100 value: 0.736 - type: precision_at_1000 value: 0.091 - type: precision_at_3 value: 13.418 - type: precision_at_5 value: 9.293999999999999 - type: recall_at_1 value: 26.976 - type: recall_at_10 value: 53.766999999999996 - type: recall_at_100 value: 72.761 - type: recall_at_1000 value: 90.148 - type: recall_at_3 value: 40.095 - type: recall_at_5 value: 46.233000000000004 - task: type: Retrieval dataset: type: C-MTEB/DuRetrieval name: MTEB DuRetrieval config: default split: dev revision: None metrics: - type: map_at_1 value: 11.285 - type: map_at_10 value: 30.259000000000004 - type: map_at_100 value: 33.772000000000006 - type: map_at_1000 value: 34.037 - type: map_at_3 value: 21.038999999999998 - type: map_at_5 value: 25.939 - type: mrr_at_1 value: 45.1 - type: mrr_at_10 value: 55.803999999999995 - type: mrr_at_100 value: 56.301 - type: mrr_at_1000 value: 56.330999999999996 - type: mrr_at_3 value: 53.333 - type: mrr_at_5 value: 54.798 - type: ndcg_at_1 value: 45.1 - type: ndcg_at_10 value: 41.156 - type: ndcg_at_100 value: 49.518 - type: ndcg_at_1000 value: 52.947 - type: ndcg_at_3 value: 39.708 - type: ndcg_at_5 value: 38.704 - type: precision_at_1 value: 45.1 - type: precision_at_10 value: 20.75 - type: precision_at_100 value: 3.424 - type: precision_at_1000 value: 0.42700000000000005 - type: precision_at_3 value: 35.632999999999996 - type: precision_at_5 value: 30.080000000000002 - type: recall_at_1 value: 11.285 - type: recall_at_10 value: 43.242000000000004 - type: recall_at_100 value: 68.604 - type: recall_at_1000 value: 85.904 - type: recall_at_3 value: 24.404 - type: recall_at_5 value: 32.757 - task: type: Retrieval dataset: type: C-MTEB/EcomRetrieval name: MTEB EcomRetrieval config: default split: dev revision: None metrics: - type: map_at_1 value: 21 - type: map_at_10 value: 28.364 - type: map_at_100 value: 29.199 - type: map_at_1000 value: 29.265 - type: map_at_3 value: 25.717000000000002 - type: map_at_5 value: 27.311999999999998 - type: mrr_at_1 value: 21 - type: mrr_at_10 value: 28.364 - type: mrr_at_100 value: 29.199 - type: mrr_at_1000 value: 29.265 - type: mrr_at_3 value: 25.717000000000002 - type: mrr_at_5 value: 27.311999999999998 - type: ndcg_at_1 value: 21 - type: ndcg_at_10 value: 32.708 - type: ndcg_at_100 value: 37.184 - type: ndcg_at_1000 value: 39.273 - type: ndcg_at_3 value: 27.372000000000003 - type: ndcg_at_5 value: 30.23 - type: precision_at_1 value: 21 - type: precision_at_10 value: 4.66 - type: precision_at_100 value: 0.685 - type: precision_at_1000 value: 0.086 - type: precision_at_3 value: 10.732999999999999 - type: precision_at_5 value: 7.82 - type: recall_at_1 value: 21 - type: recall_at_10 value: 46.6 - type: recall_at_100 value: 68.5 - type: recall_at_1000 value: 85.6 - type: recall_at_3 value: 32.2 - type: recall_at_5 value: 39.1 - task: type: Classification dataset: type: C-MTEB/IFlyTek-classification name: MTEB IFlyTek config: default split: validation revision: None metrics: - type: accuracy value: 44.878799538283964 - type: f1 value: 33.84678310261366 - task: type: Classification dataset: type: C-MTEB/JDReview-classification name: MTEB JDReview config: default split: test revision: None metrics: - type: accuracy value: 82.1951219512195 - type: ap value: 46.78292030042397 - type: f1 value: 76.20482468514128 - task: type: STS dataset: type: C-MTEB/LCQMC name: MTEB LCQMC config: default split: test revision: None metrics: - type: cos_sim_pearson value: 62.84331627244547 - type: cos_sim_spearman value: 68.39990265073726 - type: euclidean_pearson value: 66.87431827169324 - type: euclidean_spearman value: 68.39990264979167 - type: manhattan_pearson value: 66.89702078900328 - type: manhattan_spearman value: 68.42107302159141 - task: type: Reranking dataset: type: C-MTEB/Mmarco-reranking name: MTEB MMarcoReranking config: default split: dev revision: None metrics: - type: map value: 9.28600891904827 - type: mrr value: 8.057936507936509 - task: type: Retrieval dataset: type: C-MTEB/MMarcoRetrieval name: MTEB MMarcoRetrieval config: default split: dev revision: None metrics: - type: map_at_1 value: 22.820999999999998 - type: map_at_10 value: 30.44 - type: map_at_100 value: 31.35 - type: map_at_1000 value: 31.419000000000004 - type: map_at_3 value: 28.134999999999998 - type: map_at_5 value: 29.482000000000003 - type: mrr_at_1 value: 23.782 - type: mrr_at_10 value: 31.141999999999996 - type: mrr_at_100 value: 32.004 - type: mrr_at_1000 value: 32.068000000000005 - type: mrr_at_3 value: 28.904000000000003 - type: mrr_at_5 value: 30.214999999999996 - type: ndcg_at_1 value: 23.782 - type: ndcg_at_10 value: 34.625 - type: ndcg_at_100 value: 39.226 - type: ndcg_at_1000 value: 41.128 - type: ndcg_at_3 value: 29.968 - type: ndcg_at_5 value: 32.35 - type: precision_at_1 value: 23.782 - type: precision_at_10 value: 4.994 - type: precision_at_100 value: 0.736 - type: precision_at_1000 value: 0.09 - type: precision_at_3 value: 12.13 - type: precision_at_5 value: 8.495999999999999 - type: recall_at_1 value: 22.820999999999998 - type: recall_at_10 value: 47.141 - type: recall_at_100 value: 68.952 - type: recall_at_1000 value: 83.985 - type: recall_at_3 value: 34.508 - type: recall_at_5 value: 40.232 - task: type: Classification dataset: type: mteb/amazon_massive_intent name: MTEB MassiveIntentClassification (zh-CN) config: zh-CN split: test revision: 31efe3c427b0bae9c22cbb560b8f15491cc6bed7 metrics: - type: accuracy value: 57.343644922663074 - type: f1 value: 56.744802953803486 - task: type: Classification dataset: type: mteb/amazon_massive_scenario name: MTEB MassiveScenarioClassification (zh-CN) config: zh-CN split: test revision: 7d571f92784cd94a019292a1f45445077d0ef634 metrics: - type: accuracy value: 62.363819771351714 - type: f1 value: 62.15920863434656 - task: type: Retrieval dataset: type: C-MTEB/MedicalRetrieval name: MTEB MedicalRetrieval config: default split: dev revision: None metrics: - type: map_at_1 value: 14.6 - type: map_at_10 value: 18.231 - type: map_at_100 value: 18.744 - type: map_at_1000 value: 18.811 - type: map_at_3 value: 17.133000000000003 - type: map_at_5 value: 17.663 - type: mrr_at_1 value: 14.6 - type: mrr_at_10 value: 18.231 - type: mrr_at_100 value: 18.744 - type: mrr_at_1000 value: 18.811 - type: mrr_at_3 value: 17.133000000000003 - type: mrr_at_5 value: 17.663 - type: ndcg_at_1 value: 14.6 - type: ndcg_at_10 value: 20.349 - type: ndcg_at_100 value: 23.204 - type: ndcg_at_1000 value: 25.44 - type: ndcg_at_3 value: 17.995 - type: ndcg_at_5 value: 18.945999999999998 - type: precision_at_1 value: 14.6 - type: precision_at_10 value: 2.7199999999999998 - type: precision_at_100 value: 0.414 - type: precision_at_1000 value: 0.06 - type: precision_at_3 value: 6.833 - type: precision_at_5 value: 4.5600000000000005 - type: recall_at_1 value: 14.6 - type: recall_at_10 value: 27.200000000000003 - type: recall_at_100 value: 41.4 - type: recall_at_1000 value: 60 - type: recall_at_3 value: 20.5 - type: recall_at_5 value: 22.8 - task: type: Classification dataset: type: C-MTEB/MultilingualSentiment-classification name: MTEB MultilingualSentiment config: default split: validation revision: None metrics: - type: accuracy value: 66.58333333333333 - type: f1 value: 66.26700927460007 - task: type: PairClassification dataset: type: C-MTEB/OCNLI name: MTEB Ocnli config: default split: validation revision: None metrics: - type: cos_sim_accuracy value: 72.00866269626421 - type: cos_sim_ap value: 77.00520104243304 - type: cos_sim_f1 value: 74.39303710490151 - type: cos_sim_precision value: 65.69579288025889 - type: cos_sim_recall value: 85.74445617740233 - type: dot_accuracy value: 72.00866269626421 - type: dot_ap value: 77.00520104243304 - type: dot_f1 value: 74.39303710490151 - type: dot_precision value: 65.69579288025889 - type: dot_recall value: 85.74445617740233 - type: euclidean_accuracy value: 72.00866269626421 - type: euclidean_ap value: 77.00520104243304 - type: euclidean_f1 value: 74.39303710490151 - type: euclidean_precision value: 65.69579288025889 - type: euclidean_recall value: 85.74445617740233 - type: manhattan_accuracy value: 72.1710882512182 - type: manhattan_ap value: 77.00551017913976 - type: manhattan_f1 value: 74.23423423423424 - type: manhattan_precision value: 64.72898664571878 - type: manhattan_recall value: 87.0116156282999 - type: max_accuracy value: 72.1710882512182 - type: max_ap value: 77.00551017913976 - type: max_f1 value: 74.39303710490151 - task: type: Classification dataset: type: C-MTEB/OnlineShopping-classification name: MTEB OnlineShopping config: default split: test revision: None metrics: - type: accuracy value: 88.19000000000001 - type: ap value: 85.13415594781077 - type: f1 value: 88.17344156114062 - task: type: STS dataset: type: C-MTEB/PAWSX name: MTEB PAWSX config: default split: test revision: None metrics: - type: cos_sim_pearson value: 13.70522140998517 - type: cos_sim_spearman value: 15.07546667334743 - type: euclidean_pearson value: 17.49511420225285 - type: euclidean_spearman value: 15.093970931789618 - type: manhattan_pearson value: 17.44069961390521 - type: manhattan_spearman value: 15.076029291596962 - task: type: STS dataset: type: C-MTEB/QBQTC name: MTEB QBQTC config: default split: test revision: None metrics: - type: cos_sim_pearson value: 26.835294224547155 - type: cos_sim_spearman value: 27.920204597498856 - type: euclidean_pearson value: 26.153796707702803 - type: euclidean_spearman value: 27.920971379720548 - type: manhattan_pearson value: 26.21954147857523 - type: manhattan_spearman value: 27.996860049937478 - task: type: STS dataset: type: mteb/sts22-crosslingual-sts name: MTEB STS22 (zh) config: zh split: test revision: 6d1ba47164174a496b7fa5d3569dae26a6813b80 metrics: - type: cos_sim_pearson value: 55.15901259718581 - type: cos_sim_spearman value: 61.57967880874167 - type: euclidean_pearson value: 53.83523291596683 - type: euclidean_spearman value: 61.57967880874167 - type: manhattan_pearson value: 54.99971428907956 - type: manhattan_spearman value: 61.61229543613867 - task: type: STS dataset: type: mteb/sts22-crosslingual-sts name: MTEB STS22 (zh-en) config: zh-en split: test revision: 6d1ba47164174a496b7fa5d3569dae26a6813b80 metrics: - type: cos_sim_pearson value: 34.20930208460845 - type: cos_sim_spearman value: 33.879011104224524 - type: euclidean_pearson value: 35.08526425284862 - type: euclidean_spearman value: 33.879011104224524 - type: manhattan_pearson value: 35.509419089701275 - type: manhattan_spearman value: 33.30035487147621 - task: type: STS dataset: type: C-MTEB/STSB name: MTEB STSB config: default split: test revision: None metrics: - type: cos_sim_pearson value: 82.30068282185835 - type: cos_sim_spearman value: 82.16763221361724 - type: euclidean_pearson value: 80.52772752433374 - type: euclidean_spearman value: 82.16797037220333 - type: manhattan_pearson value: 80.51093859500105 - type: manhattan_spearman value: 82.17643310049654 - task: type: Reranking dataset: type: C-MTEB/T2Reranking name: MTEB T2Reranking config: default split: dev revision: None metrics: - type: map value: 65.14113035189213 - type: mrr value: 74.9589270937443 - task: type: Retrieval dataset: type: C-MTEB/T2Retrieval name: MTEB T2Retrieval config: default split: dev revision: None metrics: - type: map_at_1 value: 12.013 - type: map_at_10 value: 30.885 - type: map_at_100 value: 34.643 - type: map_at_1000 value: 34.927 - type: map_at_3 value: 21.901 - type: map_at_5 value: 26.467000000000002 - type: mrr_at_1 value: 49.623 - type: mrr_at_10 value: 58.05200000000001 - type: mrr_at_100 value: 58.61300000000001 - type: mrr_at_1000 value: 58.643 - type: mrr_at_3 value: 55.947 - type: mrr_at_5 value: 57.229 - type: ndcg_at_1 value: 49.623 - type: ndcg_at_10 value: 41.802 - type: ndcg_at_100 value: 49.975 - type: ndcg_at_1000 value: 53.504 - type: ndcg_at_3 value: 43.515 - type: ndcg_at_5 value: 41.576 - type: precision_at_1 value: 49.623 - type: precision_at_10 value: 22.052 - type: precision_at_100 value: 3.6450000000000005 - type: precision_at_1000 value: 0.45399999999999996 - type: precision_at_3 value: 38.616 - type: precision_at_5 value: 31.966 - type: recall_at_1 value: 12.013 - type: recall_at_10 value: 41.891 - type: recall_at_100 value: 67.096 - type: recall_at_1000 value: 84.756 - type: recall_at_3 value: 24.695 - type: recall_at_5 value: 32.09 - task: type: Classification dataset: type: C-MTEB/TNews-classification name: MTEB TNews config: default split: validation revision: None metrics: - type: accuracy value: 39.800999999999995 - type: f1 value: 38.5345899934575 - task: type: Clustering dataset: type: C-MTEB/ThuNewsClusteringP2P name: MTEB ThuNewsClusteringP2P config: default split: test revision: None metrics: - type: v_measure value: 40.16574242797479 - task: type: Clustering dataset: type: C-MTEB/ThuNewsClusteringS2S name: MTEB ThuNewsClusteringS2S config: default split: test revision: None metrics: - type: v_measure value: 24.232617974671754 - task: type: Retrieval dataset: type: C-MTEB/VideoRetrieval name: MTEB VideoRetrieval config: default split: dev revision: None metrics: - type: map_at_1 value: 24.6 - type: map_at_10 value: 31.328 - type: map_at_100 value: 32.088 - type: map_at_1000 value: 32.164 - type: map_at_3 value: 29.133 - type: map_at_5 value: 30.358 - type: mrr_at_1 value: 24.6 - type: mrr_at_10 value: 31.328 - type: mrr_at_100 value: 32.088 - type: mrr_at_1000 value: 32.164 - type: mrr_at_3 value: 29.133 - type: mrr_at_5 value: 30.358 - type: ndcg_at_1 value: 24.6 - type: ndcg_at_10 value: 35.150999999999996 - type: ndcg_at_100 value: 39.024 - type: ndcg_at_1000 value: 41.157 - type: ndcg_at_3 value: 30.637999999999998 - type: ndcg_at_5 value: 32.833 - type: precision_at_1 value: 24.6 - type: precision_at_10 value: 4.74 - type: precision_at_100 value: 0.66 - type: precision_at_1000 value: 0.083 - type: precision_at_3 value: 11.667 - type: precision_at_5 value: 8.06 - type: recall_at_1 value: 24.6 - type: recall_at_10 value: 47.4 - type: recall_at_100 value: 66 - type: recall_at_1000 value: 83 - type: recall_at_3 value: 35 - type: recall_at_5 value: 40.300000000000004 - task: type: Classification dataset: type: C-MTEB/waimai-classification name: MTEB Waimai config: default split: test revision: None metrics: - type: accuracy value: 83.96000000000001 - type: ap value: 65.11027167433211 - type: f1 value: 82.03549710974653 license: apache-2.0 language: - zh --- # DMetaSoul/sbert-chinese-general-v1 此模型基于 [bert-base-chinese](https://huggingface.co/bert-base-chinese) 版本 BERT 模型，在 NLI、PAWS-X、PKU-Paraphrase-Bank、STS 等语义相似数据集上进行训练，适用于**通用语义匹配**场景（此模型在 Chinese-STS 任务上效果较好，但在其它任务上效果并非最优，存在一定过拟合风险），比如文本特征抽取、文本向量聚类、文本语义搜索等业务场景。 注：此模型的[轻量化版本](https://huggingface.co/DMetaSoul/sbert-chinese-general-v1-distill)，也已经开源啦！ # Usage ## 1. Sentence-Transformers 通过 [sentence-transformers](https://www.SBERT.net) 框架来使用该模型，首先进行安装： ``` pip install -U sentence-transformers ``` 然后使用下面的代码来载入该模型并进行文本表征向量的提取： ```python from sentence_transformers import SentenceTransformer sentences = ["我的儿子！他猛然间喊道，我的儿子在哪儿？", "我的儿子呢！他突然喊道，我的儿子在哪里？"] model = SentenceTransformer('DMetaSoul/sbert-chinese-general-v1') embeddings = model.encode(sentences) print(embeddings) ``` ## 2. HuggingFace Transformers 如果不想使用 [sentence-transformers](https://www.SBERT.net) 的话，也可以通过 HuggingFace Transformers 来载入该模型并进行文本向量抽取： ```python from transformers import AutoTokenizer, AutoModel import torch #Mean Pooling - Take attention mask into account for correct averaging def mean_pooling(model_output, attention_mask): token_embeddings = model_output[0] #First element of model_output contains all token embeddings input_mask_expanded = attention_mask.unsqueeze(-1).expand(token_embeddings.size()).float() return torch.sum(token_embeddings * input_mask_expanded, 1) / torch.clamp(input_mask_expanded.sum(1), min=1e-9) # Sentences we want sentence embeddings for sentences = ["我的儿子！他猛然间喊道，我的儿子在哪儿？", "我的儿子呢！他突然喊道，我的儿子在哪里？"] # Load model from HuggingFace Hub tokenizer = AutoTokenizer.from_pretrained('DMetaSoul/sbert-chinese-general-v1') model = AutoModel.from_pretrained('DMetaSoul/sbert-chinese-general-v1') # Tokenize sentences encoded_input = tokenizer(sentences, padding=True, truncation=True, return_tensors='pt') # Compute token embeddings with torch.no_grad(): model_output = model(**encoded_input) # Perform pooling. In this case, mean pooling. sentence_embeddings = mean_pooling(model_output, encoded_input['attention_mask']) print("Sentence embeddings:") print(sentence_embeddings) ``` ## Evaluation 该模型在公开的几个语义匹配数据集上进行了评测，计算了向量相似度跟真实标签之间的相关性系数： | | **csts_dev** | **csts_test** | **afqmc** | **lcqmc** | **bqcorpus** | **pawsx** | **xiaobu** | | ------------ | ------------ | ------------- | --------- | --------- | ------------ | --------- | ---------- | | **spearman** | 84.54% | 82.17% | 23.80% | 65.94% | 45.52% | 11.52% | 48.51% | ## Citing & Authors E-mail: xiaowenbin@dmetasoul.com