--- language: - en license: apache-2.0 tags: - sentence-transformers - sentence-similarity - feature-extraction - generated_from_trainer - dataset_size:50000 - loss:CachedGISTEmbedLoss base_model: microsoft/mpnet-base widget: - source_sentence: what does the accounts receivable turnover measure? sentences: - The accounts receivable turnover ratio is an accounting measure used to quantify a company's effectiveness in collecting its receivables or money owed by clients. The ratio shows how well a company uses and manages the credit it extends to customers and how quickly that short-term debt is collected or is paid. - Capital budgeting, and investment appraisal, is the planning process used to determine whether an organization's long term investments such as new machinery, replacement of machinery, new plants, new products, and research development projects are worth the funding of cash through the firm's capitalization structure ( ... - The accounts receivable turnover ratio is an accounting measure used to quantify a company's effectiveness in collecting its receivables or money owed by clients. The ratio shows how well a company uses and manages the credit it extends to customers and how quickly that short-term debt is collected or is paid. - source_sentence: does gabapentin cause liver problems? sentences: - Gabapentin has no appreciable liver metabolism, yet, suspected cases of gabapentin-induced hepatotoxicity have been reported. Per literature review, two cases of possible gabapentin-induced liver injury have been reported. - Strongholds are a type of story mission which only unlocks after enough progression through the game. There are three Stronghold's during the first section of progression through The Division 2. You'll need to complete the first two and have reached level 30 before being able to unlock the final Stronghold. - The most-common side effects attributed to Gabapentin include mild sedation, ataxia, and occasional diarrhea. Sedation can be minimized by tapering from a smaller starting dose to the desired dose. When treating seizures, it is ideal to wean off the drug to reduce the risk of withdrawal seizures. - source_sentence: how long should you wait to give blood after eating? sentences: - Until the bleeding has stopped it is natural to taste blood or to see traces of blood in your saliva. You may stop using gauze after the flow stops – usually around 8 hours after surgery. - Before donation The first and most important rule—never donate blood on an empty stomach. “Eat a wholesome meal about 2-3 hours before donating to keep your blood sugar stable," says Dr Chaturvedi. The timing of the meal is important too. You need to allow the food to be digested properly before the blood is drawn. - While grid computing involves virtualizing computing resources to store massive amounts of data, whereas cloud computing is where an application doesn't access resources directly, rather it accesses them through a service over the internet. ... - source_sentence: what is the difference between chicken francese and chicken marsala? sentences: - Chicken is the species name, equivalent to our “human.” Rooster is an adult male, equivalent to “man.” Hen is an adult female, equivalent to “woman.” Cockerel is a juvenile male, equivalent to “boy/young man.” - What is 99 kg in pounds? - 99 kg is equal to 218.26 pounds. - The difference between the two is for Francese, the chicken breast is first dipped in flour, then into a beaten egg mixture, before being cooked. For piccata, the chicken is first dipped in egg and then in flour. Both are then simmered in a lemony butter sauce, but the piccata sauce includes capers.” - source_sentence: what energy is released when coal is burned? sentences: - When coal is burned, it reacts with the oxygen in the air. This chemical reaction converts the stored solar energy into thermal energy, which is released as heat. But it also produces carbon dioxide and methane. - When coal is burned it releases a number of airborne toxins and pollutants. They include mercury, lead, sulfur dioxide, nitrogen oxides, particulates, and various other heavy metals. - Squad Building Challenges allow you to exchange sets of players for coins, packs, and special items in FUT 20. Each of these challenges come with specific requirements, such as including players from certain teams. ... Live SBCs are time-limited challenges which often give out unique, high-rated versions of players. datasets: - tomaarsen/gooaq-hard-negatives pipeline_tag: sentence-similarity library_name: sentence-transformers metrics: - cosine_accuracy@1 - cosine_accuracy@3 - cosine_accuracy@5 - cosine_accuracy@10 - cosine_precision@1 - cosine_precision@3 - cosine_precision@5 - cosine_precision@10 - cosine_recall@1 - cosine_recall@3 - cosine_recall@5 - cosine_recall@10 - cosine_ndcg@10 - cosine_mrr@10 - cosine_map@100 co2_eq_emissions: emissions: 40.414352043491995 energy_consumed: 0.10397258579449552 source: codecarbon training_type: fine-tuning on_cloud: false cpu_model: 13th Gen Intel(R) Core(TM) i7-13700K ram_total_size: 31.777088165283203 hours_used: 0.297 hardware_used: 1 x NVIDIA GeForce RTX 3090 model-index: - name: MPNet base trained on Natural Questions pairs results: - task: type: information-retrieval name: Information Retrieval dataset: name: NanoClimateFEVER type: NanoClimateFEVER metrics: - type: cosine_accuracy@1 value: 0.22 name: Cosine Accuracy@1 - type: cosine_accuracy@3 value: 0.44 name: Cosine Accuracy@3 - type: cosine_accuracy@5 value: 0.5 name: Cosine Accuracy@5 - type: cosine_accuracy@10 value: 0.72 name: Cosine Accuracy@10 - type: cosine_precision@1 value: 0.22 name: Cosine Precision@1 - type: cosine_precision@3 value: 0.16666666666666663 name: Cosine Precision@3 - type: cosine_precision@5 value: 0.11599999999999999 name: Cosine Precision@5 - type: cosine_precision@10 value: 0.092 name: Cosine Precision@10 - type: cosine_recall@1 value: 0.09333333333333332 name: Cosine Recall@1 - type: cosine_recall@3 value: 0.195 name: Cosine Recall@3 - type: cosine_recall@5 value: 0.22666666666666666 name: Cosine Recall@5 - type: cosine_recall@10 value: 0.36733333333333335 name: Cosine Recall@10 - type: cosine_ndcg@10 value: 0.27399441682859144 name: Cosine Ndcg@10 - type: cosine_mrr@10 value: 0.36331746031746026 name: Cosine Mrr@10 - type: cosine_map@100 value: 0.20383864617106084 name: Cosine Map@100 - task: type: information-retrieval name: Information Retrieval dataset: name: NanoDBPedia type: NanoDBPedia metrics: - type: cosine_accuracy@1 value: 0.46 name: Cosine Accuracy@1 - type: cosine_accuracy@3 value: 0.64 name: Cosine Accuracy@3 - type: cosine_accuracy@5 value: 0.76 name: Cosine Accuracy@5 - type: cosine_accuracy@10 value: 0.84 name: Cosine Accuracy@10 - type: cosine_precision@1 value: 0.46 name: Cosine Precision@1 - type: cosine_precision@3 value: 0.4 name: Cosine Precision@3 - type: cosine_precision@5 value: 0.3760000000000001 name: Cosine Precision@5 - type: cosine_precision@10 value: 0.34 name: Cosine Precision@10 - type: cosine_recall@1 value: 0.03065300183409328 name: Cosine Recall@1 - type: cosine_recall@3 value: 0.07824513873584021 name: Cosine Recall@3 - type: cosine_recall@5 value: 0.12190077086725051 name: Cosine Recall@5 - type: cosine_recall@10 value: 0.21649668807903738 name: Cosine Recall@10 - type: cosine_ndcg@10 value: 0.38922276974007985 name: Cosine Ndcg@10 - type: cosine_mrr@10 value: 0.5768571428571428 name: Cosine Mrr@10 - type: cosine_map@100 value: 0.27830317958127815 name: Cosine Map@100 - task: type: information-retrieval name: Information Retrieval dataset: name: NanoFEVER type: NanoFEVER metrics: - type: cosine_accuracy@1 value: 0.38 name: Cosine Accuracy@1 - type: cosine_accuracy@3 value: 0.54 name: Cosine Accuracy@3 - type: cosine_accuracy@5 value: 0.58 name: Cosine Accuracy@5 - type: cosine_accuracy@10 value: 0.7 name: Cosine Accuracy@10 - type: cosine_precision@1 value: 0.38 name: Cosine Precision@1 - type: cosine_precision@3 value: 0.18 name: Cosine Precision@3 - type: cosine_precision@5 value: 0.12 name: Cosine Precision@5 - type: cosine_precision@10 value: 0.07200000000000001 name: Cosine Precision@10 - type: cosine_recall@1 value: 0.37 name: Cosine Recall@1 - type: cosine_recall@3 value: 0.52 name: Cosine Recall@3 - type: cosine_recall@5 value: 0.57 name: Cosine Recall@5 - type: cosine_recall@10 value: 0.68 name: Cosine Recall@10 - type: cosine_ndcg@10 value: 0.5226736410648857 name: Cosine Ndcg@10 - type: cosine_mrr@10 value: 0.4790238095238095 name: Cosine Mrr@10 - type: cosine_map@100 value: 0.4780826341570998 name: Cosine Map@100 - task: type: information-retrieval name: Information Retrieval dataset: name: NanoFiQA2018 type: NanoFiQA2018 metrics: - type: cosine_accuracy@1 value: 0.28 name: Cosine Accuracy@1 - type: cosine_accuracy@3 value: 0.5 name: Cosine Accuracy@3 - type: cosine_accuracy@5 value: 0.52 name: Cosine Accuracy@5 - type: cosine_accuracy@10 value: 0.58 name: Cosine Accuracy@10 - type: cosine_precision@1 value: 0.28 name: Cosine Precision@1 - type: cosine_precision@3 value: 0.22 name: Cosine Precision@3 - type: cosine_precision@5 value: 0.16 name: Cosine Precision@5 - type: cosine_precision@10 value: 0.09599999999999997 name: Cosine Precision@10 - type: cosine_recall@1 value: 0.1371904761904762 name: Cosine Recall@1 - type: cosine_recall@3 value: 0.32535714285714284 name: Cosine Recall@3 - type: cosine_recall@5 value: 0.3653571428571428 name: Cosine Recall@5 - type: cosine_recall@10 value: 0.42940476190476184 name: Cosine Recall@10 - type: cosine_ndcg@10 value: 0.34407947120145826 name: Cosine Ndcg@10 - type: cosine_mrr@10 value: 0.3881666666666667 name: Cosine Mrr@10 - type: cosine_map@100 value: 0.2878001762783797 name: Cosine Map@100 - task: type: information-retrieval name: Information Retrieval dataset: name: NanoHotpotQA type: NanoHotpotQA metrics: - type: cosine_accuracy@1 value: 0.34 name: Cosine Accuracy@1 - type: cosine_accuracy@3 value: 0.58 name: Cosine Accuracy@3 - type: cosine_accuracy@5 value: 0.64 name: Cosine Accuracy@5 - type: cosine_accuracy@10 value: 0.74 name: Cosine Accuracy@10 - type: cosine_precision@1 value: 0.34 name: Cosine Precision@1 - type: cosine_precision@3 value: 0.21333333333333332 name: Cosine Precision@3 - type: cosine_precision@5 value: 0.14800000000000002 name: Cosine Precision@5 - type: cosine_precision@10 value: 0.094 name: Cosine Precision@10 - type: cosine_recall@1 value: 0.17 name: Cosine Recall@1 - type: cosine_recall@3 value: 0.32 name: Cosine Recall@3 - type: cosine_recall@5 value: 0.37 name: Cosine Recall@5 - type: cosine_recall@10 value: 0.47 name: Cosine Recall@10 - type: cosine_ndcg@10 value: 0.3814617400581295 name: Cosine Ndcg@10 - type: cosine_mrr@10 value: 0.4695793650793651 name: Cosine Mrr@10 - type: cosine_map@100 value: 0.30635937490171045 name: Cosine Map@100 - task: type: information-retrieval name: Information Retrieval dataset: name: NanoMSMARCO type: NanoMSMARCO metrics: - type: cosine_accuracy@1 value: 0.12 name: Cosine Accuracy@1 - type: cosine_accuracy@3 value: 0.3 name: Cosine Accuracy@3 - type: cosine_accuracy@5 value: 0.56 name: Cosine Accuracy@5 - type: cosine_accuracy@10 value: 0.66 name: Cosine Accuracy@10 - type: cosine_precision@1 value: 0.12 name: Cosine Precision@1 - type: cosine_precision@3 value: 0.1 name: Cosine Precision@3 - type: cosine_precision@5 value: 0.11200000000000002 name: Cosine Precision@5 - type: cosine_precision@10 value: 0.066 name: Cosine Precision@10 - type: cosine_recall@1 value: 0.12 name: Cosine Recall@1 - type: cosine_recall@3 value: 0.3 name: Cosine Recall@3 - type: cosine_recall@5 value: 0.56 name: Cosine Recall@5 - type: cosine_recall@10 value: 0.66 name: Cosine Recall@10 - type: cosine_ndcg@10 value: 0.3660111210554949 name: Cosine Ndcg@10 - type: cosine_mrr@10 value: 0.27407936507936503 name: Cosine Mrr@10 - type: cosine_map@100 value: 0.2870484235732714 name: Cosine Map@100 - task: type: information-retrieval name: Information Retrieval dataset: name: NanoNFCorpus type: NanoNFCorpus metrics: - type: cosine_accuracy@1 value: 0.32 name: Cosine Accuracy@1 - type: cosine_accuracy@3 value: 0.4 name: Cosine Accuracy@3 - type: cosine_accuracy@5 value: 0.44 name: Cosine Accuracy@5 - type: cosine_accuracy@10 value: 0.5 name: Cosine Accuracy@10 - type: cosine_precision@1 value: 0.32 name: Cosine Precision@1 - type: cosine_precision@3 value: 0.21333333333333332 name: Cosine Precision@3 - type: cosine_precision@5 value: 0.196 name: Cosine Precision@5 - type: cosine_precision@10 value: 0.144 name: Cosine Precision@10 - type: cosine_recall@1 value: 0.012173283062756207 name: Cosine Recall@1 - type: cosine_recall@3 value: 0.02038195250132044 name: Cosine Recall@3 - type: cosine_recall@5 value: 0.028711609969173105 name: Cosine Recall@5 - type: cosine_recall@10 value: 0.04001132454412933 name: Cosine Recall@10 - type: cosine_ndcg@10 value: 0.17348600988460589 name: Cosine Ndcg@10 - type: cosine_mrr@10 value: 0.37288095238095237 name: Cosine Mrr@10 - type: cosine_map@100 value: 0.04905096401443591 name: Cosine Map@100 - task: type: information-retrieval name: Information Retrieval dataset: name: NanoNQ type: NanoNQ metrics: - type: cosine_accuracy@1 value: 0.16 name: Cosine Accuracy@1 - type: cosine_accuracy@3 value: 0.38 name: Cosine Accuracy@3 - type: cosine_accuracy@5 value: 0.46 name: Cosine Accuracy@5 - type: cosine_accuracy@10 value: 0.58 name: Cosine Accuracy@10 - type: cosine_precision@1 value: 0.16 name: Cosine Precision@1 - type: cosine_precision@3 value: 0.12666666666666665 name: Cosine Precision@3 - type: cosine_precision@5 value: 0.09200000000000001 name: Cosine Precision@5 - type: cosine_precision@10 value: 0.06000000000000001 name: Cosine Precision@10 - type: cosine_recall@1 value: 0.15 name: Cosine Recall@1 - type: cosine_recall@3 value: 0.36 name: Cosine Recall@3 - type: cosine_recall@5 value: 0.43 name: Cosine Recall@5 - type: cosine_recall@10 value: 0.54 name: Cosine Recall@10 - type: cosine_ndcg@10 value: 0.34266003995975836 name: Cosine Ndcg@10 - type: cosine_mrr@10 value: 0.2898015873015873 name: Cosine Mrr@10 - type: cosine_map@100 value: 0.29298840458552056 name: Cosine Map@100 - task: type: information-retrieval name: Information Retrieval dataset: name: NanoQuoraRetrieval type: NanoQuoraRetrieval metrics: - type: cosine_accuracy@1 value: 0.8 name: Cosine Accuracy@1 - type: cosine_accuracy@3 value: 0.9 name: Cosine Accuracy@3 - type: cosine_accuracy@5 value: 0.92 name: Cosine Accuracy@5 - type: cosine_accuracy@10 value: 0.96 name: Cosine Accuracy@10 - type: cosine_precision@1 value: 0.8 name: Cosine Precision@1 - type: cosine_precision@3 value: 0.3666666666666666 name: Cosine Precision@3 - type: cosine_precision@5 value: 0.244 name: Cosine Precision@5 - type: cosine_precision@10 value: 0.13399999999999998 name: Cosine Precision@10 - type: cosine_recall@1 value: 0.7006666666666667 name: Cosine Recall@1 - type: cosine_recall@3 value: 0.8553333333333333 name: Cosine Recall@3 - type: cosine_recall@5 value: 0.8993333333333333 name: Cosine Recall@5 - type: cosine_recall@10 value: 0.9566666666666666 name: Cosine Recall@10 - type: cosine_ndcg@10 value: 0.8760206475896655 name: Cosine Ndcg@10 - type: cosine_mrr@10 value: 0.8545238095238095 name: Cosine Mrr@10 - type: cosine_map@100 value: 0.8464432234432234 name: Cosine Map@100 - task: type: information-retrieval name: Information Retrieval dataset: name: NanoSCIDOCS type: NanoSCIDOCS metrics: - type: cosine_accuracy@1 value: 0.36 name: Cosine Accuracy@1 - type: cosine_accuracy@3 value: 0.48 name: Cosine Accuracy@3 - type: cosine_accuracy@5 value: 0.54 name: Cosine Accuracy@5 - type: cosine_accuracy@10 value: 0.68 name: Cosine Accuracy@10 - type: cosine_precision@1 value: 0.36 name: Cosine Precision@1 - type: cosine_precision@3 value: 0.25999999999999995 name: Cosine Precision@3 - type: cosine_precision@5 value: 0.21600000000000003 name: Cosine Precision@5 - type: cosine_precision@10 value: 0.15200000000000002 name: Cosine Precision@10 - type: cosine_recall@1 value: 0.07566666666666669 name: Cosine Recall@1 - type: cosine_recall@3 value: 0.16166666666666665 name: Cosine Recall@3 - type: cosine_recall@5 value: 0.22266666666666668 name: Cosine Recall@5 - type: cosine_recall@10 value: 0.31466666666666665 name: Cosine Recall@10 - type: cosine_ndcg@10 value: 0.2989790025477086 name: Cosine Ndcg@10 - type: cosine_mrr@10 value: 0.44196825396825395 name: Cosine Mrr@10 - type: cosine_map@100 value: 0.23644335943955802 name: Cosine Map@100 - task: type: information-retrieval name: Information Retrieval dataset: name: NanoArguAna type: NanoArguAna metrics: - type: cosine_accuracy@1 value: 0.18 name: Cosine Accuracy@1 - type: cosine_accuracy@3 value: 0.56 name: Cosine Accuracy@3 - type: cosine_accuracy@5 value: 0.64 name: Cosine Accuracy@5 - type: cosine_accuracy@10 value: 0.84 name: Cosine Accuracy@10 - type: cosine_precision@1 value: 0.18 name: Cosine Precision@1 - type: cosine_precision@3 value: 0.18666666666666668 name: Cosine Precision@3 - type: cosine_precision@5 value: 0.128 name: Cosine Precision@5 - type: cosine_precision@10 value: 0.08399999999999999 name: Cosine Precision@10 - type: cosine_recall@1 value: 0.18 name: Cosine Recall@1 - type: cosine_recall@3 value: 0.56 name: Cosine Recall@3 - type: cosine_recall@5 value: 0.64 name: Cosine Recall@5 - type: cosine_recall@10 value: 0.84 name: Cosine Recall@10 - type: cosine_ndcg@10 value: 0.5000949127836057 name: Cosine Ndcg@10 - type: cosine_mrr@10 value: 0.392515873015873 name: Cosine Mrr@10 - type: cosine_map@100 value: 0.4000896669795642 name: Cosine Map@100 - task: type: information-retrieval name: Information Retrieval dataset: name: NanoSciFact type: NanoSciFact metrics: - type: cosine_accuracy@1 value: 0.36 name: Cosine Accuracy@1 - type: cosine_accuracy@3 value: 0.46 name: Cosine Accuracy@3 - type: cosine_accuracy@5 value: 0.48 name: Cosine Accuracy@5 - type: cosine_accuracy@10 value: 0.62 name: Cosine Accuracy@10 - type: cosine_precision@1 value: 0.36 name: Cosine Precision@1 - type: cosine_precision@3 value: 0.16666666666666663 name: Cosine Precision@3 - type: cosine_precision@5 value: 0.10400000000000001 name: Cosine Precision@5 - type: cosine_precision@10 value: 0.068 name: Cosine Precision@10 - type: cosine_recall@1 value: 0.325 name: Cosine Recall@1 - type: cosine_recall@3 value: 0.44 name: Cosine Recall@3 - type: cosine_recall@5 value: 0.46 name: Cosine Recall@5 - type: cosine_recall@10 value: 0.605 name: Cosine Recall@10 - type: cosine_ndcg@10 value: 0.4619142281323308 name: Cosine Ndcg@10 - type: cosine_mrr@10 value: 0.43007936507936506 name: Cosine Mrr@10 - type: cosine_map@100 value: 0.42744465636932816 name: Cosine Map@100 - task: type: information-retrieval name: Information Retrieval dataset: name: NanoTouche2020 type: NanoTouche2020 metrics: - type: cosine_accuracy@1 value: 0.5306122448979592 name: Cosine Accuracy@1 - type: cosine_accuracy@3 value: 0.7142857142857143 name: Cosine Accuracy@3 - type: cosine_accuracy@5 value: 0.8571428571428571 name: Cosine Accuracy@5 - type: cosine_accuracy@10 value: 0.9591836734693877 name: Cosine Accuracy@10 - type: cosine_precision@1 value: 0.5306122448979592 name: Cosine Precision@1 - type: cosine_precision@3 value: 0.44217687074829926 name: Cosine Precision@3 - type: cosine_precision@5 value: 0.4122448979591837 name: Cosine Precision@5 - type: cosine_precision@10 value: 0.3530612244897959 name: Cosine Precision@10 - type: cosine_recall@1 value: 0.03881638827876476 name: Cosine Recall@1 - type: cosine_recall@3 value: 0.09899647775241191 name: Cosine Recall@3 - type: cosine_recall@5 value: 0.14503016807403868 name: Cosine Recall@5 - type: cosine_recall@10 value: 0.23921899286976872 name: Cosine Recall@10 - type: cosine_ndcg@10 value: 0.40436873020254516 name: Cosine Ndcg@10 - type: cosine_mrr@10 value: 0.6520651117589893 name: Cosine Mrr@10 - type: cosine_map@100 value: 0.3151658515102787 name: Cosine Map@100 - task: type: nano-beir name: Nano BEIR dataset: name: NanoBEIR mean type: NanoBEIR_mean metrics: - type: cosine_accuracy@1 value: 0.3469701726844584 name: Cosine Accuracy@1 - type: cosine_accuracy@3 value: 0.5303296703296704 name: Cosine Accuracy@3 - type: cosine_accuracy@5 value: 0.6074725274725276 name: Cosine Accuracy@5 - type: cosine_accuracy@10 value: 0.7214756671899529 name: Cosine Accuracy@10 - type: cosine_precision@1 value: 0.3469701726844584 name: Cosine Precision@1 - type: cosine_precision@3 value: 0.23401360544217684 name: Cosine Precision@3 - type: cosine_precision@5 value: 0.18648037676609108 name: Cosine Precision@5 - type: cosine_precision@10 value: 0.13500470957613817 name: Cosine Precision@10 - type: cosine_recall@1 value: 0.18488460123328906 name: Cosine Recall@1 - type: cosine_recall@3 value: 0.32576774706513195 name: Cosine Recall@3 - type: cosine_recall@5 value: 0.3876666429564824 name: Cosine Recall@5 - type: cosine_recall@10 value: 0.48913834108187415 name: Cosine Recall@10 - type: cosine_ndcg@10 value: 0.4103820562345277 name: Cosine Ndcg@10 - type: cosine_mrr@10 value: 0.4603737509655877 name: Cosine Mrr@10 - type: cosine_map@100 value: 0.3391583508465161 name: Cosine Map@100 --- # MPNet base trained on Natural Questions pairs This is a [sentence-transformers](https://www.SBERT.net) model finetuned from [microsoft/mpnet-base](https://huggingface.co/microsoft/mpnet-base) on the [gooaq-hard-negatives](https://huggingface.co/datasets/tomaarsen/gooaq-hard-negatives) dataset. It maps sentences & paragraphs to a 768-dimensional dense vector space and can be used for semantic textual similarity, semantic search, paraphrase mining, text classification, clustering, and more. ## Model Details ### Model Description - **Model Type:** Sentence Transformer - **Base model:** [microsoft/mpnet-base](https://huggingface.co/microsoft/mpnet-base) - **Maximum Sequence Length:** 512 tokens - **Output Dimensionality:** 768 dimensions - **Similarity Function:** Cosine Similarity - **Training Dataset:** - [gooaq-hard-negatives](https://huggingface.co/datasets/tomaarsen/gooaq-hard-negatives) - **Language:** en - **License:** apache-2.0 ### Model Sources - **Documentation:** [Sentence Transformers Documentation](https://sbert.net) - **Repository:** [Sentence Transformers on GitHub](https://github.com/UKPLab/sentence-transformers) - **Hugging Face:** [Sentence Transformers on Hugging Face](https://huggingface.co/models?library=sentence-transformers) ### Full Model Architecture ``` SentenceTransformer( (0): Transformer({'max_seq_length': 512, 'do_lower_case': False}) with Transformer model: MPNetModel (1): Pooling({'word_embedding_dimension': 768, 'pooling_mode_cls_token': False, 'pooling_mode_mean_tokens': True, 'pooling_mode_max_tokens': False, 'pooling_mode_mean_sqrt_len_tokens': False, 'pooling_mode_weightedmean_tokens': False, 'pooling_mode_lasttoken': False, 'include_prompt': True}) ) ``` ## Usage ### Direct Usage (Sentence Transformers) First install the Sentence Transformers library: ```bash pip install -U sentence-transformers ``` Then you can load this model and run inference. ```python from sentence_transformers import SentenceTransformer # Download from the 🤗 Hub model = SentenceTransformer("tomaarsen/mpnet-base-nq-cgist-triplet-gt") # Run inference sentences = [ 'what energy is released when coal is burned?', 'When coal is burned, it reacts with the oxygen in the air. This chemical reaction converts the stored solar energy into thermal energy, which is released as heat. But it also produces carbon dioxide and methane.', 'When coal is burned it releases a number of airborne toxins and pollutants. They include mercury, lead, sulfur dioxide, nitrogen oxides, particulates, and various other heavy metals.', ] embeddings = model.encode(sentences) print(embeddings.shape) # [3, 768] # Get the similarity scores for the embeddings similarities = model.similarity(embeddings, embeddings) print(similarities.shape) # [3, 3] ``` ## Evaluation ### Metrics #### Information Retrieval * Datasets: `NanoClimateFEVER`, `NanoDBPedia`, `NanoFEVER`, `NanoFiQA2018`, `NanoHotpotQA`, `NanoMSMARCO`, `NanoNFCorpus`, `NanoNQ`, `NanoQuoraRetrieval`, `NanoSCIDOCS`, `NanoArguAna`, `NanoSciFact` and `NanoTouche2020` * Evaluated with [InformationRetrievalEvaluator](https://sbert.net/docs/package_reference/sentence_transformer/evaluation.html#sentence_transformers.evaluation.InformationRetrievalEvaluator) | Metric | NanoClimateFEVER | NanoDBPedia | NanoFEVER | NanoFiQA2018 | NanoHotpotQA | NanoMSMARCO | NanoNFCorpus | NanoNQ | NanoQuoraRetrieval | NanoSCIDOCS | NanoArguAna | NanoSciFact | NanoTouche2020 | |:--------------------|:-----------------|:------------|:-----------|:-------------|:-------------|:------------|:-------------|:-----------|:-------------------|:------------|:------------|:------------|:---------------| | cosine_accuracy@1 | 0.22 | 0.46 | 0.38 | 0.28 | 0.34 | 0.12 | 0.32 | 0.16 | 0.8 | 0.36 | 0.18 | 0.36 | 0.5306 | | cosine_accuracy@3 | 0.44 | 0.64 | 0.54 | 0.5 | 0.58 | 0.3 | 0.4 | 0.38 | 0.9 | 0.48 | 0.56 | 0.46 | 0.7143 | | cosine_accuracy@5 | 0.5 | 0.76 | 0.58 | 0.52 | 0.64 | 0.56 | 0.44 | 0.46 | 0.92 | 0.54 | 0.64 | 0.48 | 0.8571 | | cosine_accuracy@10 | 0.72 | 0.84 | 0.7 | 0.58 | 0.74 | 0.66 | 0.5 | 0.58 | 0.96 | 0.68 | 0.84 | 0.62 | 0.9592 | | cosine_precision@1 | 0.22 | 0.46 | 0.38 | 0.28 | 0.34 | 0.12 | 0.32 | 0.16 | 0.8 | 0.36 | 0.18 | 0.36 | 0.5306 | | cosine_precision@3 | 0.1667 | 0.4 | 0.18 | 0.22 | 0.2133 | 0.1 | 0.2133 | 0.1267 | 0.3667 | 0.26 | 0.1867 | 0.1667 | 0.4422 | | cosine_precision@5 | 0.116 | 0.376 | 0.12 | 0.16 | 0.148 | 0.112 | 0.196 | 0.092 | 0.244 | 0.216 | 0.128 | 0.104 | 0.4122 | | cosine_precision@10 | 0.092 | 0.34 | 0.072 | 0.096 | 0.094 | 0.066 | 0.144 | 0.06 | 0.134 | 0.152 | 0.084 | 0.068 | 0.3531 | | cosine_recall@1 | 0.0933 | 0.0307 | 0.37 | 0.1372 | 0.17 | 0.12 | 0.0122 | 0.15 | 0.7007 | 0.0757 | 0.18 | 0.325 | 0.0388 | | cosine_recall@3 | 0.195 | 0.0782 | 0.52 | 0.3254 | 0.32 | 0.3 | 0.0204 | 0.36 | 0.8553 | 0.1617 | 0.56 | 0.44 | 0.099 | | cosine_recall@5 | 0.2267 | 0.1219 | 0.57 | 0.3654 | 0.37 | 0.56 | 0.0287 | 0.43 | 0.8993 | 0.2227 | 0.64 | 0.46 | 0.145 | | cosine_recall@10 | 0.3673 | 0.2165 | 0.68 | 0.4294 | 0.47 | 0.66 | 0.04 | 0.54 | 0.9567 | 0.3147 | 0.84 | 0.605 | 0.2392 | | **cosine_ndcg@10** | **0.274** | **0.3892** | **0.5227** | **0.3441** | **0.3815** | **0.366** | **0.1735** | **0.3427** | **0.876** | **0.299** | **0.5001** | **0.4619** | **0.4044** | | cosine_mrr@10 | 0.3633 | 0.5769 | 0.479 | 0.3882 | 0.4696 | 0.2741 | 0.3729 | 0.2898 | 0.8545 | 0.442 | 0.3925 | 0.4301 | 0.6521 | | cosine_map@100 | 0.2038 | 0.2783 | 0.4781 | 0.2878 | 0.3064 | 0.287 | 0.0491 | 0.293 | 0.8464 | 0.2364 | 0.4001 | 0.4274 | 0.3152 | #### Nano BEIR * Dataset: `NanoBEIR_mean` * Evaluated with [NanoBEIREvaluator](https://sbert.net/docs/package_reference/sentence_transformer/evaluation.html#sentence_transformers.evaluation.NanoBEIREvaluator) | Metric | Value | |:--------------------|:-----------| | cosine_accuracy@1 | 0.347 | | cosine_accuracy@3 | 0.5303 | | cosine_accuracy@5 | 0.6075 | | cosine_accuracy@10 | 0.7215 | | cosine_precision@1 | 0.347 | | cosine_precision@3 | 0.234 | | cosine_precision@5 | 0.1865 | | cosine_precision@10 | 0.135 | | cosine_recall@1 | 0.1849 | | cosine_recall@3 | 0.3258 | | cosine_recall@5 | 0.3877 | | cosine_recall@10 | 0.4891 | | **cosine_ndcg@10** | **0.4104** | | cosine_mrr@10 | 0.4604 | | cosine_map@100 | 0.3392 | ## Training Details ### Training Dataset #### gooaq-hard-negatives * Dataset: [gooaq-hard-negatives](https://huggingface.co/datasets/tomaarsen/gooaq-hard-negatives) at [87594a1](https://huggingface.co/datasets/tomaarsen/gooaq-hard-negatives/tree/87594a1e6c58e88b5843afa9da3a97ffd75d01c2) * Size: 50,000 training samples * Columns: question, answer, and negative * Approximate statistics based on the first 1000 samples: | | question | answer | negative | |:--------|:----------------------------------------------------------------------------------|:------------------------------------------------------------------------------------|:------------------------------------------------------------------------------------| | type | string | string | string | | details | | | | * Samples: | question | answer | negative | |:--------------------------------------------------------------------------------------|:--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|:-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------| | what is the difference between calories from fat and total fat? | Fat has more than twice as many calories per gram as carbohydrates and proteins. A gram of fat has about 9 calories, while a gram of carbohydrate or protein has about 4 calories. In other words, you could eat twice as much carbohydrates or proteins as fat for the same amount of calories. | Fat has more than twice as many calories per gram as carbohydrates and proteins. A gram of fat has about 9 calories, while a gram of carbohydrate or protein has about 4 calories. In other words, you could eat twice as much carbohydrates or proteins as fat for the same amount of calories. | | what is the difference between return transcript and account transcript? | A tax return transcript usually meets the needs of lending institutions offering mortgages and student loans. ... Tax Account Transcript - shows basic data such as return type, marital status, adjusted gross income, taxable income and all payment types. It also shows changes made after you filed your original return. | Trial balance is not a financial statement whereas a balance sheet is a financial statement. Trial balance is solely used for internal purposes whereas a balance sheet is used for purposes other than internal i.e. external. In a trial balance, each and every account is divided into debit (dr.) and credit (cr.) | | how long does my dog need to fast before sedation? | Now, guidelines are aimed towards 6-8 hours before surgery. This pre-op fasting time is much more beneficial for your pets because you have enough food in there to neutralize the stomach acid, preventing it from coming up the esophagus that causes regurgitation under anesthetic. | Try not to let your pooch rapidly wolf down his/her food! Do not let the dog play or exercise (e.g. go for a walk) for at least two hours after having a meal. Ensure continuous fresh water is available to avoid your pet gulping down a large amount after eating. | * Loss: [CachedGISTEmbedLoss](https://sbert.net/docs/package_reference/sentence_transformer/losses.html#cachedgistembedloss) with these parameters: ```json {'guide': SentenceTransformer( (0): Transformer({'max_seq_length': 256, 'do_lower_case': False}) with Transformer model: BertModel (1): Pooling({'word_embedding_dimension': 384, 'pooling_mode_cls_token': False, 'pooling_mode_mean_tokens': True, 'pooling_mode_max_tokens': False, 'pooling_mode_mean_sqrt_len_tokens': False, 'pooling_mode_weightedmean_tokens': False, 'pooling_mode_lasttoken': False, 'include_prompt': True}) (2): Normalize() ), 'temperature': 0.01} ``` ### Evaluation Dataset #### gooaq-hard-negatives * Dataset: [gooaq-hard-negatives](https://huggingface.co/datasets/tomaarsen/gooaq-hard-negatives) at [87594a1](https://huggingface.co/datasets/tomaarsen/gooaq-hard-negatives/tree/87594a1e6c58e88b5843afa9da3a97ffd75d01c2) * Size: 10,048,700 evaluation samples * Columns: question, answer, and negative * Approximate statistics based on the first 1000 samples: | | question | answer | negative | |:--------|:----------------------------------------------------------------------------------|:------------------------------------------------------------------------------------|:------------------------------------------------------------------------------------| | type | string | string | string | | details | | | | * Samples: | question | answer | negative | |:--------------------------------------------------------------------|:----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|:--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------| | how is height width and length written? | The Graphics' industry standard is width by height (width x height). Meaning that when you write your measurements, you write them from your point of view, beginning with the width. | The Graphics' industry standard is width by height (width x height). Meaning that when you write your measurements, you write them from your point of view, beginning with the width. That's important. | | what is the difference between pork shoulder and loin? | All the recipes I've found for pulled pork recommends a shoulder/butt. Shoulders take longer to cook than a loin, because they're tougher. Loins are lean, while shoulders have marbled fat inside. | They are extracted from the loin, which runs from the hip to the shoulder, and it has a small strip of meat called the tenderloin. Unlike other pork, this pork chop is cut from four major sections, which are the shoulder, also known as the blade chops, ribs chops, loin chops, and the last, which is the sirloin chops. | | is the yin yang symbol religious? | The ubiquitous yin-yang symbol holds its roots in Taoism/Daoism, a Chinese religion and philosophy. The yin, the dark swirl, is associated with shadows, femininity, and the trough of a wave; the yang, the light swirl, represents brightness, passion and growth. | Yin energy is in the calm colors around you, in the soft music, in the soothing sound of a water fountain, or the relaxing images of water. Yang (active energy) is the feng shui energy expressed in strong, vibrant sounds and colors, bright lights, upward moving energy, tall plants, etc. | * Loss: [CachedGISTEmbedLoss](https://sbert.net/docs/package_reference/sentence_transformer/losses.html#cachedgistembedloss) with these parameters: ```json {'guide': SentenceTransformer( (0): Transformer({'max_seq_length': 256, 'do_lower_case': False}) with Transformer model: BertModel (1): Pooling({'word_embedding_dimension': 384, 'pooling_mode_cls_token': False, 'pooling_mode_mean_tokens': True, 'pooling_mode_max_tokens': False, 'pooling_mode_mean_sqrt_len_tokens': False, 'pooling_mode_weightedmean_tokens': False, 'pooling_mode_lasttoken': False, 'include_prompt': True}) (2): Normalize() ), 'temperature': 0.01} ``` ### Training Hyperparameters #### Non-Default Hyperparameters - `eval_strategy`: steps - `per_device_train_batch_size`: 2048 - `per_device_eval_batch_size`: 2048 - `learning_rate`: 2e-05 - `num_train_epochs`: 1 - `warmup_ratio`: 0.1 - `seed`: 12 - `bf16`: True #### All Hyperparameters
Click to expand - `overwrite_output_dir`: False - `do_predict`: False - `eval_strategy`: steps - `prediction_loss_only`: True - `per_device_train_batch_size`: 2048 - `per_device_eval_batch_size`: 2048 - `per_gpu_train_batch_size`: None - `per_gpu_eval_batch_size`: None - `gradient_accumulation_steps`: 1 - `eval_accumulation_steps`: None - `torch_empty_cache_steps`: None - `learning_rate`: 2e-05 - `weight_decay`: 0.0 - `adam_beta1`: 0.9 - `adam_beta2`: 0.999 - `adam_epsilon`: 1e-08 - `max_grad_norm`: 1.0 - `num_train_epochs`: 1 - `max_steps`: -1 - `lr_scheduler_type`: linear - `lr_scheduler_kwargs`: {} - `warmup_ratio`: 0.1 - `warmup_steps`: 0 - `log_level`: passive - `log_level_replica`: warning - `log_on_each_node`: True - `logging_nan_inf_filter`: True - `save_safetensors`: True - `save_on_each_node`: False - `save_only_model`: False - `restore_callback_states_from_checkpoint`: False - `no_cuda`: False - `use_cpu`: False - `use_mps_device`: False - `seed`: 12 - `data_seed`: None - `jit_mode_eval`: False - `use_ipex`: False - `bf16`: True - `fp16`: False - `fp16_opt_level`: O1 - `half_precision_backend`: auto - `bf16_full_eval`: False - `fp16_full_eval`: False - `tf32`: None - `local_rank`: 0 - `ddp_backend`: None - `tpu_num_cores`: None - `tpu_metrics_debug`: False - `debug`: [] - `dataloader_drop_last`: False - `dataloader_num_workers`: 0 - `dataloader_prefetch_factor`: None - `past_index`: -1 - `disable_tqdm`: False - `remove_unused_columns`: True - `label_names`: None - `load_best_model_at_end`: False - `ignore_data_skip`: False - `fsdp`: [] - `fsdp_min_num_params`: 0 - `fsdp_config`: {'min_num_params': 0, 'xla': False, 'xla_fsdp_v2': False, 'xla_fsdp_grad_ckpt': False} - `fsdp_transformer_layer_cls_to_wrap`: None - `accelerator_config`: {'split_batches': False, 'dispatch_batches': None, 'even_batches': True, 'use_seedable_sampler': True, 'non_blocking': False, 'gradient_accumulation_kwargs': None} - `deepspeed`: None - `label_smoothing_factor`: 0.0 - `optim`: adamw_torch - `optim_args`: None - `adafactor`: False - `group_by_length`: False - `length_column_name`: length - `ddp_find_unused_parameters`: None - `ddp_bucket_cap_mb`: None - `ddp_broadcast_buffers`: False - `dataloader_pin_memory`: True - `dataloader_persistent_workers`: False - `skip_memory_metrics`: True - `use_legacy_prediction_loop`: False - `push_to_hub`: False - `resume_from_checkpoint`: None - `hub_model_id`: None - `hub_strategy`: every_save - `hub_private_repo`: False - `hub_always_push`: False - `gradient_checkpointing`: False - `gradient_checkpointing_kwargs`: None - `include_inputs_for_metrics`: False - `include_for_metrics`: [] - `eval_do_concat_batches`: True - `fp16_backend`: auto - `push_to_hub_model_id`: None - `push_to_hub_organization`: None - `mp_parameters`: - `auto_find_batch_size`: False - `full_determinism`: False - `torchdynamo`: None - `ray_scope`: last - `ddp_timeout`: 1800 - `torch_compile`: False - `torch_compile_backend`: None - `torch_compile_mode`: None - `dispatch_batches`: None - `split_batches`: None - `include_tokens_per_second`: False - `include_num_input_tokens_seen`: False - `neftune_noise_alpha`: None - `optim_target_modules`: None - `batch_eval_metrics`: False - `eval_on_start`: False - `use_liger_kernel`: False - `eval_use_gather_object`: False - `average_tokens_across_devices`: False - `prompts`: None - `batch_sampler`: batch_sampler - `multi_dataset_batch_sampler`: proportional
### Training Logs | Epoch | Step | Training Loss | Validation Loss | NanoClimateFEVER_cosine_ndcg@10 | NanoDBPedia_cosine_ndcg@10 | NanoFEVER_cosine_ndcg@10 | NanoFiQA2018_cosine_ndcg@10 | NanoHotpotQA_cosine_ndcg@10 | NanoMSMARCO_cosine_ndcg@10 | NanoNFCorpus_cosine_ndcg@10 | NanoNQ_cosine_ndcg@10 | NanoQuoraRetrieval_cosine_ndcg@10 | NanoSCIDOCS_cosine_ndcg@10 | NanoArguAna_cosine_ndcg@10 | NanoSciFact_cosine_ndcg@10 | NanoTouche2020_cosine_ndcg@10 | NanoBEIR_mean_cosine_ndcg@10 | |:-----:|:----:|:-------------:|:---------------:|:-------------------------------:|:--------------------------:|:------------------------:|:---------------------------:|:---------------------------:|:--------------------------:|:---------------------------:|:---------------------:|:---------------------------------:|:--------------------------:|:--------------------------:|:--------------------------:|:-----------------------------:|:----------------------------:| | 0.04 | 1 | 11.5142 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | | 0.2 | 5 | 9.438 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | | 0.4 | 10 | 5.5516 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | | 0.6 | 15 | 3.7045 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | | 0.8 | 20 | 2.7618 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | | 1.0 | 25 | 2.1657 | 1.3177 | 0.2740 | 0.3892 | 0.5227 | 0.3441 | 0.3815 | 0.3660 | 0.1735 | 0.3427 | 0.8760 | 0.2990 | 0.5001 | 0.4619 | 0.4044 | 0.4104 | ### Environmental Impact Carbon emissions were measured using [CodeCarbon](https://github.com/mlco2/codecarbon). - **Energy Consumed**: 0.104 kWh - **Carbon Emitted**: 0.040 kg of CO2 - **Hours Used**: 0.297 hours ### Training Hardware - **On Cloud**: No - **GPU Model**: 1 x NVIDIA GeForce RTX 3090 - **CPU Model**: 13th Gen Intel(R) Core(TM) i7-13700K - **RAM Size**: 31.78 GB ### Framework Versions - Python: 3.11.6 - Sentence Transformers: 3.4.0.dev0 - Transformers: 4.46.2 - PyTorch: 2.5.0+cu121 - Accelerate: 0.35.0.dev0 - Datasets: 2.20.0 - Tokenizers: 0.20.3 ## Citation ### BibTeX #### Sentence Transformers ```bibtex @inproceedings{reimers-2019-sentence-bert, title = "Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks", author = "Reimers, Nils and Gurevych, Iryna", booktitle = "Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing", month = "11", year = "2019", publisher = "Association for Computational Linguistics", url = "https://arxiv.org/abs/1908.10084", } ```