kperkins411/mpnet-base-all-nli-triplet

SentenceTransformer

This is a sentence-transformers model trained. It maps sentences & paragraphs to a 384-dimensional dense vector space and can be used for semantic textual similarity, semantic search, paraphrase mining, text classification, clustering, and more.

Model Details

Model Description

• Model Type: Sentence Transformer
• Maximum Sequence Length: 512 tokens
• Output Dimensionality: 384 tokens
• Similarity Function: Cosine Similarity

Model Sources

• Documentation: Sentence Transformers Documentation
• Repository: Sentence Transformers on GitHub
• Hugging Face: Sentence Transformers on Hugging Face

Full Model Architecture

SentenceTransformer(
  (0): Transformer({'max_seq_length': 512, 'do_lower_case': False}) with Transformer model: BertModel 
  (1): Pooling({'word_embedding_dimension': 384, 'pooling_mode_cls_token': False, 'pooling_mode_mean_tokens': True, 'pooling_mode_max_tokens': False, 'pooling_mode_mean_sqrt_len_tokens': False, 'pooling_mode_weightedmean_tokens': False, 'pooling_mode_lasttoken': False, 'include_prompt': True})
)

Usage

Direct Usage (Sentence Transformers)

First install the Sentence Transformers library:

pip install -U sentence-transformers

Then you can load this model and run inference.

from sentence_transformers import SentenceTransformer

# Download from the 🤗 Hub
model = SentenceTransformer("kperkins411/mpnet-base-all-nli-triplet")
# Run inference
sentences = [
    '11.8 no agency. except as expressly stated otherwise, nothing in this agreement shall create an agency, partnership or joint venture of any kind between the parties.',
    '9.2 relationship of parties. the parties are independent contractors ------------------------- under this agreement and no other relationship is intended, including a partnership, franchise, joint venture, agency, employer/employee, fiduciary, master/servant relationship, or other special relationship. neither party shall act in a manner which expresses or implies a relationship other than that of independent contractor, nor bind the other party.',
    'any agency relationship implied?',
]
embeddings = model.encode(sentences)
print(embeddings.shape)
# [3, 384]

# Get the similarity scores for the embeddings
similarities = model.similarity(embeddings, embeddings)
print(similarities.shape)
# [3, 3]

Evaluation

Metrics

Triplet

• Dataset: all-nli-dev
• Evaluated with TripletEvaluator

Metric	Value
cosine_accuracy	0.4892
dot_accuracy	0.5158
manhattan_accuracy	0.4823
euclidean_accuracy	0.4808
max_accuracy	0.4892

Triplet

• Dataset: all-nli-test
• Evaluated with TripletEvaluator

Metric	Value
cosine_accuracy	0.4892
dot_accuracy	0.5158
manhattan_accuracy	0.4823
euclidean_accuracy	0.4808
max_accuracy	0.4892

Training Details

Training Dataset

Unnamed Dataset

• Size: 35,258 training samples
• Columns: positive, negative, and anchor
• Approximate statistics based on the first 1000 samples:

  	positive	negative	anchor
  type	string	string	string
  details	min: 6 tokens mean: 101.64 tokens max: 512 tokens	min: 6 tokens mean: 80.74 tokens max: 512 tokens	min: 5 tokens mean: 17.19 tokens max: 167 tokens

• Samples:

  positive	negative	anchor
  information we collect from other sources we may also receive information from other sources and combine that with information we collect through our services. for example: if you choose to link, create, or log in to your uber account with a payment provider (e.g., google wallet) or social media service (e.g., facebook), or if you engage with a separate app or website that uses our api (or whose api we use), we may receive information about you or your connections from that site or app.	c. the obligations specified in this article shall not apply to information for which the receiving party can reasonably demonstrate that such information: iii. becomes known to the receiving party through disclosure by sources other than the disclosing party, having a right to disclose such information,	what safeguards are in place to protect the information obtained from third-party sources?
  each of the suppliers warrants that the products shall comply with the specifications and documentation agreed by the relevant supplier and the company in writing that is applicable to such products for the warranty period.	3.2 manufacturing standards the manufacturer covenants that it is and will remain for the term of this agreement in compliance with all international standards in production and manufacturing.	is there a guarantee from the manufacturers regarding the conformity of the items to the mutually approved written standards for a certain duration?
  skype hereby grants to online bvi and the company a limited, non-exclusive, non-sublicensable (except as set forth herein), non-transferable, non-assignable (except as provided in section 14.4), royalty-free (but subject to the provisions of section 5), license during the term to use, market, provide access to, promote, reproduce and display the skype intellectual property solely (i) as incorporated in the company-skype branded application and/or the company-skype toolbar, and (ii) as incorporated in, for the development of, and for transmission pursuant to this agreement of, the company-skype branded content and the company-skype branded web site, in each case for the sole purposes (unless otherwise mutually agreed by the parties) of promoting and distributing, pursuant to this agreement, the company-skype branded application, the company-skype toolbar, the company-skype branded content and the company-skype branded web site in the territory; (a) provided, that it is understood that the company-skype branded customers will have the right under the eula to use the company- skype branded application and the company-skype toolbar and will have the right to access the company-skype branded content, the company-skype branded web site and the online bvi web site through the internet and to otherwise receive support from the company anywhere in the world, and that the company shall be permitted to provide access to and reproduce and display the skype intellectual property through the internet anywhere in the world, and (b) provided further, that online bvi and the company shall ensure that no company-skype branded customer (or potential company-skype branded customer) shall be permitted to access, using the company-skype branded application or the company-skype toolbar or through the company-skype branded web site, any skype premium features requiring payment by the company-skype branded customer (or potential company-skype branded customer), including, but not limited to, skypein, skypeout, or skype plus, unless such company-skype branded customer (or potential company-skype branded customer) uses the payment methods made available by the company pursuant to section 2.5 for the purchase of such premium features.	planetcad hereby grants to dassault systemes a fully-paid, non-exclusive, worldwide, revocable limited license to the server software and infrastructure for the sole purpose of (i) hosting the co-branded service and (ii) fulfilling itsobligations under this agreement.	what type of authorization has the video conferencing service provided to the british virgin islands-based entity and its associated organization regarding their intellectual property, with respect to the customized software and web platform, including the conditions for customer access to enhanced functionalities that incur additional charges?

• Loss: TripletLoss with these parameters:

  {
      "distance_metric": "TripletDistanceMetric.EUCLIDEAN",
      "triplet_margin": 5
  }
  

Evaluation Dataset

Unnamed Dataset

• Size: 2,633 evaluation samples
• Columns: negative, positive, and anchor
• Approximate statistics based on the first 1000 samples:

  	negative	positive	anchor
  type	string	string	string
  details	min: 6 tokens mean: 96.02 tokens max: 512 tokens	min: 6 tokens mean: 107.62 tokens max: 512 tokens	min: 5 tokens mean: 18.05 tokens max: 512 tokens

• Samples:

  negative	positive	anchor
  9.1. confidentiality obligations. except as permitted elsewhere under this agreement, each party agrees to take reasonable steps (as defined below) (a) to receive and maintain the confidential information of the other party in confidence and (b) not to disclose such confidential information to any third parties, provided, the receiving party may disclose such confidential information to its employees, representatives and agents who have a need to know such information for purposes of carrying out the terms of this agreement. neither party hereto shall use all or any part of the confidential information of the other party for any purpose other than to perform its obligations under this agreement. the parties will take reasonable steps (as defined below) to ensure that their employees, representatives and agents comply with this provision. as used herein, "reasonable steps" means at least the same degree of care that the receiving party uses to protect its own confidential information, and, in any event, no less than reasonable care.	8.1. each party acknowledges the other's confidential information is unique and valuable and was developed or otherwise acquired by the other at great expense, and that any unauthorized disclosure or use of the other's confidential information would cause the other irreparable injury loss for which damages would be an inadequate remedy. the party agrees to hold such confidential information in strictest confidence, to use all efforts reasonable under the circumstances to maintain the secrecy thereof, and not to make use thereof other than in accordance with this agreement, and not to release or disclose confidential information to any third party without the other's prior written consent, subject to a court order, or subject to a sublicense consistent with this agreement and requiring the sublicensee to maintain the confidential information in strictest confidence, to use all efforts reasonable under the circumstances to maintain the secrecy thereof, not to make use thereof other than in accordance with the sublicense agreement, and not to release or disclose confidential information to any third party without the other's prior written consent. 6 source: legacy education alliance, inc., 10-k, 3/30/2020 certain identified information has been excluded from this exhibit because it is both (i) not material and (ii) would be competitively harmful if publicly disclosed.	what efforts are deemed 'reasonable under the circumstances' to protect confidential information?
  14.9 no assignment. neither party may assign this agreement without the other party's prior written consent. notwithstanding the foregoing, either party may assign this agreement without the other party's prior written consent in the event of a merger, acquisition, reorganization, change in control, or sale of substantially all of the assets or business of such assigning party. any assignment in conflict with this provision shall be void.	2.2.1 this agreement does not limit our right, or the right of the entities, to own, license or operate any other business of any nature, whether in the lodging or hospitality industry or not, and whether under the brand, a competing brand, or otherwise. we and the entities have the right to engage in any other businesses, even if they compete with the hotel, the system, or the brand, and whether we or the entities start those businesses, or purchase, merge with, acquire, are acquired by, come under common ownership with, or associate with, such other businesses.	are there any restrictions on mergers or acquisitions involving other businesses?
  1.2 pnc or its tpms will place specific orders for ingredients from supplier by issuing a purchase order that specifies, at minimum, the item, quantities, price, delivery dates, and delivery and payment terms (each a "purchase order").	1.5 supplier represents and warrants that at the time and date of delivery, the ingredients will comply with all specifications ("specifications"), a copy of which will be attached to the relevant master purchase commitment or purchase order accordingly. a specification may be updated from time to time by pnc in its sole discretion, provided pnc provides supplier with reasonable prior notice on any updates ("change notification"). within [] from receipt of the change notification, supplier will either: (1) accept the specification change at the current price and terms; or (2) submit to pnc a proposal ("proposal") setting forth the conditions of acceptance that may include a change in price and/or other terms, including documentation to support same. within [] the parties will discuss the proposal in good faith and exercise their best efforts to agree on the appropriate adjustment if any. pnc will not issue any purchase orders, nor be required to issue any purchase orders to supplier until pnc and supplier have agreed on required ingredient specifications and any associated price and/or term adjustment. in the event the parties fail to agree on required ingredient specifications or price and/or term adjustments despite their best good faith efforts, neither party will have any further obligation with regard to purchase or supply of those ingredients under any master purchase commitments except that pnc shall take and pay for [***] of ingredient inventory manufactured according to the then-current specification.	are purchase orders mandatory before agreeing on updated ingredient specifications and adjustments?

• Loss: TripletLoss with these parameters:

  {
      "distance_metric": "TripletDistanceMetric.EUCLIDEAN",
      "triplet_margin": 5
  }
  

Training Hyperparameters

Non-Default Hyperparameters

• eval_strategy: steps
• per_device_train_batch_size: 16
• per_device_eval_batch_size: 16
• learning_rate: 2e-05
• num_train_epochs: 4
• warmup_ratio: 0.1
• fp16: True
• batch_sampler: no_duplicates

All Hyperparameters

Click to expand

• overwrite_output_dir: False
• do_predict: False
• eval_strategy: steps
• prediction_loss_only: True
• per_device_train_batch_size: 16
• per_device_eval_batch_size: 16
• per_gpu_train_batch_size: None
• per_gpu_eval_batch_size: None
• gradient_accumulation_steps: 1
• eval_accumulation_steps: None
• learning_rate: 2e-05
• weight_decay: 0.0
• adam_beta1: 0.9
• adam_beta2: 0.999
• adam_epsilon: 1e-08
• max_grad_norm: 1.0
• num_train_epochs: 4
• max_steps: -1
• lr_scheduler_type: linear
• lr_scheduler_kwargs: {}
• warmup_ratio: 0.1
• warmup_steps: 0
• log_level: passive
• log_level_replica: warning
• log_on_each_node: True
• logging_nan_inf_filter: True
• save_safetensors: True
• save_on_each_node: False
• save_only_model: False
• restore_callback_states_from_checkpoint: False
• no_cuda: False
• use_cpu: False
• use_mps_device: False
• seed: 42
• data_seed: None
• jit_mode_eval: False
• use_ipex: False
• bf16: False
• fp16: True
• fp16_opt_level: O1
• half_precision_backend: auto
• bf16_full_eval: False
• fp16_full_eval: False
• tf32: None
• local_rank: 0
• ddp_backend: None
• tpu_num_cores: None
• tpu_metrics_debug: False
• debug: []
• dataloader_drop_last: False
• dataloader_num_workers: 0
• dataloader_prefetch_factor: None
• past_index: -1
• disable_tqdm: False
• remove_unused_columns: True
• label_names: None
• load_best_model_at_end: False
• ignore_data_skip: False
• fsdp: []
• fsdp_min_num_params: 0
• fsdp_config: {'min_num_params': 0, 'xla': False, 'xla_fsdp_v2': False, 'xla_fsdp_grad_ckpt': False}
• fsdp_transformer_layer_cls_to_wrap: None
• accelerator_config: {'split_batches': False, 'dispatch_batches': None, 'even_batches': True, 'use_seedable_sampler': True, 'non_blocking': False, 'gradient_accumulation_kwargs': None}
• deepspeed: None
• label_smoothing_factor: 0.0
• optim: adamw_torch
• optim_args: None
• adafactor: False
• group_by_length: False
• length_column_name: length
• ddp_find_unused_parameters: None
• ddp_bucket_cap_mb: None
• ddp_broadcast_buffers: False
• dataloader_pin_memory: True
• dataloader_persistent_workers: False
• skip_memory_metrics: True
• use_legacy_prediction_loop: False
• push_to_hub: False
• resume_from_checkpoint: None
• hub_model_id: None
• hub_strategy: every_save
• hub_private_repo: False
• hub_always_push: False
• gradient_checkpointing: False
• gradient_checkpointing_kwargs: None
• include_inputs_for_metrics: False
• eval_do_concat_batches: True
• fp16_backend: auto
• push_to_hub_model_id: None
• push_to_hub_organization: None
• mp_parameters:
• auto_find_batch_size: False
• full_determinism: False
• torchdynamo: None
• ray_scope: last
• ddp_timeout: 1800
• torch_compile: False
• torch_compile_backend: None
• torch_compile_mode: None
• dispatch_batches: None
• split_batches: None
• include_tokens_per_second: False
• include_num_input_tokens_seen: False
• neftune_noise_alpha: None
• optim_target_modules: None
• batch_eval_metrics: False
• batch_sampler: no_duplicates
• multi_dataset_batch_sampler: proportional

Training Logs

Epoch	Step	Training Loss	loss	all-nli-dev_max_accuracy	all-nli-test_max_accuracy
0	0	-	-	0.7235	-
0.0454	100	4.2756	3.5710	0.7091	-
0.0907	200	1.7605	0.2244	0.6005	-
0.1361	300	0.0792	0.1934	0.5856	-
0.1815	400	0.0783	0.1707	0.5636	-
0.2269	500	0.067	0.1520	0.5534	-
0.2722	600	0.0748	0.1315	0.5518	-
0.3176	700	0.0673	0.1061	0.5313	-
0.3630	800	0.0348	0.0989	0.5063	-
0.4083	900	0.0614	0.0783	0.5025	-
0.4537	1000	0.0195	0.0735	0.5241	-
0.4991	1100	0.0279	0.0670	0.5093	-
0.5445	1200	0.0318	0.0537	0.5158	-
0.5898	1300	0.0281	0.0511	0.5074	-
0.6352	1400	0.0162	0.0520	0.5063	-
0.6806	1500	0.0072	0.0508	0.5028	-
0.7260	1600	0.0227	0.0561	0.4861	-
0.7713	1700	0.0162	0.0465	0.4911	-
0.8167	1800	0.0185	0.0440	0.5192	-
0.8621	1900	0.03	0.0452	0.5180	-
0.9074	2000	0.0281	0.0450	0.4839	-
0.9528	2100	0.0133	0.0443	0.4994	-
0.9982	2200	0.0154	0.0363	0.4968	-
1.0436	2300	0.0198	0.0355	0.4869	-
1.0889	2400	0.0083	0.1174	0.5222	-
1.1343	2500	0.0108	0.0430	0.4911	-
1.1797	2600	0.0079	0.0411	0.4873	-
1.2250	2700	0.0077	0.0437	0.4804	-
1.2704	2800	0.017	0.0331	0.4812	-
1.3158	2900	0.0126	0.0310	0.4979	-
1.3612	3000	0.0105	0.0555	0.4918	-
1.4065	3100	0.0161	0.0425	0.4801	-
1.4519	3200	0.0017	0.0274	0.4865	-
1.4973	3300	0.0062	0.0265	0.4808	-
1.5426	3400	0.0069	0.0338	0.4854	-
1.5880	3500	0.0038	0.0304	0.5120	-
1.6334	3600	0.0067	0.0320	0.4941	-
1.6788	3700	0.0013	0.0300	0.5013	-
1.7241	3800	0.0047	0.0265	0.5154	-
1.7695	3900	0.0068	0.0245	0.4956	-
1.8149	4000	0.005	0.0203	0.5127	-
1.8603	4100	0.0137	0.0240	0.5158	-
1.9056	4200	0.0095	0.0404	0.5028	-
1.9510	4300	0.0102	0.0312	0.4808	-
1.9964	4400	0.0056	0.0339	0.4823	-
2.0417	4500	0.0124	0.0250	0.4839	-
2.0871	4600	0.0131	0.0230	0.4945	-
2.1325	4700	0.0024	0.0180	0.5025	-
2.1779	4800	0.0078	0.0216	0.5066	-
2.2232	4900	0.0022	0.0181	0.5013	-
2.2686	5000	0.013	0.0200	0.4759	-
2.3140	5100	0.009	0.0175	0.4926	-
2.3593	5200	0.0046	0.0206	0.4880	-
2.4047	5300	0.0034	0.0225	0.4972	-
2.4501	5400	0.0006	0.0206	0.4956	-
2.4955	5500	0.0009	0.0275	0.4865	-
2.5408	5600	0.0098	0.0246	0.4873	-
2.5862	5700	0.0017	0.0203	0.4861	-
2.6316	5800	0.0004	0.0219	0.4930	-
2.6770	5900	0.001	0.0172	0.4892	-
2.7223	6000	0.002	0.0254	0.4850	-
2.7677	6100	0.0002	0.0242	0.4888	-
2.8131	6200	0.0039	0.0237	0.4877	-
2.8584	6300	0.0148	0.0310	0.5078	-
2.9038	6400	0.0	0.0234	0.4865	-
2.9492	6500	0.0036	0.0187	0.4899	-
2.9946	6600	0.0	0.0196	0.4823	-
3.0399	6700	0.0015	0.0166	0.4850	-
3.0853	6800	0.0058	0.0165	0.4877	-
3.1307	6900	0.0	0.0165	0.4869	-
3.1760	7000	0.0023	0.0169	0.4873	-
3.2214	7100	0.0	0.0169	0.4877	-
3.2668	7200	0.004	0.0163	0.4850	-
3.3122	7300	0.0015	0.0155	0.4926	-
3.3575	7400	0.0007	0.0136	0.4918	-
3.4029	7500	0.0	0.0128	0.4892	-
3.4483	7600	0.0	0.0128	0.4888	-
3.4936	7700	0.0002	0.0132	0.4964	-
3.5390	7800	0.0062	0.0167	0.4869	-
3.5844	7900	0.0008	0.0194	0.4907	-
3.6298	8000	0.0	0.0194	0.4907	-
3.6751	8100	0.0	0.0179	0.4869	-
3.7205	8200	0.0	0.0178	0.4865	-
3.7659	8300	0.0002	0.0155	0.4827	-
3.8113	8400	0.0019	0.0155	0.4842	-
3.8566	8500	0.0008	0.0171	0.4880	-
3.9020	8600	0.0026	0.0177	0.4888	-
3.9474	8700	0.0	0.0179	0.4892	-
3.9927	8800	0.0	0.0179	0.4892	-
4.0	8816	-	-	-	0.4892

Framework Versions

• Python: 3.11.9
• Sentence Transformers: 3.0.1
• Transformers: 4.41.2
• PyTorch: 2.1.2+cu121
• Accelerate: 0.31.0
• Datasets: 2.19.1
• Tokenizers: 0.19.1

Citation

BibTeX

Sentence Transformers

@inproceedings{reimers-2019-sentence-bert,
    title = "Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks",
    author = "Reimers, Nils and Gurevych, Iryna",
    booktitle = "Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing",
    month = "11",
    year = "2019",
    publisher = "Association for Computational Linguistics",
    url = "https://arxiv.org/abs/1908.10084",
}

TripletLoss

@misc{hermans2017defense,
    title={In Defense of the Triplet Loss for Person Re-Identification}, 
    author={Alexander Hermans and Lucas Beyer and Bastian Leibe},
    year={2017},
    eprint={1703.07737},
    archivePrefix={arXiv},
    primaryClass={cs.CV}
}