Hugging Face's logo

MugheesAwan11
/
bge-base-securiti-dataset-1-v20

Sentence Similarity
sentence-transformers
Safetensors
English
bert
feature-extraction
Generated from Trainer
dataset_size:494
loss:MatryoshkaLoss
loss:MultipleNegativesRankingLoss
Eval Results
Inference Endpoints
text-embeddings-inference
Model card Files Community
bge-base-securiti-dataset-1-v20 / README.md
MugheesAwan11's picture
MugheesAwan11
Add new SentenceTransformer model.
89a30b8 verified
preview code
|
raw history blame contribute delete
No virus
39 kB
metadata
base_model: BAAI/bge-base-en-v1.5
datasets: []
language:
  - en
library_name: sentence-transformers
license: apache-2.0
metrics:
  - cosine_accuracy@1
  - cosine_accuracy@3
  - cosine_accuracy@5
  - cosine_accuracy@10
  - cosine_precision@1
  - cosine_precision@3
  - cosine_precision@5
  - cosine_precision@10
  - cosine_recall@1
  - cosine_recall@3
  - cosine_recall@5
  - cosine_recall@10
  - cosine_ndcg@10
  - cosine_mrr@10
  - cosine_map@100
pipeline_tag: sentence-similarity
tags:
  - sentence-transformers
  - sentence-similarity
  - feature-extraction
  - generated_from_trainer
  - dataset_size:494
  - loss:MatryoshkaLoss
  - loss:MultipleNegativesRankingLoss
widget:
  - source_sentence: >-
      Program Join our Partner Program Contact Us Contact us to learn more or
      schedule a demo News Coverage Read about Securiti in the news Press
      Releases Find our latest press releases Careers Join the talented Securiti
      team Knowledge Center » Data Privacy Automation # New Zealand's Privacy
      Act of 2020 By Securiti Research Team Published March 7, 2022 / Updated
      August 11, 2023 New Zealand was one of the first countries that enacted a
      law specifically dedicated to its residents' right to privacy with its
      Privacy Act of 1993. Whilst the entire definition of what "privacy" means
      has undergone a radical shift since then New Zealand’s principles based
      legislation has remained relatively fit for purpose. Even with the advent
      of social media and the internet adding an entirely new paradigm to that
      topic. In recognition of the evolution of privacy, New Zealand updated its
    sentences:
      - Where can I find Securiti's latest press releases?
      - >-
        What are the requirements for data transfer under Spain's data
        protection law, including certifications and information for data
        subjects?
      - >-
        What is the term for the right to delete personal data upon request,
        also known as 'the right to be forgotten', and what are the other data
        protection rights under GDPR?
  - source_sentence: >-
      that the third party: has appropriate policies and processes in place; has
      trained its staff to ensure information is appropriately safeguarded at
      all times; has adequate security measures in place. Simultaneously, the
      Cross-border Guidelines also specify that organizations must provide
      notice to customers that: their personal information may be sent to
      another jurisdiction for processing; while the information is in the other
      jurisdiction, it may be accessed by the courts, law enforcement, and
      national security authorities. ## 10\. Data Subject Rights PIPEDA bestows
      the following rights to data subjects: Right to access Right to accuracy
      and completeness Right to withdraw consent and submit complaints ## 11\.
      Penalties for PIPEDA Non-Compliance PIPEDA imposes administrative
      penalties for non-compliance, where the amount may vary depending upon the
      severity and the kind of violation. According to PIPEDA,  : organizations
      must keep personal information accurate. 7. **Safeguards** : organizations
      must protect personal information against loss or theft. 8. **Openness** :
      privacy policy and practices must be understandable and easily available.
      9. **Individual access** : data subjects have a right to access the
      personal information an organization holds about them. 10. **Resource** :
      organizations must develop accessible complaint procedures. ## 3\.
      Obligations for the Data Controller and Data Processor PIPEDA does not
      differentiate between data controllers and data processors and provides a
      similar set of responsibilities for both controllers and processors.
      PIPEDA demands all organizations appoint individuals who will be
      accountable for ensuring streamlined compliance of an organization’s data
      activities in accordance with the provisions of PIPEDA. ## 4\. Consent
      Requirements In many circumstances, PIPEDA requires organizations to
      obtain the data subject’s consent to use, disclose, and retain any
      personal information.
    sentences:
      - What are the key provisions of South Korea's data privacy law?
      - >-
        What are the circumstances in which the data subject must be notified
        about the collection of personal data?
      - >-
        How does PIPEDA ensure staff's compliance with guidelines and
        obligations regarding information protection?
  - source_sentence: >-
      The criteria used The purpose of processing This information must be
      provided within 15 days from the date of the data subject’s request. vs
      GDPR states that, when responding to an access request, a data controller
      must indicate the following: The categories of personal data concerned The
      recipients or categories of recipients to whom personal data have been
      disclosed to The retention period The right to lodge a complaint with the
      supervisory authority The existence of data transfers The existence of
      automated decision making The information must be provided without undue
      delay and in any event within one month of the receipt of the request.
      LGPD grants the right to data portability through an express request and
      subject to commercial and industrial secrecy, pursuant to the regulation
      of the controlling agency. This right, however, does not include data that
      has already been anonymised by the controller. vs GDPR defines the right
      to
    sentences:
      - >-
        What is considered an offense related to obstructing the OPC in an
        investigation?
      - What does LGPD grant the right to in terms of data portability?
      - >-
        How does automation aid in complying with data privacy regulations like
        the PDPO?
  - source_sentence: >-
      uriti Research Team Published December 3, 2020 / Updated October 3, 2023
      On 1 December 2020, New Zealand’s new Privacy Act 2020 came into effect.
      Our experts at Securiti have compiled the following list of compliance
      actions to remind organizations of their obligations under New Zealand’s
      new Privacy Act. ## 1\. Notify privacy breaches within 72 hours
      Organizations must notify privacy breach that has caused serious harm to
      the affected individual or is likely to do so, to the Privacy Commissioner
      and the affected individuals as soon as practicable or within 72 hours
      after becoming aware of the breach. Where it is not reasonably practicable
      to notify the affected individual or each member of a group of affected
      individuals, organizations must notify the public in a manner that no
      individual is identified. Companies that fail to notify privacy breaches
      without any reasonable excuse would be liable on conviction to a fine not
      exceeding $10,000. ## 2\. Notify privacy breaches caused by any
    sentences:
      - >-
        When are controllers and data processors required to appoint a DPO
        according to the PDP Law and state regulations in Indonesia?
      - >-
        What is the time frame for notifying privacy breaches under New
        Zealand's new Privacy Act?
      - >-
        What rights do Colorado residents have over their personal data under
        the Colorado Privacy Act?
  - source_sentence: >-
      Careers View Events Spotlight Talks IDC Names Securiti a Worldwide Leader
      in Data Privacy View Events Spotlight Talks Education Contact Us Schedule
      a Demo Products By Use Cases By Roles Data Command Center View Learn more
      Asset and Data Discovery Discover dark and native data assets Learn more
      Data Access Intelligence & Governance Identify which users have access to
      sensitive data and prevent unauthorized access Learn more Data Privacy
      Automation PrivacyCenter.Cloud | Data Mapping | DSR Automation |
      Assessment Automation | Vendor Assessment | Breach Management | Privacy
      Notice Learn more Sensitive Data Intelligence Discover & Classify
      Structured and Unstructured Data | People Data Graph Learn more Data Flow
      Intelligence & Governance Prevent sensitive data sprawl through real-,
      Press Releases View Careers View Events Spotlight Talks IDC Names Securiti
      a Worldwide Leader in Data Privacy View Events Spotlight Talks Education
      Contact Us Schedule a Demo Products By Use Cases By Roles Data Command
      Center View Learn more Asset and Data Discovery Discover dark and native
      data assets Learn more Data Access Intelligence & Governance Identify
      which users have access to sensitive data and prevent unauthorized access
      Learn more Data Privacy Automation PrivacyCenter.Cloud | Data Mapping |
      DSR Automation | Assessment Automation | Vendor Assessment | Breach
      Management | Privacy Notice Learn more Sensitive Data Intelligence
      Discover & Classify Structured and Unstructured Data | People Data Graph
      Learn more Data Flow Intelligence & Governance Prevent
    sentences:
      - What is the purpose of the Data Command Center?
      - >-
        What are IBM's future prospects and preparedness for new business
        opportunities?
      - What is the US California CCPA?
model-index:
  - name: SentenceTransformer based on BAAI/bge-base-en-v1.5
    results:
      - task:
          type: information-retrieval
          name: Information Retrieval
        dataset:
          name: dim 768
          type: dim_768
        metrics:
          - type: cosine_accuracy@1
            value: 0.34845360824742266
            name: Cosine Accuracy@1
          - type: cosine_accuracy@3
            value: 0.5855670103092784
            name: Cosine Accuracy@3
          - type: cosine_accuracy@5
            value: 0.6701030927835051
            name: Cosine Accuracy@5
          - type: cosine_accuracy@10
            value: 0.756701030927835
            name: Cosine Accuracy@10
          - type: cosine_precision@1
            value: 0.34845360824742266
            name: Cosine Precision@1
          - type: cosine_precision@3
            value: 0.1951890034364261
            name: Cosine Precision@3
          - type: cosine_precision@5
            value: 0.13402061855670103
            name: Cosine Precision@5
          - type: cosine_precision@10
            value: 0.0756701030927835
            name: Cosine Precision@10
          - type: cosine_recall@1
            value: 0.34845360824742266
            name: Cosine Recall@1
          - type: cosine_recall@3
            value: 0.5855670103092784
            name: Cosine Recall@3
          - type: cosine_recall@5
            value: 0.6701030927835051
            name: Cosine Recall@5
          - type: cosine_recall@10
            value: 0.756701030927835
            name: Cosine Recall@10
          - type: cosine_ndcg@10
            value: 0.5507373799577976
            name: Cosine Ndcg@10
          - type: cosine_mrr@10
            value: 0.4849337260677468
            name: Cosine Mrr@10
          - type: cosine_map@100
            value: 0.4942402452655515
            name: Cosine Map@100
      - task:
          type: information-retrieval
          name: Information Retrieval
        dataset:
          name: dim 512
          type: dim_512
        metrics:
          - type: cosine_accuracy@1
            value: 0.3463917525773196
            name: Cosine Accuracy@1
          - type: cosine_accuracy@3
            value: 0.5938144329896907
            name: Cosine Accuracy@3
          - type: cosine_accuracy@5
            value: 0.668041237113402
            name: Cosine Accuracy@5
          - type: cosine_accuracy@10
            value: 0.756701030927835
            name: Cosine Accuracy@10
          - type: cosine_precision@1
            value: 0.3463917525773196
            name: Cosine Precision@1
          - type: cosine_precision@3
            value: 0.1979381443298969
            name: Cosine Precision@3
          - type: cosine_precision@5
            value: 0.13360824742268038
            name: Cosine Precision@5
          - type: cosine_precision@10
            value: 0.07567010309278348
            name: Cosine Precision@10
          - type: cosine_recall@1
            value: 0.3463917525773196
            name: Cosine Recall@1
          - type: cosine_recall@3
            value: 0.5938144329896907
            name: Cosine Recall@3
          - type: cosine_recall@5
            value: 0.668041237113402
            name: Cosine Recall@5
          - type: cosine_recall@10
            value: 0.756701030927835
            name: Cosine Recall@10
          - type: cosine_ndcg@10
            value: 0.5517739147624575
            name: Cosine Ndcg@10
          - type: cosine_mrr@10
            value: 0.48604565537555244
            name: Cosine Mrr@10
          - type: cosine_map@100
            value: 0.4956303541940711
            name: Cosine Map@100
      - task:
          type: information-retrieval
          name: Information Retrieval
        dataset:
          name: dim 256
          type: dim_256
        metrics:
          - type: cosine_accuracy@1
            value: 0.3422680412371134
            name: Cosine Accuracy@1
          - type: cosine_accuracy@3
            value: 0.5670103092783505
            name: Cosine Accuracy@3
          - type: cosine_accuracy@5
            value: 0.6618556701030928
            name: Cosine Accuracy@5
          - type: cosine_accuracy@10
            value: 0.7484536082474227
            name: Cosine Accuracy@10
          - type: cosine_precision@1
            value: 0.3422680412371134
            name: Cosine Precision@1
          - type: cosine_precision@3
            value: 0.1890034364261168
            name: Cosine Precision@3
          - type: cosine_precision@5
            value: 0.13237113402061854
            name: Cosine Precision@5
          - type: cosine_precision@10
            value: 0.07484536082474226
            name: Cosine Precision@10
          - type: cosine_recall@1
            value: 0.3422680412371134
            name: Cosine Recall@1
          - type: cosine_recall@3
            value: 0.5670103092783505
            name: Cosine Recall@3
          - type: cosine_recall@5
            value: 0.6618556701030928
            name: Cosine Recall@5
          - type: cosine_recall@10
            value: 0.7484536082474227
            name: Cosine Recall@10
          - type: cosine_ndcg@10
            value: 0.5412682955861301
            name: Cosine Ndcg@10
          - type: cosine_mrr@10
            value: 0.475321551300933
            name: Cosine Mrr@10
          - type: cosine_map@100
            value: 0.48455040697749474
            name: Cosine Map@100

SentenceTransformer based on BAAI/bge-base-en-v1.5

This is a sentence-transformers model finetuned from BAAI/bge-base-en-v1.5. It maps sentences & paragraphs to a 768-dimensional dense vector space and can be used for semantic textual similarity, semantic search, paraphrase mining, text classification, clustering, and more.

Model Details

Model Description

  • Model Type: Sentence Transformer
  • Base model: BAAI/bge-base-en-v1.5
  • Maximum Sequence Length: 512 tokens
  • Output Dimensionality: 768 tokens
  • Similarity Function: Cosine Similarity
  • Language: en
  • License: apache-2.0

Model Sources

Full Model Architecture 

SentenceTransformer(
  (0): Transformer({'max_seq_length': 512, 'do_lower_case': True}) with Transformer model: BertModel 
  (1): Pooling({'word_embedding_dimension': 768, 'pooling_mode_cls_token': True, 'pooling_mode_mean_tokens': False, 'pooling_mode_max_tokens': False, 'pooling_mode_mean_sqrt_len_tokens': False, 'pooling_mode_weightedmean_tokens': False, 'pooling_mode_lasttoken': False, 'include_prompt': True})
  (2): Normalize()
)

Usage

Direct Usage (Sentence Transformers)

First install the Sentence Transformers library:

pip install -U sentence-transformers

Then you can load this model and run inference.

from sentence_transformers import SentenceTransformer

# Download from the 🤗 Hub
model = SentenceTransformer("MugheesAwan11/bge-base-securiti-dataset-1-v20")
# Run inference
sentences = [
    'Careers View Events Spotlight Talks IDC Names Securiti a Worldwide Leader in Data Privacy View Events Spotlight Talks Education Contact Us Schedule a Demo Products By Use Cases By Roles Data Command Center View Learn more Asset and Data Discovery Discover dark and native data assets Learn more Data Access Intelligence & Governance Identify which users have access to sensitive data and prevent unauthorized access Learn more Data Privacy Automation PrivacyCenter.Cloud | Data Mapping | DSR Automation | Assessment Automation | Vendor Assessment | Breach Management | Privacy Notice Learn more Sensitive Data Intelligence Discover & Classify Structured and Unstructured Data | People Data Graph Learn more Data Flow Intelligence & Governance Prevent sensitive data sprawl through real-, Press Releases View Careers View Events Spotlight Talks IDC Names Securiti a Worldwide Leader in Data Privacy View Events Spotlight Talks Education Contact Us Schedule a Demo Products By Use Cases By Roles Data Command Center View Learn more Asset and Data Discovery Discover dark and native data assets Learn more Data Access Intelligence & Governance Identify which users have access to sensitive data and prevent unauthorized access Learn more Data Privacy Automation PrivacyCenter.Cloud | Data Mapping | DSR Automation | Assessment Automation | Vendor Assessment | Breach Management | Privacy Notice Learn more Sensitive Data Intelligence Discover & Classify Structured and Unstructured Data | People Data Graph Learn more Data Flow Intelligence & Governance Prevent',
    'What is the purpose of the Data Command Center?',
    "What are IBM's future prospects and preparedness for new business opportunities?",
]
embeddings = model.encode(sentences)
print(embeddings.shape)
# [3, 768]

# Get the similarity scores for the embeddings
similarities = model.similarity(embeddings, embeddings)
print(similarities.shape)
# [3, 3]

Evaluation

Metrics

Information Retrieval

Metric Value
cosine_accuracy@1 0.3485
cosine_accuracy@3 0.5856
cosine_accuracy@5 0.6701
cosine_accuracy@10 0.7567
cosine_precision@1 0.3485
cosine_precision@3 0.1952
cosine_precision@5 0.134
cosine_precision@10 0.0757
cosine_recall@1 0.3485
cosine_recall@3 0.5856
cosine_recall@5 0.6701
cosine_recall@10 0.7567
cosine_ndcg@10 0.5507
cosine_mrr@10 0.4849
cosine_map@100 0.4942

Information Retrieval

Metric Value
cosine_accuracy@1 0.3464
cosine_accuracy@3 0.5938
cosine_accuracy@5 0.668
cosine_accuracy@10 0.7567
cosine_precision@1 0.3464
cosine_precision@3 0.1979
cosine_precision@5 0.1336
cosine_precision@10 0.0757
cosine_recall@1 0.3464
cosine_recall@3 0.5938
cosine_recall@5 0.668
cosine_recall@10 0.7567
cosine_ndcg@10 0.5518
cosine_mrr@10 0.486
cosine_map@100 0.4956

Information Retrieval

Metric Value
cosine_accuracy@1 0.3423
cosine_accuracy@3 0.567
cosine_accuracy@5 0.6619
cosine_accuracy@10 0.7485
cosine_precision@1 0.3423
cosine_precision@3 0.189
cosine_precision@5 0.1324
cosine_precision@10 0.0748
cosine_recall@1 0.3423
cosine_recall@3 0.567
cosine_recall@5 0.6619
cosine_recall@10 0.7485
cosine_ndcg@10 0.5413
cosine_mrr@10 0.4753
cosine_map@100 0.4846

Training Details

Training Dataset

Unnamed Dataset

  • Size: 494 training samples
  • Columns: positive and anchor
  • Approximate statistics based on the first 1000 samples:
    positive anchor
    type string string
    details
    • min: 18 tokens
    • mean: 223.56 tokens
    • max: 414 tokens
    • min: 10 tokens
    • mean: 21.87 tokens
    • max: 102 tokens
  • Samples:
    positive anchor
    ### Denmark #### Denmark Effective Date : May 25, 2018 Region : EMEA (Europe, Middle East, Africa) Similar to other EU countries, Denmark has enacted a data protection act for the purpose of implementing the GDPR in the country. The Danish Data Protection Act (Act No. 502 of 23 May 2018) was enacted for the protection of natural persons with respect to personal data processing and to regulate the free movement of personal data. The Act replaced the previous Danish Act on Processing of Personal Data (Act no. 429 of 31/05/2000). Under the new Act, the Danish Data Protection Authority (Datatilsynet) oversees all aspects related to the supervision and enforcement of the Data Protection Act and the GDPR within the country as well as representing Denmark in the European Data Protection Board. ### Finland #### Finland Effective Date : January 1, 2019 Region : EMEA (Europe What is the role of the Danish Data Protection Authority in Denmark's implementation of the GDPR?
    CPRA compliance involves adhering to the requirements outlined in the California Privacy Rights Act (CPRA) to protect consumer privacy and data rights. ## Join Our Newsletter Get all the latest information, law updates and more delivered to your inbox ### Share Copy 91 ### More Stories that May Interest You View More September 13, 2023 ## Kuwait's DPPR Kuwait didn’t have any data protection law until the Communication and Information Technology Regulatory Authority (CITRA) introduced the Data Privacy Protection Regulation (DPPR). The... View More September 11, 2023 ## Indonesia’s Protection of Personal Data Law: Explained In January 2020, Indonesia joined the burgeoning list of countries with their own data protection regulations. Provisions for data protection had existed within various... View More August 31, 2023 ## Why is it important to comply with CPRA requirements and how does it protect data rights?
    Data Access Intelligence & Governance Identify which users have access to sensitive data and prevent unauthorized access Learn more Data Privacy Automation PrivacyCenter.Cloud Data Mapping
  • Loss: MatryoshkaLoss with these parameters:
    {
    "loss": "MultipleNegativesRankingLoss",
    "matryoshka_dims": [
        768,
        512,
        256
    ],
    "matryoshka_weights": [
        1,
        1,
        1
    ],
    "n_dims_per_step": -1
}

Training Hyperparameters

Non-Default Hyperparameters

  • eval_strategy: epoch
  • per_device_train_batch_size: 32
  • per_device_eval_batch_size: 16
  • learning_rate: 2e-05
  • num_train_epochs: 4
  • lr_scheduler_type: cosine
  • warmup_ratio: 0.1
  • bf16: True
  • tf32: True
  • load_best_model_at_end: True
  • optim: adamw_torch_fused
  • batch_sampler: no_duplicates

All Hyperparameters

Click to expand
  • overwrite_output_dir: False
  • do_predict: False
  • eval_strategy: epoch
  • prediction_loss_only: True
  • per_device_train_batch_size: 32
  • per_device_eval_batch_size: 16
  • per_gpu_train_batch_size: None
  • per_gpu_eval_batch_size: None
  • gradient_accumulation_steps: 1
  • eval_accumulation_steps: None
  • learning_rate: 2e-05
  • weight_decay: 0.0
  • adam_beta1: 0.9
  • adam_beta2: 0.999
  • adam_epsilon: 1e-08
  • max_grad_norm: 1.0
  • num_train_epochs: 4
  • max_steps: -1
  • lr_scheduler_type: cosine
  • lr_scheduler_kwargs: {}
  • warmup_ratio: 0.1
  • warmup_steps: 0
  • log_level: passive
  • log_level_replica: warning
  • log_on_each_node: True
  • logging_nan_inf_filter: True
  • save_safetensors: True
  • save_on_each_node: False
  • save_only_model: False
  • restore_callback_states_from_checkpoint: False
  • no_cuda: False
  • use_cpu: False
  • use_mps_device: False
  • seed: 42
  • data_seed: None
  • jit_mode_eval: False
  • use_ipex: False
  • bf16: True
  • fp16: False
  • fp16_opt_level: O1
  • half_precision_backend: auto
  • bf16_full_eval: False
  • fp16_full_eval: False
  • tf32: True
  • local_rank: 0
  • ddp_backend: None
  • tpu_num_cores: None
  • tpu_metrics_debug: False
  • debug: []
  • dataloader_drop_last: False
  • dataloader_num_workers: 0
  • dataloader_prefetch_factor: None
  • past_index: -1
  • disable_tqdm: False
  • remove_unused_columns: True
  • label_names: None
  • load_best_model_at_end: True
  • ignore_data_skip: False
  • fsdp: []
  • fsdp_min_num_params: 0
  • fsdp_config: {'min_num_params': 0, 'xla': False, 'xla_fsdp_v2': False, 'xla_fsdp_grad_ckpt': False}
  • fsdp_transformer_layer_cls_to_wrap: None
  • accelerator_config: {'split_batches': False, 'dispatch_batches': None, 'even_batches': True, 'use_seedable_sampler': True, 'non_blocking': False, 'gradient_accumulation_kwargs': None}
  • deepspeed: None
  • label_smoothing_factor: 0.0
  • optim: adamw_torch_fused
  • optim_args: None
  • adafactor: False
  • group_by_length: False
  • length_column_name: length
  • ddp_find_unused_parameters: None
  • ddp_bucket_cap_mb: None
  • ddp_broadcast_buffers: False
  • dataloader_pin_memory: True
  • dataloader_persistent_workers: False
  • skip_memory_metrics: True
  • use_legacy_prediction_loop: False
  • push_to_hub: False
  • resume_from_checkpoint: None
  • hub_model_id: None
  • hub_strategy: every_save
  • hub_private_repo: False
  • hub_always_push: False
  • gradient_checkpointing: False
  • gradient_checkpointing_kwargs: None
  • include_inputs_for_metrics: False
  • eval_do_concat_batches: True
  • fp16_backend: auto
  • push_to_hub_model_id: None
  • push_to_hub_organization: None
  • mp_parameters:
  • auto_find_batch_size: False
  • full_determinism: False
  • torchdynamo: None
  • ray_scope: last
  • ddp_timeout: 1800
  • torch_compile: False
  • torch_compile_backend: None
  • torch_compile_mode: None
  • dispatch_batches: None
  • split_batches: None
  • include_tokens_per_second: False
  • include_num_input_tokens_seen: False
  • neftune_noise_alpha: None
  • optim_target_modules: None
  • batch_eval_metrics: False
  • batch_sampler: no_duplicates
  • multi_dataset_batch_sampler: proportional

Training Logs

Epoch Step Training Loss dim_256_cosine_map@100 dim_512_cosine_map@100 dim_768_cosine_map@100
0.625 10 3.7981 - - -
1.0 16 - 0.4653 0.4819 0.4810
1.25 20 2.2066 - - -
1.875 30 1.668 - - -
2.0 32 - 0.4837 0.4905 0.4933
2.5 40 0.9807 - - -
3.0 48 - 0.4846 0.4954 0.4949
3.125 50 1.0226 - - -
3.75 60 1.0564 - - -
4.0 64 - 0.4846 0.4956 0.4942
  • The bold row denotes the saved checkpoint.

Framework Versions

  • Python: 3.10.14
  • Sentence Transformers: 3.0.1
  • Transformers: 4.41.2
  • PyTorch: 2.1.2+cu121
  • Accelerate: 0.31.0
  • Datasets: 2.19.1
  • Tokenizers: 0.19.1

Citation

BibTeX

Sentence Transformers 

@inproceedings{reimers-2019-sentence-bert,
    title = "Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks",
    author = "Reimers, Nils and Gurevych, Iryna",
    booktitle = "Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing",
    month = "11",
    year = "2019",
    publisher = "Association for Computational Linguistics",
    url = "https://arxiv.org/abs/1908.10084",
}

MatryoshkaLoss 

@misc{kusupati2024matryoshka,
    title={Matryoshka Representation Learning}, 
    author={Aditya Kusupati and Gantavya Bhatt and Aniket Rege and Matthew Wallingford and Aditya Sinha and Vivek Ramanujan and William Howard-Snyder and Kaifeng Chen and Sham Kakade and Prateek Jain and Ali Farhadi},
    year={2024},
    eprint={2205.13147},
    archivePrefix={arXiv},
    primaryClass={cs.LG}
}

MultipleNegativesRankingLoss 

@misc{henderson2017efficient,
    title={Efficient Natural Language Response Suggestion for Smart Reply}, 
    author={Matthew Henderson and Rami Al-Rfou and Brian Strope and Yun-hsuan Sung and Laszlo Lukacs and Ruiqi Guo and Sanjiv Kumar and Balint Miklos and Ray Kurzweil},
    year={2017},
    eprint={1705.00652},
    archivePrefix={arXiv},
    primaryClass={cs.CL}
}