Debbie89889/esci-biencoder

SentenceTransformer based on sentence-transformers/all-MiniLM-L6-v2

This is a sentence-transformers model finetuned from sentence-transformers/all-MiniLM-L6-v2. It maps sentences & paragraphs to a 384-dimensional dense vector space and can be used for retrieval.

Model Details

Model Description

• Model Type: Sentence Transformer
• Base model: sentence-transformers/all-MiniLM-L6-v2
• Maximum Sequence Length: 128 tokens
• Output Dimensionality: 384 dimensions
• Similarity Function: Cosine Similarity
• Supported Modality: Text

Model Sources

• Documentation: Sentence Transformers Documentation
• Repository: Sentence Transformers on GitHub
• Hugging Face: Sentence Transformers on Hugging Face

Full Model Architecture

SentenceTransformer(
  (0): Transformer({'transformer_task': 'feature-extraction', 'modality_config': {'text': {'method': 'forward', 'method_output_name': 'last_hidden_state'}}, 'module_output_name': 'token_embeddings', 'architecture': 'BertModel'})
  (1): Pooling({'embedding_dimension': 384, 'pooling_mode': 'mean', 'include_prompt': True})
  (2): Normalize({})
)

Usage

Direct Usage (Sentence Transformers)

First install the Sentence Transformers library:

pip install -U sentence-transformers

Then you can load this model and run inference.

from sentence_transformers import SentenceTransformer

# Download from the 🤗 Hub
model = SentenceTransformer("sentence_transformers_model_id")
# Run inference
sentences = [
    'blue q socks',
    "Blue Q Women's Crew Socks , You Fancy B---h, Women's Shoe Size 5-10 Blue Q YOU FANCY BITCH: Pretty, just like you. Special, just like you. Not afraid to be a little flip, also like you.\nVERY BEST QUALITY: Blue Q Socks are woven from soft, luxurious combed cotton, nylon for strength, and spandex for long-lasting fun.\nONE SIZE FITS MOST: Fits women's shoe size 5-10.\nEASY CARE: Machine wash cold, tumble dry medium heat, do not bleach.\nGIVE BACK: 1% of the sale of Blue Q socks supports the humanitarian work of Doctors Without Borders/Medecins Sans Frontieres. They're good people doing great things around the world.",
    "Blue Q Socks, Men's Crew, Multi, Men's 7-12 Blue Q Blue Q Mens Let Her Win Men's Socks, Blue/Grey, OS",
]
embeddings = model.encode(sentences)
print(embeddings.shape)
# [3, 384]

# Get the similarity scores for the embeddings
similarities = model.similarity(embeddings, embeddings)
print(similarities)
# tensor([[1.0000, 0.7282, 0.7263],
#         [0.7282, 1.0000, 0.6520],
#         [0.7263, 0.6520, 1.0000]])

Training Details

Training Dataset

Unnamed Dataset

• Size: 148,465 training samples
• Columns: sentence_0, sentence_1, and sentence_2
• Approximate statistics based on the first 100 samples:

  	sentence_0	sentence_1	sentence_2
  type	string	string	string
  modality	text	text	text
  details	min: 3 tokens mean: 6.83 tokens max: 14 tokens	min: 6 tokens mean: 98.45 tokens max: 128 tokens	min: 3 tokens mean: 97.85 tokens max: 128 tokens

• Samples:

  sentence_0	sentence_1	sentence_2
  ipod touch	Original AppleiPod Compatible for mp3 mp4 Player Apple iPod Touch 5th gen 16GB Blue Apple	Apple iPod Touch (32GB) - Gold (Latest Model) Apple 4-Inch Retina display Your favorite music, gaming, and AR apps in your pocket A10 Fusion chip for up to 2x faster performance, Up to 3x faster graphics Group FaceTime with family and friends Connect using Wi-Fi and Bluetooth
  honeywell truefresh filters	Honeywell Premium Odor-Reducing Air Purifier Replacement Pre-Filter, HRF-APP1 / Filter (A+) Honeywell TRAP DUST & OTHER PARTICLES: This Certified Honeywell A+ filter deodorizes the air and traps larger particles such as dust, lint, fibers and pet fur For best results, replace this Odor & Gas Reducing Universal Pre-filter every 3 months ODOR REDUCING FILTER – This A+ air purifier replacement filter includes zeolite and activated carbon for extra odor reduction performance They help deodorize air that passes through the filter and absorb certain common harmful household gases and fumes WORKS WITH HONEYWELL AIR PURIFIERS FOR HOME - This air purifier filter can be easily cut to size to fit any Honeywell HEPA Air Purifier models Replace every 3 months for best performance #1 BRAND RECOMMENDED BY ALLERGISTS** – Honeywell Air Purifier, The Doctor’s Choice, is the #1 brand recommended by allergists Honeywell air purifiers are a trusted partner in the fight against airborne particles COMPATIBLE...	Honeywell HRF-B2 / Filter (B) Household Odor & Gas Reducing Pre-filter, 2 Pack Honeywell Helps capture certain harmful gases such as Ammonia and other volatile organic compounds from the air that passes through the filter Deodorizes the air, reducing odors from the air that passes through the filters with activated carbon and zeolite Maximizes life of the air purifier's primary filter Contains 2 pre-filters To ensure stated product performance, use only Genuine Honeywell Replacement Filters.
  mini measuring cup	Fox Run Brands 4-Ounce Mini Measuring Glass, Regular, Clear Fox Run Brands NEVER MISS A MEASUREMENT: With easy to read lines, this small measuring glass is sure to hit the mark every time. STURDY CONSTRUCTION: This mini measuring glass is made of thick and clear walled glass, so you can see what you're measuring without having to worry about wear and tear. CONVENIENT INDICATIONS: With line markings graduated in teaspoons, tablespoons, fluid ounces, and millilters all the way up to 4oz, this mini measuring glass has the information you're looking for. TAKE THE SHOT: This measuring glass isn't just ideal for spices and oils. It's also great for measuring small increments of liquids, so try using it to craft the perfect cocktails! MEASUREMENTS: This measuring cup weighs 3.2 ounces and weighs 2.25 x 2.25 x 3.25 inches.	3 Pcs Plastic Measuring Cups Set,4/2/1 Cup Capacity Measuring Jug with Handle Grip and Spout,Stackable Heat-resistant Cup for Measure Liquid,Baking,Mixing,BPA-free,Clear Fvcisshhu Package:3 Pack of plastic measuring cups with 3 different sizes.The size of the small measuring jug is 3.6 x 3.2 inch,1 cup/8 oz;the size of the middle one is 4 x 3.9 inch,2 cup/16 oz;the size of the large one is 5 x 4.9 inch,4 cup/32 oz. Design:The stackable measuring cup was design with a handle grip and a spout,the handle grip make you use the measure cup comfortably,the spout can guarantees accuracy targeting without dripping. Material:These measuring jug is made from high quality plastic,can directly contact food,lightweight and durable,no more worry about breaking. Easy to read:Both metric and US measurements are printed on the measuring cup in red color,cup ounce and milliliter markings are easy to convert.Please hand wash the cup to protect the scale. Function:These measuring jugs with spout can be us...

• Loss: MultipleNegativesRankingLoss with these parameters:

  {
      "scale": 20.0,
      "similarity_fct": "cos_sim",
      "gather_across_devices": false,
      "directions": [
          "query_to_doc"
      ],
      "partition_mode": "joint",
      "hardness_mode": null,
      "hardness_strength": 0.0
  }
  

Training Hyperparameters

Non-Default Hyperparameters

• per_device_train_batch_size: 64
• num_train_epochs: 1
• per_device_eval_batch_size: 64
• multi_dataset_batch_sampler: round_robin

All Hyperparameters

Click to expand

• per_device_train_batch_size: 64
• num_train_epochs: 1
• max_steps: -1
• learning_rate: 5e-05
• lr_scheduler_type: linear
• lr_scheduler_kwargs: None
• warmup_steps: 0
• optim: adamw_torch
• optim_args: None
• weight_decay: 0.0
• adam_beta1: 0.9
• adam_beta2: 0.999
• adam_epsilon: 1e-08
• optim_target_modules: None
• gradient_accumulation_steps: 1
• average_tokens_across_devices: True
• max_grad_norm: 1
• label_smoothing_factor: 0.0
• bf16: False
• fp16: False
• bf16_full_eval: False
• fp16_full_eval: False
• tf32: None
• gradient_checkpointing: False
• gradient_checkpointing_kwargs: None
• torch_compile: False
• torch_compile_backend: None
• torch_compile_mode: None
• use_liger_kernel: False
• liger_kernel_config: None
• use_cache: False
• neftune_noise_alpha: None
• torch_empty_cache_steps: None
• auto_find_batch_size: False
• log_on_each_node: True
• logging_nan_inf_filter: True
• include_num_input_tokens_seen: no
• log_level: passive
• log_level_replica: warning
• disable_tqdm: False
• project: huggingface
• trackio_space_id: None
• trackio_bucket_id: None
• trackio_static_space_id: None
• per_device_eval_batch_size: 64
• prediction_loss_only: True
• eval_on_start: False
• eval_do_concat_batches: True
• eval_use_gather_object: False
• eval_accumulation_steps: None
• include_for_metrics: []
• batch_eval_metrics: False
• save_only_model: False
• save_on_each_node: False
• enable_jit_checkpoint: False
• push_to_hub: False
• hub_private_repo: None
• hub_model_id: None
• hub_strategy: every_save
• hub_always_push: False
• hub_revision: None
• load_best_model_at_end: False
• ignore_data_skip: False
• restore_callback_states_from_checkpoint: False
• full_determinism: False
• seed: 42
• data_seed: None
• use_cpu: False
• accelerator_config: {'split_batches': False, 'dispatch_batches': None, 'even_batches': True, 'use_seedable_sampler': True, 'non_blocking': False, 'gradient_accumulation_kwargs': None}
• parallelism_config: None
• dataloader_drop_last: False
• dataloader_num_workers: 0
• dataloader_pin_memory: True
• dataloader_persistent_workers: False
• dataloader_prefetch_factor: None
• remove_unused_columns: True
• label_names: None
• train_sampling_strategy: random
• length_column_name: length
• ddp_find_unused_parameters: None
• ddp_bucket_cap_mb: None
• ddp_broadcast_buffers: False
• ddp_static_graph: None
• ddp_backend: None
• ddp_timeout: 1800
• fsdp: None
• fsdp_config: None
• deepspeed: None
• debug: []
• skip_memory_metrics: True
• do_predict: False
• resume_from_checkpoint: None
• warmup_ratio: None
• local_rank: -1
• prompts: None
• batch_sampler: batch_sampler
• multi_dataset_batch_sampler: round_robin
• router_mapping: {}
• learning_rate_mapping: {}

Training Logs

Epoch	Step	Training Loss
0.2155	500	1.5239
0.4310	1000	1.2718
0.6466	1500	1.1702
0.8621	2000	1.1148

Training Time

• Training: 8.6 minutes

Framework Versions

• Python: 3.11.15
• Sentence Transformers: 5.6.0
• Transformers: 5.12.1
• PyTorch: 2.5.1+cu121
• Accelerate: 1.14.0
• Datasets: 5.0.0
• Tokenizers: 0.22.2

Citation

BibTeX

Sentence Transformers

@inproceedings{reimers-2019-sentence-bert,
    title = "Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks",
    author = "Reimers, Nils and Gurevych, Iryna",
    booktitle = "Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing",
    month = "11",
    year = "2019",
    publisher = "Association for Computational Linguistics",
    url = "https://arxiv.org/abs/1908.10084",
}

MultipleNegativesRankingLoss

@misc{oord2019representationlearningcontrastivepredictive,
      title={Representation Learning with Contrastive Predictive Coding},
      author={Aaron van den Oord and Yazhe Li and Oriol Vinyals},
      year={2019},
      eprint={1807.03748},
      archivePrefix={arXiv},
      primaryClass={cs.LG},
      url={https://arxiv.org/abs/1807.03748},
}