Hugging Face's logo

archisgore
/
bge-small-winetone

Sentence Similarity
sentence-transformers
Safetensors
bert
feature-extraction
Generated from Trainer
dataset_size:173708
loss:MultipleNegativesRankingLoss
text-embeddings-inference
Model card Files
xet
 Community

SentenceTransformer based on BAAI/bge-small-en-v1.5

This is a sentence-transformers model finetuned from BAAI/bge-small-en-v1.5. It maps sentences & paragraphs to a 384-dimensional dense vector space and can be used for retrieval.

Model Details

Model Description

  • Model Type: Sentence Transformer
  • Base model: BAAI/bge-small-en-v1.5
  • Maximum Sequence Length: 512 tokens
  • Output Dimensionality: 384 dimensions
  • Similarity Function: Cosine Similarity
  • Supported Modality: Text

Model Sources

Full Model Architecture 

SentenceTransformer(
  (0): Transformer({'transformer_task': 'feature-extraction', 'modality_config': {'text': {'method': 'forward', 'method_output_name': 'last_hidden_state'}}, 'module_output_name': 'token_embeddings', 'architecture': 'BertModel'})
  (1): Pooling({'embedding_dimension': 384, 'pooling_mode': 'cls', 'include_prompt': True})
  (2): Normalize({})
)

Usage

Direct Usage (Sentence Transformers)

First install the Sentence Transformers library:

pip install -U sentence-transformers

Then you can load this model and run inference.

from sentence_transformers import SentenceTransformer

# Download from the 🤗 Hub
model = SentenceTransformer("sentence_transformers_model_id")
# Run inference
sentences = [
    "Light cherry-red in color, this nuanced Pinot is bright in raspberry and tart grenadine, floral, with a spicy undercurrent of garden herb. Easy to like and approachable it's well-priced, high-quality and elegantly balanced.",
    'Made from a majority of Alexander Valley fruit, this is a smoky, chocolaty red wine, ripe in blackberry and an accent of clove. Rich, it remains fresh on the palate, with soft, supple tannins and an easygoing, lingering finish.',
    'A superfine Zin, lush, fruity and balanced, and with extra richness and finesse. Really shows off its ripe blackberries, cherries and mocha, in a dry package with smooth tannins.',
]
embeddings = model.encode(sentences)
print(embeddings.shape)
# [3, 384]

# Get the similarity scores for the embeddings
similarities = model.similarity(embeddings, embeddings)
print(similarities)
# tensor([[1.0000, 0.5768, 0.4416],
#         [0.5768, 1.0000, 0.5093],
#         [0.4416, 0.5093, 1.0000]])

Training Details

Training Dataset

Unnamed Dataset

  • Size: 173,708 training samples
  • Columns: sentence_0 and sentence_1
  • Approximate statistics based on the first 100 samples:
    sentence_0 sentence_1
    type string string
    modality text text
    details
    • min: 29 tokens
    • mean: 58.56 tokens
    • max: 131 tokens
    • min: 14 tokens
    • mean: 58.42 tokens
    • max: 131 tokens
  • Samples:
    sentence_0 sentence_1
    Ripe and easy to drink, this Cab is fully ready now. It shows a wealth of blackberry, cherry, currant, licorice, chocolate mint and vanilla flavors, wrapped into sweet, smooth tannins. A lush, flavorful Cabernet that offers plenty of richness. It floods the mouth with ripe blackberries, cherries, black currants and cassis, with hints of chocolate and mushu plum sauce. The thick tannins need a good steak to play against.
    Made in the sweet, crisp and fruity style that made Moscato catch on with so many consumers, this could double as a dessert sipper. It has flavors of orange meringue pie and vanilla honey. Considered as a dessert wine, this Moscato is O.K. It has orange and vanilla pastry flavors, brightened by acidity. Don't even think about drinking it as a dry table wine, because it's very sweet.
    Aromas of black cherry and green pepper lead into flavors of cherry, menthol, pickled jalapeño, oregano and lavender. Tannins mildly outweigh acidity. Although all the pieces of the puzzle are evident on the palate, somehow they don't fit together perfectly. Aromas of black cherry and green pepper lead into flavors of cherry, menthol, pickled jalapeño, oregano and lavender. Tannins mildly outweigh acidity. Although all the pieces of the puzzle are evident on the palate, somehow they don't fit together perfectly.
  • Loss: MultipleNegativesRankingLoss with these parameters:
    {
    "scale": 20.0,
    "similarity_fct": "cos_sim",
    "gather_across_devices": false,
    "directions": [
        "query_to_doc"
    ],
    "partition_mode": "joint",
    "hardness_mode": null,
    "hardness_strength": 0.0
}

Training Hyperparameters

Non-Default Hyperparameters

  • per_device_train_batch_size: 32
  • num_train_epochs: 1
  • per_device_eval_batch_size: 32
  • multi_dataset_batch_sampler: round_robin

All Hyperparameters

Click to expand
  • per_device_train_batch_size: 32
  • num_train_epochs: 1
  • max_steps: -1
  • learning_rate: 5e-05
  • lr_scheduler_type: linear
  • lr_scheduler_kwargs: None
  • warmup_steps: 0
  • optim: adamw_torch_fused
  • optim_args: None
  • weight_decay: 0.0
  • adam_beta1: 0.9
  • adam_beta2: 0.999
  • adam_epsilon: 1e-08
  • optim_target_modules: None
  • gradient_accumulation_steps: 1
  • average_tokens_across_devices: True
  • max_grad_norm: 1
  • label_smoothing_factor: 0.0
  • bf16: False
  • fp16: False
  • bf16_full_eval: False
  • fp16_full_eval: False
  • tf32: None
  • gradient_checkpointing: False
  • gradient_checkpointing_kwargs: None
  • torch_compile: False
  • torch_compile_backend: None
  • torch_compile_mode: None
  • use_liger_kernel: False
  • liger_kernel_config: None
  • use_cache: False
  • neftune_noise_alpha: None
  • torch_empty_cache_steps: None
  • auto_find_batch_size: False
  • log_on_each_node: True
  • logging_nan_inf_filter: True
  • include_num_input_tokens_seen: no
  • log_level: passive
  • log_level_replica: warning
  • disable_tqdm: False
  • project: huggingface
  • trackio_space_id: None
  • trackio_bucket_id: None
  • trackio_static_space_id: None
  • per_device_eval_batch_size: 32
  • prediction_loss_only: True
  • eval_on_start: False
  • eval_do_concat_batches: True
  • eval_use_gather_object: False
  • eval_accumulation_steps: None
  • include_for_metrics: []
  • batch_eval_metrics: False
  • save_only_model: False
  • save_on_each_node: False
  • enable_jit_checkpoint: False
  • push_to_hub: False
  • hub_private_repo: None
  • hub_model_id: None
  • hub_strategy: every_save
  • hub_always_push: False
  • hub_revision: None
  • load_best_model_at_end: False
  • ignore_data_skip: False
  • restore_callback_states_from_checkpoint: False
  • full_determinism: False
  • seed: 42
  • data_seed: None
  • use_cpu: False
  • accelerator_config: {'split_batches': False, 'dispatch_batches': None, 'even_batches': True, 'use_seedable_sampler': True, 'non_blocking': False, 'gradient_accumulation_kwargs': None}
  • parallelism_config: None
  • dataloader_drop_last: False
  • dataloader_num_workers: 0
  • dataloader_pin_memory: True
  • dataloader_persistent_workers: False
  • dataloader_prefetch_factor: None
  • remove_unused_columns: True
  • label_names: None
  • train_sampling_strategy: random
  • length_column_name: length
  • ddp_find_unused_parameters: None
  • ddp_bucket_cap_mb: None
  • ddp_broadcast_buffers: False
  • ddp_static_graph: None
  • ddp_backend: None
  • ddp_timeout: 1800
  • fsdp: []
  • fsdp_config: {'min_num_params': 0, 'xla': False, 'xla_fsdp_v2': False, 'xla_fsdp_grad_ckpt': False}
  • deepspeed: None
  • debug: []
  • skip_memory_metrics: True
  • do_predict: False
  • resume_from_checkpoint: None
  • warmup_ratio: None
  • local_rank: -1
  • prompts: None
  • batch_sampler: batch_sampler
  • multi_dataset_batch_sampler: round_robin
  • router_mapping: {}
  • learning_rate_mapping: {}

Training Logs

Epoch Step Training Loss
0.0921 500 1.8935
0.1842 1000 1.5796
0.2763 1500 1.4531
0.3684 2000 1.3640
0.4605 2500 1.3082
0.5526 3000 1.2474
0.6447 3500 1.2045
0.7368 4000 1.1782
0.8289 4500 1.1714
0.9210 5000 1.1460

Training Time

  • Training: 31.2 minutes

Framework Versions

  • Python: 3.14.4
  • Sentence Transformers: 5.5.1
  • Transformers: 5.9.0
  • PyTorch: 2.12.0
  • Accelerate: 1.13.0
  • Datasets: 4.8.5
  • Tokenizers: 0.22.2

Citation

BibTeX

Sentence Transformers 

@inproceedings{reimers-2019-sentence-bert,
    title = "Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks",
    author = "Reimers, Nils and Gurevych, Iryna",
    booktitle = "Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing",
    month = "11",
    year = "2019",
    publisher = "Association for Computational Linguistics",
    url = "https://arxiv.org/abs/1908.10084",
}

MultipleNegativesRankingLoss 

@misc{oord2019representationlearningcontrastivepredictive,
      title={Representation Learning with Contrastive Predictive Coding},
      author={Aaron van den Oord and Yazhe Li and Oriol Vinyals},
      year={2019},
      eprint={1807.03748},
      archivePrefix={arXiv},
      primaryClass={cs.LG},
      url={https://arxiv.org/abs/1807.03748},
}
Downloads last month
92
Safetensors
Model size
33.4M params
Tensor type
F32
·
Inference Providers NEW
Sentence Similarity
This model isn't deployed by any Inference Provider. 🙋 Ask for provider support

Model tree for archisgore/bge-small-winetone

Base model

BAAI/bge-small-en-v1.5
Finetuned
(551)
this model

Papers for archisgore/bge-small-winetone