SentenceTransformer based on sentence-transformers/LaBSE

This is a sentence-transformers model finetuned from sentence-transformers/LaBSE. It maps inputs to a 768-dimensional dense vector space and can be used for semantic textual similarity, semantic search, paraphrase mining, classification, clustering, and more.

Model Details

Model Description

  • Model Type: Sentence Transformer
  • Base model: sentence-transformers/LaBSE
  • Maximum Sequence Length: 256 tokens
  • Output Dimensionality: 768 dimensions
  • Similarity Function: Cosine Similarity
  • Supported Modality: Text

Model Sources

Full Model Architecture

SentenceTransformer(
  (0): Transformer({'transformer_task': 'feature-extraction', 'modality_config': {'text': {'method': 'forward', 'method_output_name': 'last_hidden_state'}}, 'module_output_name': 'token_embeddings', 'architecture': 'BertModel'})
  (1): Pooling({'embedding_dimension': 768, 'pooling_mode': 'cls', 'include_prompt': True})
  (2): Dense({'in_features': 768, 'out_features': 768, 'bias': True, 'activation_function': 'torch.nn.modules.activation.Tanh', 'module_input_name': 'sentence_embedding', 'module_output_name': 'sentence_embedding'})
  (3): Normalize({'module_input_name': 'sentence_embedding', 'module_output_name': 'sentence_embedding'})
)

Usage

Direct Usage (Sentence Transformers)

First install the Sentence Transformers library:

pip install -U sentence-transformers

Then you can load this model and run inference.

from sentence_transformers import SentenceTransformer

# Download from the 🤗 Hub
model = SentenceTransformer("sentence_transformers_model_id")
# Run inference
queries = [
    'Stag | 122 Bruce Street, Scotia, NY',
]
documents = [
    'Stag Corporation | BRUCE STREET, SCOTIA, NY',
    '#2 HUB CO | 142 COTTAGE STREET, LEWISTON, ME',
    'ORTHOPEDIC  PHYSICIANS | 75, ALTOONA, AL',
]
query_embeddings = model.encode_query(queries)
document_embeddings = model.encode_document(documents)
print(query_embeddings.shape, document_embeddings.shape)
# [1, 768] [3, 768]

# Get the similarity scores for the embeddings
similarities = model.similarity(query_embeddings, document_embeddings)
print(similarities)
# tensor([[ 0.7677,  0.1634, -0.0688]])

Evaluation

Metrics

Information Retrieval

Metric Value
cosine_accuracy@1 0.929
cosine_accuracy@5 1.0
cosine_accuracy@10 1.0
cosine_accuracy@30 1.0
cosine_precision@1 0.929
cosine_precision@3 0.3333
cosine_precision@5 0.2
cosine_precision@10 0.1
cosine_recall@1 0.929
cosine_recall@3 1.0
cosine_recall@5 1.0
cosine_recall@10 1.0
cosine_ndcg@10 0.9733
cosine_mrr@10 0.9638
cosine_map@100 0.9638

Training Details

Training Dataset

Unnamed Dataset

  • Size: 623,829 training samples
  • Columns: anchor and positive
  • Approximate statistics based on the first 100 samples:
    anchor positive
    type string string
    modality text text
    details
    • min: 18 tokens
    • mean: 31.59 tokens
    • max: 55 tokens
    • min: 7 tokens
    • mean: 33.45 tokens
    • max: 64 tokens
  • Samples:
    anchor positive
    Bangalore South Automotive Private Limited | C-901, Goyalco Orchid, Lakeview, Bangalore South, Bangalore, Karnataka Bangalore South Automotive Limited Private | C-901., GOYALCO ORCHID, LAKEVIEW, BANGALORE SOUTH, ಕರ್ನಾಟಕ
    Bangalore South Automotive Private Limited | C-901, Goyalco Orchid, Lakeview, Bangalore South, Bangalore, Karnataka Bangalore Sóuth Automotive Private Limited |
    Software Pinnacle Tourism | 3016/5, 1St Floor, Ranjit Nagar, New Delhi, Central Delhi, Delhi Software Pinnacle Tóurism | A-3016/5, 1ST FLOOR, RANJIT NAGAR, NEW DELHI, दिल्ली
  • Loss: MultipleNegativesRankingLoss with these parameters:
    {
        "scale": 20.0,
        "similarity_fct": "cos_sim",
        "gather_across_devices": false,
        "directions": [
            "query_to_doc"
        ],
        "partition_mode": "joint",
        "hardness_mode": null,
        "hardness_strength": 0.0
    }
    

Training Hyperparameters

Non-Default Hyperparameters

  • per_device_train_batch_size: 64
  • num_train_epochs: 2
  • learning_rate: 2e-05
  • warmup_steps: 0.1

All Hyperparameters

Click to expand
  • per_device_train_batch_size: 64
  • num_train_epochs: 2
  • max_steps: -1
  • learning_rate: 2e-05
  • lr_scheduler_type: linear
  • lr_scheduler_kwargs: None
  • warmup_steps: 0.1
  • optim: adamw_torch_fused
  • optim_args: None
  • weight_decay: 0.0
  • adam_beta1: 0.9
  • adam_beta2: 0.999
  • adam_epsilon: 1e-08
  • optim_target_modules: None
  • gradient_accumulation_steps: 1
  • average_tokens_across_devices: True
  • max_grad_norm: 1.0
  • label_smoothing_factor: 0.0
  • bf16: False
  • fp16: False
  • bf16_full_eval: False
  • fp16_full_eval: False
  • tf32: None
  • gradient_checkpointing: False
  • gradient_checkpointing_kwargs: None
  • torch_compile: False
  • torch_compile_backend: None
  • torch_compile_mode: None
  • use_liger_kernel: False
  • liger_kernel_config: None
  • use_cache: False
  • neftune_noise_alpha: None
  • torch_empty_cache_steps: None
  • auto_find_batch_size: False
  • log_on_each_node: True
  • logging_nan_inf_filter: True
  • include_num_input_tokens_seen: no
  • log_level: passive
  • log_level_replica: warning
  • disable_tqdm: False
  • project: huggingface
  • trackio_space_id: None
  • trackio_bucket_id: None
  • trackio_static_space_id: None
  • per_device_eval_batch_size: 8
  • prediction_loss_only: True
  • eval_on_start: False
  • eval_do_concat_batches: True
  • eval_use_gather_object: False
  • eval_accumulation_steps: None
  • include_for_metrics: []
  • batch_eval_metrics: False
  • save_only_model: False
  • save_on_each_node: False
  • enable_jit_checkpoint: False
  • push_to_hub: False
  • hub_private_repo: None
  • hub_model_id: None
  • hub_strategy: every_save
  • hub_always_push: False
  • hub_revision: None
  • load_best_model_at_end: False
  • ignore_data_skip: False
  • restore_callback_states_from_checkpoint: False
  • full_determinism: False
  • seed: 42
  • data_seed: None
  • use_cpu: False
  • accelerator_config: {'split_batches': False, 'dispatch_batches': None, 'even_batches': True, 'use_seedable_sampler': True, 'non_blocking': False, 'gradient_accumulation_kwargs': None}
  • parallelism_config: None
  • dataloader_drop_last: False
  • dataloader_num_workers: 0
  • dataloader_pin_memory: True
  • dataloader_persistent_workers: False
  • dataloader_prefetch_factor: None
  • dataloader_multiprocessing_context: None
  • dataloader_in_order: True
  • remove_unused_columns: True
  • label_names: None
  • train_sampling_strategy: random
  • length_column_name: length
  • ddp_find_unused_parameters: None
  • ddp_bucket_cap_mb: None
  • ddp_broadcast_buffers: False
  • ddp_static_graph: None
  • ddp_backend: None
  • ddp_timeout: 1800
  • fsdp: None
  • fsdp_config: None
  • deepspeed: None
  • debug: []
  • skip_memory_metrics: True
  • do_predict: False
  • resume_from_checkpoint: None
  • local_rank: -1
  • prompts: None
  • batch_sampler: batch_sampler
  • multi_dataset_batch_sampler: proportional
  • router_mapping: {}
  • learning_rate_mapping: {}
  • warmup_ratio: None

Training Logs

Click to expand
Epoch Step Training Loss biencoder-val_cosine_ndcg@10
0.0051 50 0.2099 -
0.0103 100 0.0801 -
0.0154 150 0.0250 -
0.0205 200 0.0196 -
0.0256 250 0.0155 -
0.0308 300 0.0151 -
0.0359 350 0.0150 -
0.0410 400 0.0117 -
0.0462 450 0.0124 -
0.0513 500 0.0111 -
0.0564 550 0.0070 -
0.0616 600 0.0107 -
0.0667 650 0.0091 -
0.0718 700 0.0060 -
0.0769 750 0.0094 -
0.0821 800 0.0067 -
0.0872 850 0.0061 -
0.0923 900 0.0076 -
0.0975 950 0.0060 -
0.1026 1000 0.0047 -
0.1077 1050 0.0071 -
0.1128 1100 0.0063 -
0.1180 1150 0.0046 -
0.1231 1200 0.0039 -
0.1282 1250 0.0049 -
0.1334 1300 0.0055 -
0.1385 1350 0.0038 -
0.1436 1400 0.0041 -
0.1487 1450 0.0060 -
0.1539 1500 0.0055 -
0.1590 1550 0.0053 -
0.1641 1600 0.0047 -
0.1693 1650 0.0044 -
0.1744 1700 0.0071 -
0.1795 1750 0.0044 -
0.1847 1800 0.0025 -
0.1898 1850 0.0055 -
0.1949 1900 0.0053 -
0.2000 1950 0.0045 -
0.2052 2000 0.0047 -
0.2103 2050 0.0048 -
0.2154 2100 0.0041 -
0.2206 2150 0.0030 -
0.2257 2200 0.0037 -
0.2308 2250 0.0047 -
0.2359 2300 0.0042 -
0.2411 2350 0.0027 -
0.2462 2400 0.0047 -
0.2513 2450 0.0028 -
0.2565 2500 0.0043 -
0.2616 2550 0.0063 -
0.2667 2600 0.0027 -
0.2719 2650 0.0021 -
0.2770 2700 0.0028 -
0.2821 2750 0.0024 -
0.2872 2800 0.0045 -
0.2924 2850 0.0033 -
0.2975 2900 0.0048 -
0.3026 2950 0.0021 -
0.3078 3000 0.0031 -
0.3129 3050 0.0027 -
0.3180 3100 0.0033 -
0.3231 3150 0.0040 -
0.3283 3200 0.0012 -
0.3334 3250 0.0031 -
0.3385 3300 0.0033 -
0.3437 3350 0.0020 -
0.3488 3400 0.0028 -
0.3539 3450 0.0022 -
0.3590 3500 0.0025 -
0.3642 3550 0.0026 -
0.3693 3600 0.0028 -
0.3744 3650 0.0035 -
0.3796 3700 0.0023 -
0.3847 3750 0.0029 -
0.3898 3800 0.0031 -
0.3950 3850 0.0042 -
0.4001 3900 0.0027 -
0.4052 3950 0.0035 -
0.4103 4000 0.0026 -
0.4155 4050 0.0017 -
0.4206 4100 0.0022 -
0.4257 4150 0.0020 -
0.4309 4200 0.0025 -
0.4360 4250 0.0024 -
0.4411 4300 0.0025 -
0.4462 4350 0.0025 -
0.4514 4400 0.0021 -
0.4565 4450 0.0011 -
0.4616 4500 0.0018 -
0.4668 4550 0.0039 -
0.4719 4600 0.0016 -
0.4770 4650 0.0023 -
0.4822 4700 0.0012 -
0.4873 4750 0.0027 -
0.4924 4800 0.0020 -
0.4975 4850 0.0022 -
0.5027 4900 0.0022 -
0.5078 4950 0.0016 -
0.5129 5000 0.0027 -
0.5181 5050 0.0017 -
0.5232 5100 0.0020 -
0.5283 5150 0.0024 -
0.5334 5200 0.0024 -
0.5386 5250 0.0024 -
0.5437 5300 0.0022 -
0.5488 5350 0.0029 -
0.5540 5400 0.0016 -
0.5591 5450 0.0010 -
0.5642 5500 0.0017 -
0.5693 5550 0.0029 -
0.5745 5600 0.0017 -
0.5796 5650 0.0021 -
0.5847 5700 0.0019 -
0.5899 5750 0.0021 -
0.5950 5800 0.0030 -
0.6001 5850 0.0015 -
0.6053 5900 0.0028 -
0.6104 5950 0.0022 -
0.6155 6000 0.0020 -
0.6206 6050 0.0021 -
0.6258 6100 0.0017 -
0.6309 6150 0.0021 -
0.6360 6200 0.0015 -
0.6412 6250 0.0010 -
0.6463 6300 0.0018 -
0.6514 6350 0.0011 -
0.6565 6400 0.0035 -
0.6617 6450 0.0015 -
0.6668 6500 0.0022 -
0.6719 6550 0.0019 -
0.6771 6600 0.0017 -
0.6822 6650 0.0011 -
0.6873 6700 0.0021 -
0.6924 6750 0.0017 -
0.6976 6800 0.0014 -
0.7027 6850 0.0024 -
0.7078 6900 0.0012 -
0.7130 6950 0.0032 -
0.7181 7000 0.0020 -
0.7232 7050 0.0013 -
0.7284 7100 0.0038 -
0.7335 7150 0.0018 -
0.7386 7200 0.0019 -
0.7437 7250 0.0038 -
0.7489 7300 0.0013 -
0.7540 7350 0.0025 -
0.7591 7400 0.0022 -
0.7643 7450 0.0014 -
0.7694 7500 0.0015 -
0.7745 7550 0.0030 -
0.7796 7600 0.0016 -
0.7848 7650 0.0017 -
0.7899 7700 0.0013 -
0.7950 7750 0.0036 -
0.8002 7800 0.0022 -
0.8053 7850 0.0015 -
0.8104 7900 0.0010 -
0.8156 7950 0.0021 -
0.8207 8000 0.0019 -
0.8258 8050 0.0011 -
0.8309 8100 0.0012 -
0.8361 8150 0.0026 -
0.8412 8200 0.0011 -
0.8463 8250 0.0019 -
0.8515 8300 0.0017 -
0.8566 8350 0.0011 -
0.8617 8400 0.0012 -
0.8668 8450 0.0018 -
0.8720 8500 0.0008 -
0.8771 8550 0.0017 -
0.8822 8600 0.0023 -
0.8874 8650 0.0010 -
0.8925 8700 0.0011 -
0.8976 8750 0.0022 -
0.9027 8800 0.0023 -
0.9079 8850 0.0013 -
0.9130 8900 0.0015 -
0.9181 8950 0.0034 -
0.9233 9000 0.0011 -
0.9284 9050 0.0018 -
0.9335 9100 0.0013 -
0.9387 9150 0.0009 -
0.9438 9200 0.0008 -
0.9489 9250 0.0013 -
0.9540 9300 0.0012 -
0.9592 9350 0.0012 -
0.9643 9400 0.0008 -
0.9694 9450 0.0011 -
0.9746 9500 0.0008 -
0.9797 9550 0.0015 -
0.9848 9600 0.0014 -
0.9899 9650 0.0013 -
0.9951 9700 0.0006 -
1.0 9748 - 0.9733
1.0002 9750 0.0014 -
1.0053 9800 0.0006 -
1.0105 9850 0.0005 -
1.0156 9900 0.0011 -
1.0207 9950 0.0011 -
1.0259 10000 0.0009 -
1.0310 10050 0.0013 -
1.0361 10100 0.0008 -
1.0412 10150 0.0008 -
1.0464 10200 0.0006 -
1.0515 10250 0.0006 -
1.0566 10300 0.0010 -
1.0618 10350 0.0026 -
1.0669 10400 0.0007 -
1.0720 10450 0.0007 -
1.0771 10500 0.0010 -
1.0823 10550 0.0016 -
1.0874 10600 0.0014 -
1.0925 10650 0.0007 -
1.0977 10700 0.0006 -
1.1028 10750 0.0021 -
1.1079 10800 0.0027 -
1.1130 10850 0.0006 -
1.1182 10900 0.0006 -
1.1233 10950 0.0012 -
1.1284 11000 0.0016 -
1.1336 11050 0.0015 -
1.1387 11100 0.0026 -
1.1438 11150 0.0008 -
1.1490 11200 0.0007 -
1.1541 11250 0.0007 -
1.1592 11300 0.0012 -
1.1643 11350 0.0010 -
1.1695 11400 0.0005 -
1.1746 11450 0.0008 -
1.1797 11500 0.0006 -
1.1849 11550 0.0005 -
1.1900 11600 0.0016 -
1.1951 11650 0.0009 -
1.2002 11700 0.0011 -
1.2054 11750 0.0006 -
1.2105 11800 0.0014 -
1.2156 11850 0.0006 -
1.2208 11900 0.0006 -
1.2259 11950 0.0015 -
1.2310 12000 0.0005 -
1.2362 12050 0.0037 -
1.2413 12100 0.0006 -
1.2464 12150 0.0005 -
1.2515 12200 0.0007 -
1.2567 12250 0.0016 -
1.2618 12300 0.0006 -
1.2669 12350 0.0015 -
1.2721 12400 0.0005 -
1.2772 12450 0.0007 -
1.2823 12500 0.0006 -
1.2874 12550 0.0009 -
1.2926 12600 0.0005 -
1.2977 12650 0.0004 -
1.3028 12700 0.0011 -
1.3080 12750 0.0006 -
1.3131 12800 0.0006 -
1.3182 12850 0.0006 -
1.3233 12900 0.0004 -
1.3285 12950 0.0015 -
1.3336 13000 0.0013 -
1.3387 13050 0.0008 -
1.3439 13100 0.0010 -
1.3490 13150 0.0007 -
1.3541 13200 0.0013 -
1.3593 13250 0.0005 -
1.3644 13300 0.0017 -
1.3695 13350 0.0020 -
1.3746 13400 0.0004 -
1.3798 13450 0.0004 -
1.3849 13500 0.0009 -
1.3900 13550 0.0004 -
1.3952 13600 0.0009 -
1.4003 13650 0.0004 -
1.4054 13700 0.0007 -
1.4105 13750 0.0013 -
1.4157 13800 0.0007 -
1.4208 13850 0.0010 -
1.4259 13900 0.0010 -
1.4311 13950 0.0008 -
1.4362 14000 0.0009 -
1.4413 14050 0.0015 -
1.4465 14100 0.0008 -
1.4516 14150 0.0007 -
1.4567 14200 0.0007 -
1.4618 14250 0.0011 -
1.4670 14300 0.0023 -
1.4721 14350 0.0011 -
1.4772 14400 0.0004 -
1.4824 14450 0.0009 -
1.4875 14500 0.0005 -
1.4926 14550 0.0009 -
1.4977 14600 0.0004 -
1.5029 14650 0.0004 -
1.5080 14700 0.0004 -
1.5131 14750 0.0011 -
1.5183 14800 0.0020 -
1.5234 14850 0.0007 -
1.5285 14900 0.0005 -
1.5336 14950 0.0004 -
1.5388 15000 0.0004 -
1.5439 15050 0.0009 -
1.5490 15100 0.0005 -
1.5542 15150 0.0006 -
1.5593 15200 0.0008 -
1.5644 15250 0.0006 -
1.5696 15300 0.0023 -
1.5747 15350 0.0006 -
1.5798 15400 0.0006 -
1.5849 15450 0.0013 -
1.5901 15500 0.0006 -
1.5952 15550 0.0004 -
1.6003 15600 0.0020 -
1.6055 15650 0.0016 -
1.6106 15700 0.0005 -
1.6157 15750 0.0004 -
1.6208 15800 0.0011 -
1.6260 15850 0.0014 -
1.6311 15900 0.0005 -
1.6362 15950 0.0011 -
1.6414 16000 0.0007 -
1.6465 16050 0.0005 -
1.6516 16100 0.0008 -
1.6568 16150 0.0003 -
1.6619 16200 0.0012 -
1.6670 16250 0.0004 -
1.6721 16300 0.0003 -
1.6773 16350 0.0003 -
1.6824 16400 0.0022 -
1.6875 16450 0.0004 -
1.6927 16500 0.0005 -
1.6978 16550 0.0004 -
1.7029 16600 0.0006 -
1.7080 16650 0.0007 -
1.7132 16700 0.0009 -
1.7183 16750 0.0004 -
1.7234 16800 0.0014 -
1.7286 16850 0.0012 -
1.7337 16900 0.0005 -
1.7388 16950 0.0008 -
1.7439 17000 0.0006 -
1.7491 17050 0.0007 -
1.7542 17100 0.0004 -
1.7593 17150 0.0012 -
1.7645 17200 0.0013 -
1.7696 17250 0.0010 -
1.7747 17300 0.0009 -
1.7799 17350 0.0004 -
1.7850 17400 0.0003 -
1.7901 17450 0.0007 -
1.7952 17500 0.0004 -
1.8004 17550 0.0004 -
1.8055 17600 0.0005 -
1.8106 17650 0.0004 -
1.8158 17700 0.0003 -
1.8209 17750 0.0004 -
1.8260 17800 0.0014 -
1.8311 17850 0.0004 -
1.8363 17900 0.0008 -
1.8414 17950 0.0004 -
1.8465 18000 0.0004 -
1.8517 18050 0.0007 -
1.8568 18100 0.0003 -
1.8619 18150 0.0004 -
1.8670 18200 0.0019 -
1.8722 18250 0.0005 -
1.8773 18300 0.0007 -
1.8824 18350 0.0004 -
1.8876 18400 0.0014 -
1.8927 18450 0.0006 -
1.8978 18500 0.0004 -
1.9030 18550 0.0004 -
1.9081 18600 0.0003 -
1.9132 18650 0.0006 -
1.9183 18700 0.0008 -
1.9235 18750 0.0005 -
1.9286 18800 0.0003 -
1.9337 18850 0.0003 -
1.9389 18900 0.0003 -
1.9440 18950 0.0007 -
1.9491 19000 0.0003 -
1.9542 19050 0.0008 -
1.9594 19100 0.0007 -
1.9645 19150 0.0004 -
1.9696 19200 0.0004 -
1.9748 19250 0.0014 -
1.9799 19300 0.0009 -
1.9850 19350 0.0004 -
1.9902 19400 0.0012 -
1.9953 19450 0.0010 -
2.0 19496 - 0.9733

Training Time

  • Training: 1.0 hours
  • Evaluation: 1.8 seconds
  • Total: 1.0 hours

Framework Versions

  • Python: 3.10.12
  • Sentence Transformers: 6.1.0
  • Transformers: 5.17.0
  • PyTorch: 2.14.0+cu130
  • Accelerate: 1.15.0
  • Datasets: 5.0.1
  • Tokenizers: 0.23.2

Additional Resources

Citation

BibTeX

Sentence Transformers

@inproceedings{reimers-2019-sentence-bert,
    title = "Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks",
    author = "Reimers, Nils and Gurevych, Iryna",
    booktitle = "Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing",
    month = "11",
    year = "2019",
    publisher = "Association for Computational Linguistics",
    url = "https://arxiv.org/abs/1908.10084",
}

MultipleNegativesRankingLoss

@misc{oord2019representationlearningcontrastivepredictive,
      title={Representation Learning with Contrastive Predictive Coding},
      author={Aaron van den Oord and Yazhe Li and Oriol Vinyals},
      year={2019},
      eprint={1807.03748},
      archivePrefix={arXiv},
      primaryClass={cs.LG},
      url={https://arxiv.org/abs/1807.03748},
}
Downloads last month
17
Safetensors
Model size
0.5B params
Tensor type
F32
·
Inference Providers NEW
This model isn't deployed by any Inference Provider. 🙋 Ask for provider support

Model tree for Starkdoorstep12/entity-resolution-biencoder

Finetuned
(101)
this model

Papers for Starkdoorstep12/entity-resolution-biencoder

Evaluation results