vaktibabat/heart-e5

StrictSentenceTransformer based on intfloat/e5-base-v2

This is a sentence-transformers model finetuned from intfloat/e5-base-v2 on the bge-multilingual-gemma2-data dataset. It maps sentences & paragraphs to a 768-dimensional dense vector space and can be used for semantic textual similarity, semantic search, paraphrase mining, text classification, clustering, and more.

Model Details

Model Description

• Model Type: Sentence Transformer
• Base model: intfloat/e5-base-v2
• Maximum Sequence Length: 512 tokens
• Output Dimensionality: 768 dimensions
• Similarity Function: Cosine Similarity
• Training Dataset:
  • bge-multilingual-gemma2-data

Model Sources

• Documentation: Sentence Transformers Documentation
• Repository: Sentence Transformers on GitHub
• Hugging Face: Sentence Transformers on Hugging Face

Full Model Architecture

StrictSentenceTransformer(
  (0): Transformer({'max_seq_length': 512, 'do_lower_case': False, 'architecture': 'BertModel'})
  (1): Pooling({'word_embedding_dimension': 768, 'pooling_mode_cls_token': False, 'pooling_mode_mean_tokens': True, 'pooling_mode_max_tokens': False, 'pooling_mode_mean_sqrt_len_tokens': False, 'pooling_mode_weightedmean_tokens': False, 'pooling_mode_lasttoken': False, 'include_prompt': True})
)

Usage

Direct Usage (Sentence Transformers)

First install the Sentence Transformers library:

pip install -U sentence-transformers

Then you can load this model and run inference.

from sentence_transformers import SentenceTransformer

# Download from the 🤗 Hub
model = SentenceTransformer("vaktibabat/heart-e5")
# Run inference
sentences = [
    'query: calories in chocolate covered pretzel',
    'passage: Chocolate Covered Pretzels There are 140 calories in a 2 pretzels serving of Sarris Candies Chocolate Covered Pretzels. Calorie breakdown: 39% fat, 55% carbs, 6% protein.',
    'passage: Calories In Doughnut - Glazed, Dunking, Cake, Chocolate The average amount of calories in a chocolate doughnut is approximately 340 calories. In addition to its high calorie content, chocolate doughnuts are also high in fats and processed sugars. Make sure that you check the nutritional breakdown of a chocolate doughnut box before you dig in!',
]
embeddings = model.encode(sentences)
print(embeddings.shape)
# [3, 768]

# Get the similarity scores for the embeddings
similarities = model.similarity(embeddings, embeddings)
print(similarities)
# tensor([[1.0000, 0.9357, 0.8240],
#         [0.9357, 1.0000, 0.9000],
#         [0.8240, 0.9000, 1.0000]])

Evaluation

Metrics

My Information Retrieval

• Datasets: train_subset and dev_subset
• Evaluated with training.train_utils.my_sentence_transformers.MyInformationRetrievalEvaluator.MyInformationRetrievalEvaluator

Metric	train_subset	dev_subset
cosine_ndcg@10	0.9675	0.9806
cosine_mrr@10	0.9568	0.9741

Training Details

Training Dataset

bge-multilingual-gemma2-data

• Dataset: bge-multilingual-gemma2-data at ef165e1
• Size: 1,534,495 training samples
• Columns: anchor, positive, and negative
• Approximate statistics based on the first 1000 samples:

  	anchor	positive	negative
  type	string	string	string
  details	min: 6 tokens mean: 10.85 tokens max: 36 tokens	min: 21 tokens mean: 85.96 tokens max: 206 tokens	min: 21 tokens mean: 86.98 tokens max: 237 tokens

• Samples:

  anchor	positive	negative
  query: lee price wrestler	passage: - April 15, 2017. Lee Price Star Wrestler. Lee Price, the blonde 1990’s female wrestling Icon was fun to watch and seemed to be even more enjoyable to be around. As we all know, people who take life too seriously are no fun to be around especially if they are family or friends.	passage: - JACKSONVILLE, Fla. – William Donovan Lee, 42, was found guilty as charged Thursday afternoon of two counts of Attempted Murder in the First Degree, one count of Shooting or Throwing Deadly Missiles, and one count of Tampering with….
  query: what type of soil are crops grown	passage: Types of Soils Soil types according to depth are as follows: 1) Shallow Soil - Soil depth less than 22.5cm. Only shallow rooted crops are grown in such soil, e.g. Paddy, Nagli. 2) Medium deep soil - Soil depth is 22.5 to 45cm. Crops with medium deep roots are grown in this type of soil e.g. Sugar cane, Banana, Gram. 3) Deep soil - Soil depth is more than 45cm. Crops with long and deep roots are grown in this type a soil e.g.	passage: - Cover crops usually are grown to prevent soil loss from wind and water erosion. Use fast-growing cover crops, such as winter wheat or annual rye, on fall-spaded gardens. A second, and probably more important reason home gardeners should use cover crops is to improve soil structure and increase organic matter.
  query: how long does lisinopril stay in your body	passage: How long does it take for lisonopril to get out of your body after stopping dose? Responses (1) The plasma half-life of lisinopril is approximately 12 hours. This means that in the average person, lisinopril should disappear from the body within 3-4 days. However, since some of the effects of the drug last longer than that, the effects of this drug can last up to 1-2 weeks after stopping the drug. Take care.	passage: Top 30 Doctor insights on: How Long Does Lisinopril Stay In Your System Not sure if: your symptoms are side effects of your new medication. It usually takes no more than 1 or 2 days for any drug to clear from your body, but it is a gradual process. Other potential problematic scenariors: Drug metabolites can hang around for longer, tissue drug levels can sometimes take longer than blood levels, any damage that it might have caused can take longer to heal. Consult your doctor. ...Read more.

• Loss: training.train_utils.my_sentence_transformers.MyCachedMultipleNegativesRankingLoss.MyCachedMultipleNegativesRankingLoss with these parameters:

  {
      "scale": 100.0,
      "similarity_fct": "cos_sim",
      "mini_batch_size": 96,
      "gather_across_devices": false
  }
  

Evaluation Dataset

bge-multilingual-gemma2-data

• Dataset: bge-multilingual-gemma2-data at ef165e1
• Size: 7,712 evaluation samples
• Columns: anchor, positive, and negative
• Approximate statistics based on the first 1000 samples:

  	anchor	positive	negative
  type	string	string	string
  details	min: 6 tokens mean: 10.99 tokens max: 32 tokens	min: 25 tokens mean: 87.85 tokens max: 235 tokens	min: 22 tokens mean: 87.77 tokens max: 205 tokens

• Samples:

  anchor	positive	negative
  query: what is dornase alfa meaning	passage: - Dornase alfa is a biosynthetic form of human DNase I. The enzyme is involved in endonucleolytic cleavage of extracellular DNA to 5´-phosphodinucleotide and 5´-phosphooligonucleotide end products.It has no effect on intracellular DNA.tudies in rats indicate that, following aerosol administration, the disappearance half-life of dornase alfa from the lungs is 11 hours. In humans, sputum DNase levels declined below half of those detected immediately post-administration within 2 hours but effects on sputum rheology persisted beyond 12 hours.	passage: âDead on arrivalâ DOA or dead on arrival traditionally is the official terminology used by the trauma center where a victim is received. Trauma centers (emergency rooms) are where the official pronouncement (declaration) of death is made by the attending physician, and it is pronounced at the time the victim arrives at the facility.
  query: what is a domain name?	passage: Domain name A domain name is an identification string that defines a realm of administrative autonomy, authority or control within the Internet. Domain names are formed by the rules and procedures of the Domain Name System (DNS). Any name registered in the DNS is a domain name.Domain names can also be thought of as a location where certain information or activities can be found. Domain names are used in various networking contexts and application-specific naming and addressing purposes. fictitious domain name is a domain name used in a work of fiction or popular culture to refer to a domain that does not actually exist, often with invalid or unofficial top-level domains such as .web , a usage exactly analogous to the dummy 555 telephone number prefix used in film and other media.	passage: - The Internet Domain Name System (DNS) is the Internet’s hierarchical address. system that helps users find targeted webpages. The DNS functions like a telephone. directory for the Internet; a domain name is the user -friendly form of a webpage’s. “phone number,” which is called the Internet Protocol (IP) address.
  query: harmful effects of inflammation	passage: Doctor speaks on health effects of chronic inflammation Doctor speaks on health effects of chronic inflammation. If your body is in a chronic state of inflammation, it can have serious effects on your cellular health, and has been linked to degenerative diseases including cancer, heart disease, diabetes, Alzheimer's and many others.	passage: 10 Top Foods That Prevent Inflammation in Your Body And if you want to get or remain healthy, you definitely want to reduce the damaging effects of it! Inflammation has a positive and negative affect in your body. Inflammation has a positive side because it helps your body respond to stress. But chronic low-grade inflammation is thought to be one of the leading causes of disease, premature aging and illness. When you get a cold, your body responds with inflammation in the form of a fever that helps you heal.

• Loss: training.train_utils.my_sentence_transformers.MyCachedMultipleNegativesRankingLoss.MyCachedMultipleNegativesRankingLoss with these parameters:

  {
      "scale": 100.0,
      "similarity_fct": "cos_sim",
      "mini_batch_size": 96,
      "gather_across_devices": false
  }
  

Training Hyperparameters

Non-Default Hyperparameters

• eval_strategy: steps
• per_device_train_batch_size: 256
• learning_rate: 2e-05
• weight_decay: 0.01
• num_train_epochs: 1
• warmup_ratio: 0.1
• bf16: True
• dataloader_num_workers: 15
• load_best_model_at_end: True
• gradient_checkpointing: True
• eval_on_start: True

All Hyperparameters

Click to expand

• overwrite_output_dir: False
• do_predict: False
• eval_strategy: steps
• prediction_loss_only: True
• per_device_train_batch_size: 256
• per_device_eval_batch_size: 8
• per_gpu_train_batch_size: None
• per_gpu_eval_batch_size: None
• gradient_accumulation_steps: 1
• eval_accumulation_steps: None
• torch_empty_cache_steps: None
• learning_rate: 2e-05
• weight_decay: 0.01
• adam_beta1: 0.9
• adam_beta2: 0.999
• adam_epsilon: 1e-08
• max_grad_norm: 1.0
• num_train_epochs: 1
• max_steps: -1
• lr_scheduler_type: linear
• lr_scheduler_kwargs: {}
• warmup_ratio: 0.1
• warmup_steps: 0
• log_level: passive
• log_level_replica: warning
• log_on_each_node: True
• logging_nan_inf_filter: True
• save_safetensors: True
• save_on_each_node: False
• save_only_model: False
• restore_callback_states_from_checkpoint: False
• no_cuda: False
• use_cpu: False
• use_mps_device: False
• seed: 42
• data_seed: None
• jit_mode_eval: False
• bf16: True
• fp16: False
• fp16_opt_level: O1
• half_precision_backend: auto
• bf16_full_eval: False
• fp16_full_eval: False
• tf32: None
• local_rank: 0
• ddp_backend: None
• tpu_num_cores: None
• tpu_metrics_debug: False
• debug: []
• dataloader_drop_last: False
• dataloader_num_workers: 15
• dataloader_prefetch_factor: None
• past_index: -1
• disable_tqdm: True
• remove_unused_columns: True
• label_names: None
• load_best_model_at_end: True
• ignore_data_skip: False
• fsdp: []
• fsdp_min_num_params: 0
• fsdp_config: {'min_num_params': 0, 'xla': False, 'xla_fsdp_v2': False, 'xla_fsdp_grad_ckpt': False}
• fsdp_transformer_layer_cls_to_wrap: None
• accelerator_config: {'split_batches': False, 'dispatch_batches': None, 'even_batches': True, 'use_seedable_sampler': True, 'non_blocking': False, 'gradient_accumulation_kwargs': None}
• parallelism_config: None
• deepspeed: None
• label_smoothing_factor: 0.0
• optim: adamw_torch_fused
• optim_args: None
• adafactor: False
• group_by_length: False
• length_column_name: length
• project: huggingface
• trackio_space_id: trackio
• ddp_find_unused_parameters: None
• ddp_bucket_cap_mb: None
• ddp_broadcast_buffers: False
• dataloader_pin_memory: True
• dataloader_persistent_workers: False
• skip_memory_metrics: True
• use_legacy_prediction_loop: False
• push_to_hub: False
• resume_from_checkpoint: None
• hub_model_id: None
• hub_strategy: every_save
• hub_private_repo: None
• hub_always_push: False
• hub_revision: None
• gradient_checkpointing: True
• gradient_checkpointing_kwargs: None
• include_inputs_for_metrics: False
• include_for_metrics: []
• eval_do_concat_batches: True
• fp16_backend: auto
• push_to_hub_model_id: None
• push_to_hub_organization: None
• mp_parameters:
• auto_find_batch_size: False
• full_determinism: False
• torchdynamo: None
• ray_scope: last
• ddp_timeout: 1800
• torch_compile: False
• torch_compile_backend: None
• torch_compile_mode: None
• include_tokens_per_second: False
• include_num_input_tokens_seen: no
• neftune_noise_alpha: None
• optim_target_modules: None
• batch_eval_metrics: False
• eval_on_start: True
• use_liger_kernel: False
• liger_kernel_config: None
• eval_use_gather_object: False
• average_tokens_across_devices: True
• prompts: None
• multi_dataset_batch_sampler: proportional
• router_mapping: {}
• learning_rate_mapping: {}

Training Logs

Click to expand

Epoch	Step	Training Loss	Validation Loss	train_subset_cosine_ndcg@10	dev_subset_cosine_ndcg@10
0	0	-	0.6352	0.9680	0.9815
0.0002	1	2.6597	-	-	-
0.0017	10	2.7475	-	-	-
0.0033	20	3.0591	-	-	-
0.0050	30	1.9659	-	-	-
0.0067	40	2.401	-	-	-
0.0083	50	2.5353	0.3984	0.9679	0.9821
0.0100	60	1.8871	-	-	-
0.0117	70	2.299	-	-	-
0.0133	80	1.8011	-	-	-
0.0150	90	1.8796	-	-	-
0.0167	100	2.0824	0.2494	0.9653	0.9800
0.0184	110	1.6487	-	-	-
0.0200	120	1.8189	-	-	-
0.0217	130	1.6578	-	-	-
0.0234	140	1.4038	-	-	-
0.0250	150	1.815	0.2121	0.9621	0.9784
0.0267	160	1.9897	-	-	-
0.0284	170	1.354	-	-	-
0.0300	180	1.4644	-	-	-
0.0317	190	1.32	-	-	-
0.0334	200	2.0148	0.2025	0.9619	0.9780
0.0350	210	1.5873	-	-	-
0.0367	220	1.5228	-	-	-
0.0384	230	1.5522	-	-	-
0.0400	240	1.5835	-	-	-
0.0417	250	1.48	0.1885	0.9612	0.9777
0.0434	260	1.4646	-	-	-
0.0450	270	1.2024	-	-	-
0.0467	280	1.3337	-	-	-
0.0484	290	1.384	-	-	-
0.0501	300	1.0211	0.1874	0.9618	0.9773
0.0517	310	1.4389	-	-	-
0.0534	320	1.7868	-	-	-
0.0551	330	1.479	-	-	-
0.0567	340	1.4385	-	-	-
0.0584	350	1.3644	0.1863	0.9616	0.9770
0.0601	360	1.2942	-	-	-
0.0617	370	1.6685	-	-	-
0.0634	380	1.59	-	-	-
0.0651	390	1.549	-	-	-
0.0667	400	1.6519	0.1856	0.9619	0.9761
0.0684	410	1.364	-	-	-
0.0701	420	1.7517	-	-	-
0.0717	430	1.4231	-	-	-
0.0734	440	0.9029	-	-	-
0.0751	450	1.1725	0.1899	0.9613	0.9763
0.0767	460	1.0511	-	-	-
0.0784	470	1.9078	-	-	-
0.0801	480	1.1366	-	-	-
0.0817	490	1.3541	-	-	-
0.0834	500	1.726	0.1964	0.9554	0.9737
0.0851	510	1.4338	-	-	-
0.0868	520	1.5173	-	-	-
0.0884	530	1.4797	-	-	-
0.0901	540	1.5619	-	-	-
0.0918	550	1.2511	0.1931	0.9578	0.9741
0.0934	560	1.4737	-	-	-
0.0951	570	1.7115	-	-	-
0.0968	580	1.3673	-	-	-
0.0984	590	1.3894	-	-	-
0.1001	600	1.4181	0.1881	0.9594	0.9745
0.1018	610	1.2321	-	-	-
0.1034	620	1.2452	-	-	-
0.1051	630	1.3932	-	-	-
0.1068	640	1.4217	-	-	-
0.1084	650	1.6101	0.1777	0.9594	0.9762
0.1101	660	1.4969	-	-	-
0.1118	670	1.1085	-	-	-
0.1134	680	1.4825	-	-	-
0.1151	690	1.9852	-	-	-
0.1168	700	1.5358	0.1956	0.9562	0.9729
0.1185	710	1.5894	-	-	-
0.1201	720	1.6489	-	-	-
0.1218	730	1.5265	-	-	-
0.1235	740	1.9084	-	-	-
0.1251	750	1.3985	0.1826	0.9590	0.9741
0.1268	760	1.3138	-	-	-
0.1285	770	1.5475	-	-	-
0.1301	780	1.3384	-	-	-
0.1318	790	1.1803	-	-	-
0.1335	800	1.5173	0.1942	0.9589	0.9737
0.1351	810	1.3237	-	-	-
0.1368	820	1.6342	-	-	-
0.1385	830	1.5973	-	-	-
0.1401	840	1.4051	-	-	-
0.1418	850	1.0389	0.1958	0.9572	0.9737
0.1435	860	1.151	-	-	-
0.1451	870	1.1977	-	-	-
0.1468	880	1.1199	-	-	-
0.1485	890	1.4319	-	-	-
0.1502	900	1.443	0.1775	0.9593	0.9748
0.1518	910	1.4724	-	-	-
0.1535	920	1.5401	-	-	-
0.1552	930	1.0717	-	-	-
0.1568	940	1.3163	-	-	-
0.1585	950	1.4724	0.1855	0.9589	0.9738
0.1602	960	1.7999	-	-	-
0.1618	970	1.2728	-	-	-
0.1635	980	1.3425	-	-	-
0.1652	990	1.4876	-	-	-
0.1668	1000	1.5469	0.1800	0.9580	0.9737
0.1685	1010	1.6092	-	-	-
0.1702	1020	1.3766	-	-	-
0.1718	1030	1.3398	-	-	-
0.1735	1040	1.2499	-	-	-
0.1752	1050	1.6588	0.1831	0.9584	0.9736
0.1768	1060	1.5185	-	-	-
0.1785	1070	1.8739	-	-	-
0.1802	1080	1.8322	-	-	-
0.1818	1090	1.9651	-	-	-
0.1835	1100	1.2785	0.1959	0.9527	0.9709
0.1852	1110	1.5582	-	-	-
0.1869	1120	1.4033	-	-	-
0.1885	1130	1.3886	-	-	-
0.1902	1140	1.9881	-	-	-
0.1919	1150	1.4109	0.1867	0.9552	0.9718
0.1935	1160	1.476	-	-	-
0.1952	1170	1.0531	-	-	-
0.1969	1180	1.8969	-	-	-
0.1985	1190	1.6392	-	-	-
0.2002	1200	1.3833	0.1880	0.9573	0.9744
0.2019	1210	1.2882	-	-	-
0.2035	1220	1.0809	-	-	-
0.2052	1230	1.3339	-	-	-
0.2069	1240	1.3699	-	-	-
0.2085	1250	1.2584	0.1841	0.9577	0.9745
0.2102	1260	1.1301	-	-	-
0.2119	1270	1.7374	-	-	-
0.2135	1280	1.3383	-	-	-
0.2152	1290	1.4335	-	-	-
0.2169	1300	2.0107	0.1814	0.9576	0.9742
0.2186	1310	0.9604	-	-	-
0.2202	1320	1.5119	-	-	-
0.2219	1330	1.8981	-	-	-
0.2236	1340	1.3383	-	-	-
0.2252	1350	2.0782	0.1796	0.9572	0.9741
0.2269	1360	1.1343	-	-	-
0.2286	1370	1.3632	-	-	-
0.2302	1380	1.1679	-	-	-
0.2319	1390	1.7809	-	-	-
0.2336	1400	1.2453	0.1753	0.9594	0.9748
0.2352	1410	1.3163	-	-	-
0.2369	1420	1.5274	-	-	-
0.2386	1430	1.0484	-	-	-
0.2402	1440	1.4328	-	-	-
0.2419	1450	1.223	0.1929	0.9528	0.9728
0.2436	1460	1.8317	-	-	-
0.2452	1470	1.2675	-	-	-
0.2469	1480	1.1635	-	-	-
0.2486	1490	1.388	-	-	-
0.2503	1500	1.5595	0.1824	0.9579	0.9730
0.2519	1510	1.6658	-	-	-
0.2536	1520	1.3936	-	-	-
0.2553	1530	1.3174	-	-	-
0.2569	1540	0.9513	-	-	-
0.2586	1550	1.3942	0.1777	0.9604	0.9750
0.2603	1560	0.8234	-	-	-
0.2619	1570	1.3258	-	-	-
0.2636	1580	1.7316	-	-	-
0.2653	1590	1.0866	-	-	-
0.2669	1600	1.2803	0.1824	0.9575	0.9739
0.2686	1610	2.0348	-	-	-
0.2703	1620	1.4267	-	-	-
0.2719	1630	1.3216	-	-	-
0.2736	1640	1.7163	-	-	-
0.2753	1650	1.2593	0.1797	0.9594	0.9739
0.2769	1660	1.5829	-	-	-
0.2786	1670	1.1519	-	-	-
0.2803	1680	1.2319	-	-	-
0.2819	1690	1.6375	-	-	-
0.2836	1700	1.3678	0.1783	0.9580	0.9744
0.2853	1710	1.3747	-	-	-
0.2870	1720	1.2215	-	-	-
0.2886	1730	1.6477	-	-	-
0.2903	1740	1.0739	-	-	-
0.2920	1750	1.5498	0.1816	0.9579	0.9731
0.2936	1760	0.9723	-	-	-
0.2953	1770	1.2058	-	-	-
0.2970	1780	1.1969	-	-	-
0.2986	1790	1.5849	-	-	-
0.3003	1800	1.5434	0.1763	0.9594	0.9752
0.3020	1810	2.0073	-	-	-
0.3036	1820	1.4212	-	-	-
0.3053	1830	1.4124	-	-	-
0.3070	1840	1.4971	-	-	-
0.3086	1850	1.5307	0.1847	0.9576	0.9718
0.3103	1860	1.4123	-	-	-
0.3120	1870	1.3761	-	-	-
0.3136	1880	1.4105	-	-	-
0.3153	1890	1.1107	-	-	-
0.3170	1900	1.1098	0.1835	0.9563	0.9735
0.3187	1910	1.171	-	-	-
0.3203	1920	1.507	-	-	-
0.3220	1930	1.8424	-	-	-
0.3237	1940	1.5738	-	-	-
0.3253	1950	1.7357	0.1726	0.9592	0.9754
0.3270	1960	1.5224	-	-	-
0.3287	1970	1.0211	-	-	-
0.3303	1980	1.7991	-	-	-
0.3320	1990	1.8582	-	-	-
0.3337	2000	1.6377	0.1819	0.9569	0.9734
0.3353	2010	1.3359	-	-	-
0.3370	2020	1.2785	-	-	-
0.3387	2030	1.6818	-	-	-
0.3403	2040	1.4165	-	-	-
0.3420	2050	1.4343	0.1752	0.9594	0.9758
0.3437	2060	1.0041	-	-	-
0.3453	2070	1.4913	-	-	-
0.3470	2080	1.2282	-	-	-
0.3487	2090	1.3613	-	-	-
0.3504	2100	1.6876	0.1818	0.9577	0.9741
0.3520	2110	1.2597	-	-	-
0.3537	2120	0.9067	-	-	-
0.3554	2130	1.3894	-	-	-
0.3570	2140	1.236	-	-	-
0.3587	2150	1.0968	0.1760	0.9601	0.9750
0.3604	2160	0.9847	-	-	-
0.3620	2170	0.9727	-	-	-
0.3637	2180	1.5412	-	-	-
0.3654	2190	1.7212	-	-	-
0.3670	2200	1.1937	0.1804	0.9576	0.9741
0.3687	2210	1.1247	-	-	-
0.3704	2220	1.4006	-	-	-
0.3720	2230	1.2846	-	-	-
0.3737	2240	1.0221	-	-	-
0.3754	2250	1.3055	0.1708	0.9613	0.9767
0.3770	2260	1.5123	-	-	-
0.3787	2270	1.1201	-	-	-
0.3804	2280	1.6941	-	-	-
0.3820	2290	1.2071	-	-	-
0.3837	2300	1.1235	0.1678	0.9578	0.9759
0.3854	2310	1.6867	-	-	-
0.3871	2320	1.8465	-	-	-
0.3887	2330	1.251	-	-	-
0.3904	2340	1.0698	-	-	-
0.3921	2350	0.9547	0.1719	0.9598	0.9760
0.3937	2360	1.9133	-	-	-
0.3954	2370	0.9514	-	-	-
0.3971	2380	1.0868	-	-	-
0.3987	2390	0.9413	-	-	-
0.4004	2400	1.3566	0.1703	0.9584	0.9753
0.4021	2410	1.4713	-	-	-
0.4037	2420	1.5478	-	-	-
0.4054	2430	1.2848	-	-	-
0.4071	2440	1.3043	-	-	-
0.4087	2450	1.5358	0.1693	0.9581	0.9760
0.4104	2460	1.7326	-	-	-
0.4121	2470	1.5563	-	-	-
0.4137	2480	1.3791	-	-	-
0.4154	2490	1.4596	-	-	-
0.4171	2500	1.5834	0.1815	0.9559	0.9739
0.4188	2510	1.2117	-	-	-
0.4204	2520	1.4653	-	-	-
0.4221	2530	1.3325	-	-	-
0.4238	2540	1.2315	-	-	-
0.4254	2550	0.8855	0.1733	0.9582	0.9754
0.4271	2560	1.132	-	-	-
0.4288	2570	0.9599	-	-	-
0.4304	2580	1.2157	-	-	-
0.4321	2590	1.4361	-	-	-
0.4338	2600	0.9522	0.1718	0.9577	0.9754
0.4354	2610	1.2663	-	-	-
0.4371	2620	1.0655	-	-	-
0.4388	2630	1.432	-	-	-
0.4404	2640	1.5544	-	-	-
0.4421	2650	1.5839	0.1782	0.9579	0.9740
0.4438	2660	1.2948	-	-	-
0.4454	2670	1.6878	-	-	-
0.4471	2680	1.2492	-	-	-
0.4488	2690	1.7486	-	-	-
0.4505	2700	1.5627	0.1726	0.9581	0.9751
0.4521	2710	1.3681	-	-	-
0.4538	2720	1.0276	-	-	-
0.4555	2730	1.0122	-	-	-
0.4571	2740	1.439	-	-	-
0.4588	2750	1.0747	0.1638	0.9625	0.9778
0.4605	2760	1.7649	-	-	-
0.4621	2770	1.6557	-	-	-
0.4638	2780	1.2703	-	-	-
0.4655	2790	1.2516	-	-	-
0.4671	2800	1.2412	0.1631	0.9615	0.9779
0.4688	2810	1.2219	-	-	-
0.4705	2820	1.0213	-	-	-
0.4721	2830	1.3745	-	-	-
0.4738	2840	1.1786	-	-	-
0.4755	2850	1.3811	0.1611	0.9608	0.9765
0.4771	2860	1.3502	-	-	-
0.4788	2870	1.6319	-	-	-
0.4805	2880	1.3593	-	-	-
0.4821	2890	1.5185	-	-	-
0.4838	2900	1.2277	0.1677	0.9624	0.9768
0.4855	2910	1.4502	-	-	-
0.4872	2920	1.4109	-	-	-
0.4888	2930	1.3789	-	-	-
0.4905	2940	1.2006	-	-	-
0.4922	2950	1.35	0.1660	0.9615	0.9770
0.4938	2960	1.4005	-	-	-
0.4955	2970	1.0191	-	-	-
0.4972	2980	1.5413	-	-	-
0.4988	2990	1.6717	-	-	-
0.5005	3000	1.0665	0.1665	0.9599	0.9761
0.5022	3010	1.192	-	-	-
0.5038	3020	1.3714	-	-	-
0.5055	3030	1.4804	-	-	-
0.5072	3040	1.2277	-	-	-
0.5088	3050	1.3382	0.1662	0.9615	0.9768
0.5105	3060	1.3882	-	-	-
0.5122	3070	1.5946	-	-	-
0.5138	3080	1.4463	-	-	-
0.5155	3090	1.0191	-	-	-
0.5172	3100	1.4244	0.1613	0.9598	0.9772
0.5189	3110	1.1816	-	-	-
0.5205	3120	1.429	-	-	-
0.5222	3130	0.9299	-	-	-
0.5239	3140	1.0124	-	-	-
0.5255	3150	1.5549	0.1618	0.9629	0.9772
0.5272	3160	1.3469	-	-	-
0.5289	3170	1.2358	-	-	-
0.5305	3180	1.1925	-	-	-
0.5322	3190	1.3951	-	-	-
0.5339	3200	1.4948	0.1676	0.9619	0.9770
0.5355	3210	1.1471	-	-	-
0.5372	3220	1.031	-	-	-
0.5389	3230	1.1074	-	-	-
0.5405	3240	1.2574	-	-	-
0.5422	3250	0.7336	0.1690	0.9629	0.9762
0.5439	3260	1.6822	-	-	-
0.5455	3270	1.0599	-	-	-
0.5472	3280	1.3448	-	-	-
0.5489	3290	1.4727	-	-	-
0.5506	3300	1.4699	0.1640	0.9623	0.9778
0.5522	3310	1.7457	-	-	-
0.5539	3320	0.8911	-	-	-
0.5556	3330	1.1047	-	-	-
0.5572	3340	1.1863	-	-	-
0.5589	3350	1.2383	0.1611	0.9625	0.9776
0.5606	3360	1.6234	-	-	-
0.5622	3370	1.3086	-	-	-
0.5639	3380	1.8268	-	-	-
0.5656	3390	1.0265	-	-	-
0.5672	3400	1.7241	0.1617	0.9615	0.9772
0.5689	3410	1.3361	-	-	-
0.5706	3420	1.1736	-	-	-
0.5722	3430	1.3683	-	-	-
0.5739	3440	0.7918	-	-	-
0.5756	3450	1.3101	0.1603	0.9620	0.9777
0.5772	3460	1.0229	-	-	-
0.5789	3470	1.2831	-	-	-
0.5806	3480	1.4415	-	-	-
0.5822	3490	1.4262	-	-	-
0.5839	3500	1.3445	0.1577	0.9631	0.9780
0.5856	3510	0.9935	-	-	-
0.5873	3520	1.395	-	-	-
0.5889	3530	0.8853	-	-	-
0.5906	3540	1.657	-	-	-
0.5923	3550	1.7335	0.1563	0.9638	0.9779
0.5939	3560	1.3079	-	-	-
0.5956	3570	1.2795	-	-	-
0.5973	3580	1.1531	-	-	-
0.5989	3590	1.3829	-	-	-
0.6006	3600	0.9189	0.1601	0.9619	0.9769
0.6023	3610	1.3704	-	-	-
0.6039	3620	1.5831	-	-	-
0.6056	3630	1.4339	-	-	-
0.6073	3640	1.562	-	-	-
0.6089	3650	1.2673	0.1636	0.9616	0.9781
0.6106	3660	1.0708	-	-	-
0.6123	3670	0.7601	-	-	-
0.6139	3680	1.1332	-	-	-
0.6156	3690	1.3782	-	-	-
0.6173	3700	1.1785	0.1599	0.9636	0.9788
0.6190	3710	2.2274	-	-	-
0.6206	3720	1.1271	-	-	-
0.6223	3730	1.5547	-	-	-
0.6240	3740	1.1255	-	-	-
0.6256	3750	1.166	0.1624	0.9633	0.9782
0.6273	3760	1.0723	-	-	-
0.6290	3770	1.0815	-	-	-
0.6306	3780	1.1196	-	-	-
0.6323	3790	1.0639	-	-	-
0.6340	3800	0.8608	0.1601	0.9632	0.9781
0.6356	3810	1.5905	-	-	-
0.6373	3820	1.0975	-	-	-
0.6390	3830	1.3948	-	-	-
0.6406	3840	1.1477	-	-	-
0.6423	3850	1.1946	0.1591	0.9620	0.9783
0.6440	3860	1.6395	-	-	-
0.6456	3870	0.9684	-	-	-
0.6473	3880	0.8645	-	-	-
0.6490	3890	1.1112	-	-	-
0.6507	3900	1.1952	0.1592	0.9642	0.9789
0.6523	3910	1.1385	-	-	-
0.6540	3920	1.1907	-	-	-
0.6557	3930	1.1799	-	-	-
0.6573	3940	1.8078	-	-	-
0.6590	3950	1.0055	0.1625	0.9630	0.9769
0.6607	3960	1.3389	-	-	-
0.6623	3970	1.8925	-	-	-
0.6640	3980	1.2931	-	-	-
0.6657	3990	1.0189	-	-	-
0.6673	4000	1.0552	0.1559	0.9644	0.9786
0.6690	4010	1.2743	-	-	-
0.6707	4020	1.5235	-	-	-
0.6723	4030	1.1714	-	-	-
0.6740	4040	1.5646	-	-	-
0.6757	4050	1.4891	0.1561	0.9644	0.9785
0.6773	4060	1.8763	-	-	-
0.6790	4070	0.9871	-	-	-
0.6807	4080	1.3254	-	-	-
0.6823	4090	0.9636	-	-	-
0.6840	4100	1.1808	0.1570	0.9637	0.9789
0.6857	4110	1.1133	-	-	-
0.6874	4120	0.8151	-	-	-
0.6890	4130	1.1456	-	-	-
0.6907	4140	1.4371	-	-	-
0.6924	4150	1.364	0.1595	0.9642	0.9785
0.6940	4160	0.9199	-	-	-
0.6957	4170	1.5897	-	-	-
0.6974	4180	1.137	-	-	-
0.6990	4190	1.1638	-	-	-
0.7007	4200	1.0108	0.1570	0.9640	0.9782
0.7024	4210	1.3415	-	-	-
0.7040	4220	1.509	-	-	-
0.7057	4230	1.5978	-	-	-
0.7074	4240	2.0613	-	-	-
0.7090	4250	0.9348	0.1567	0.9627	0.9779
0.7107	4260	0.8756	-	-	-
0.7124	4270	1.6209	-	-	-
0.7140	4280	0.9529	-	-	-
0.7157	4290	1.6208	-	-	-
0.7174	4300	1.3453	0.1559	0.9643	0.9787
0.7191	4310	1.1624	-	-	-
0.7207	4320	1.5533	-	-	-
0.7224	4330	2.1966	-	-	-
0.7241	4340	1.0168	-	-	-
0.7257	4350	1.0473	0.1572	0.9647	0.9793
0.7274	4360	1.1875	-	-	-
0.7291	4370	0.9037	-	-	-
0.7307	4380	0.6321	-	-	-
0.7324	4390	1.2026	-	-	-
0.7341	4400	1.6893	0.1513	0.9664	0.9794
0.7357	4410	1.3212	-	-	-
0.7374	4420	1.1606	-	-	-
0.7391	4430	0.7722	-	-	-
0.7407	4440	1.7734	-	-	-
0.7424	4450	1.2735	0.1566	0.9652	0.9794
0.7441	4460	1.0845	-	-	-
0.7457	4470	1.1587	-	-	-
0.7474	4480	0.7787	-	-	-
0.7491	4490	0.7785	-	-	-
0.7508	4500	1.338	0.1527	0.9653	0.9794
0.7524	4510	1.2097	-	-	-
0.7541	4520	1.1567	-	-	-
0.7558	4530	1.4028	-	-	-
0.7574	4540	1.2984	-	-	-
0.7591	4550	1.3341	0.1583	0.9649	0.9786
0.7608	4560	1.1737	-	-	-
0.7624	4570	1.2188	-	-	-
0.7641	4580	1.1587	-	-	-
0.7658	4590	0.8006	-	-	-
0.7674	4600	1.0408	0.1559	0.9640	0.9800
0.7691	4610	0.7271	-	-	-
0.7708	4620	1.5092	-	-	-
0.7724	4630	1.0212	-	-	-
0.7741	4640	1.5169	-	-	-
0.7758	4650	1.1277	0.1478	0.9650	0.9802
0.7774	4660	1.4542	-	-	-
0.7791	4670	0.919	-	-	-
0.7808	4680	0.8274	-	-	-
0.7824	4690	1.6908	-	-	-
0.7841	4700	1.4606	0.1493	0.9648	0.9798
0.7858	4710	1.0048	-	-	-
0.7875	4720	1.5081	-	-	-
0.7891	4730	1.3667	-	-	-
0.7908	4740	0.8121	-	-	-
0.7925	4750	0.846	0.1544	0.9651	0.9792
0.7941	4760	1.1652	-	-	-
0.7958	4770	1.3295	-	-	-
0.7975	4780	0.8999	-	-	-
0.7991	4790	0.7479	-	-	-
0.8008	4800	1.1641	0.1525	0.9663	0.9798
0.8025	4810	1.1484	-	-	-
0.8041	4820	1.0466	-	-	-
0.8058	4830	1.2883	-	-	-
0.8075	4840	1.0301	-	-	-
0.8091	4850	0.775	0.1521	0.9659	0.9793
0.8108	4860	0.9239	-	-	-
0.8125	4870	1.2264	-	-	-
0.8141	4880	1.0727	-	-	-
0.8158	4890	1.204	-	-	-
0.8175	4900	1.0139	0.1532	0.9664	0.9794
0.8192	4910	1.2817	-	-	-
0.8208	4920	1.2511	-	-	-
0.8225	4930	0.6509	-	-	-
0.8242	4940	1.2564	-	-	-
0.8258	4950	1.0229	0.1548	0.9665	0.9794
0.8275	4960	1.1361	-	-	-
0.8292	4970	1.5503	-	-	-
0.8308	4980	0.7501	-	-	-
0.8325	4990	0.8413	-	-	-
0.8342	5000	2.0418	0.1509	0.9662	0.9798
0.8358	5010	1.5189	-	-	-
0.8375	5020	0.8113	-	-	-
0.8392	5030	1.4386	-	-	-
0.8408	5040	1.9761	-	-	-
0.8425	5050	1.2063	0.1531	0.9653	0.9797
0.8442	5060	0.8974	-	-	-
0.8458	5070	0.8486	-	-	-
0.8475	5080	1.3162	-	-	-
0.8492	5090	0.7764	-	-	-
0.8509	5100	0.5992	0.1515	0.9663	0.9800
0.8525	5110	0.9097	-	-	-
0.8542	5120	0.9613	-	-	-
0.8559	5130	0.6625	-	-	-
0.8575	5140	1.2888	-	-	-
0.8592	5150	1.9466	0.1505	0.9670	0.9796
0.8609	5160	0.8824	-	-	-
0.8625	5170	1.1087	-	-	-
0.8642	5180	0.9121	-	-	-
0.8659	5190	1.0143	-	-	-
0.8675	5200	0.8754	0.1492	0.9674	0.9803
0.8692	5210	1.8478	-	-	-
0.8709	5220	0.7857	-	-	-
0.8725	5230	0.6678	-	-	-
0.8742	5240	1.0645	-	-	-
0.8759	5250	1.2357	0.1495	0.9664	0.9801
0.8775	5260	0.6015	-	-	-
0.8792	5270	0.7159	-	-	-
0.8809	5280	1.1851	-	-	-
0.8825	5290	1.305	-	-	-
0.8842	5300	0.8235	0.1511	0.9672	0.9800
0.8859	5310	0.8938	-	-	-
0.8876	5320	0.9763	-	-	-
0.8892	5330	0.9663	-	-	-
0.8909	5340	0.7495	-	-	-
0.8926	5350	0.9178	0.1499	0.9671	0.9801
0.8942	5360	2.1349	-	-	-
0.8959	5370	1.3178	-	-	-
0.8976	5380	0.6819	-	-	-
0.8992	5390	0.9205	-	-	-
0.9009	5400	1.4803	0.1489	0.9670	0.9803
0.9026	5410	1.2049	-	-	-
0.9042	5420	1.1764	-	-	-
0.9059	5430	1.0023	-	-	-
0.9076	5440	0.974	-	-	-
0.9092	5450	1.2879	0.1488	0.9663	0.9802
0.9109	5460	0.8264	-	-	-
0.9126	5470	1.2608	-	-	-
0.9142	5480	0.7675	-	-	-
0.9159	5490	0.9659	-	-	-
0.9176	5500	1.3379	0.1501	0.9674	0.9806
0.9193	5510	0.6342	-	-	-
0.9209	5520	0.7475	-	-	-
0.9226	5530	1.0678	-	-	-
0.9243	5540	1.5172	-	-	-
0.9259	5550	0.7501	0.1483	0.9670	0.9805
0.9276	5560	1.1677	-	-	-
0.9293	5570	0.9074	-	-	-
0.9309	5580	0.9299	-	-	-
0.9326	5590	1.1987	-	-	-
0.9343	5600	1.2389	0.1489	0.9672	0.9801
0.9359	5610	0.8598	-	-	-
0.9376	5620	0.5813	-	-	-
0.9393	5630	0.7057	-	-	-
0.9409	5640	0.8717	-	-	-
0.9426	5650	1.0794	0.1493	0.9673	0.9802
0.9443	5660	0.8884	-	-	-
0.9459	5670	0.9711	-	-	-
0.9476	5680	1.4591	-	-	-
0.9493	5690	0.8323	-	-	-
0.9510	5700	0.6926	0.1525	0.9667	0.9804
0.9526	5710	0.8444	-	-	-
0.9543	5720	1.1166	-	-	-
0.9560	5730	1.0011	-	-	-
0.9576	5740	1.4902	-	-	-
0.9593	5750	1.2285	0.1507	0.9668	0.9803
0.9610	5760	0.6565	-	-	-
0.9626	5770	0.8254	-	-	-
0.9643	5780	0.815	-	-	-
0.9660	5790	1.0356	-	-	-
0.9676	5800	1.049	0.1488	0.9672	0.9804
0.9693	5810	1.4211	-	-	-
0.9710	5820	0.8726	-	-	-
0.9726	5830	1.0317	-	-	-
0.9743	5840	0.7001	-	-	-
0.9760	5850	1.1893	0.1489	0.9673	0.9805
0.9776	5860	1.2126	-	-	-
0.9793	5870	0.9355	-	-	-
0.9810	5880	1.6504	-	-	-
0.9826	5890	0.9891	-	-	-
0.9843	5900	1.017	0.1479	0.9672	0.9807
0.9860	5910	0.9459	-	-	-
0.9877	5920	0.8204	-	-	-
0.9893	5930	1.057	-	-	-
0.9910	5940	0.7916	-	-	-
0.9927	5950	1.0222	0.1506	0.9675	0.9806
0.9943	5960	0.7094	-	-	-
0.9960	5970	0.9955	-	-	-
0.9977	5980	0.736	-	-	-
0.9993	5990	0.7751	-	-	-

Framework Versions

• Python: 3.10.18
• Sentence Transformers: 5.1.0
• Transformers: 4.57.1
• PyTorch: 2.9.1+cu128
• Accelerate: 1.10.1
• Datasets: 4.0.0
• Tokenizers: 0.22.0

Citation

BibTeX

Sentence Transformers

@inproceedings{reimers-2019-sentence-bert,
    title = "Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks",
    author = "Reimers, Nils and Gurevych, Iryna",
    booktitle = "Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing",
    month = "11",
    year = "2019",
    publisher = "Association for Computational Linguistics",
    url = "https://arxiv.org/abs/1908.10084",
}

MyCachedMultipleNegativesRankingLoss

@misc{gao2021scaling,
    title={Scaling Deep Contrastive Learning Batch Size under Memory Limited Setup},
    author={Luyu Gao and Yunyi Zhang and Jiawei Han and Jamie Callan},
    year={2021},
    eprint={2101.06983},
    archivePrefix={arXiv},
    primaryClass={cs.LG}
}