suh4s/insightflow-balanced-team-embed-v1-7099e82c-e4c8-48ed-88a8-36bd9255036b

SentenceTransformer based on Snowflake/snowflake-arctic-embed-l

This is a sentence-transformers model finetuned from Snowflake/snowflake-arctic-embed-l. It maps sentences & paragraphs to a 1024-dimensional dense vector space and can be used for semantic textual similarity, semantic search, paraphrase mining, text classification, clustering, and more.

Model Details

Model Description

• Model Type: Sentence Transformer
• Base model: Snowflake/snowflake-arctic-embed-l
• Maximum Sequence Length: 512 tokens
• Output Dimensionality: 1024 dimensions
• Similarity Function: Cosine Similarity

Model Sources

• Documentation: Sentence Transformers Documentation
• Repository: Sentence Transformers on GitHub
• Hugging Face: Sentence Transformers on Hugging Face

Full Model Architecture

SentenceTransformer(
  (0): Transformer({'max_seq_length': 512, 'do_lower_case': False}) with Transformer model: BertModel 
  (1): Pooling({'word_embedding_dimension': 1024, 'pooling_mode_cls_token': True, 'pooling_mode_mean_tokens': False, 'pooling_mode_max_tokens': False, 'pooling_mode_mean_sqrt_len_tokens': False, 'pooling_mode_weightedmean_tokens': False, 'pooling_mode_lasttoken': False, 'include_prompt': True})
  (2): Normalize()
)

Usage

Direct Usage (Sentence Transformers)

First install the Sentence Transformers library:

pip install -U sentence-transformers

Then you can load this model and run inference.

from sentence_transformers import SentenceTransformer

# Download from the 🤗 Hub
model = SentenceTransformer("suh4s/insightflow-balanced-team-embed-v1-7099e82c-e4c8-48ed-88a8-36bd9255036b")
# Run inference
sentences = [
    'What benefits are mentioned for those who comply with the terms of the Project Gutenberg agreement?',
    '1.B. "Project Gutenberg" is a registered trademark. It may only be\nused on or associated in any way with an electronic work by people who\nagree to be bound by the terms of this agreement. There are a few\nthings that you can do with most Project Gutenberg-tm electronic works\neven without complying with the full terms of this agreement. See\nparagraph 1.C below. There are a lot of things you can do with Project\nGutenberg-tm electronic works if you follow the terms of this\nagreement and help preserve free future access to Project Gutenberg-tm\nelectronic works. See paragraph 1.E below.',
    'This is about all that he has to say. He will not entreat the judges to\nspare his life; neither will he present a spectacle of weeping\nchildren, although he, too, is not made of “rock or oak.” Some of the\njudges themselves may have complied with this practice on similar\noccasions, and he trusts that they will not be angry with him for not\nfollowing their example. But he feels that such conduct brings\ndiscredit on the name of Athens: he feels too, that the judge has sworn\nnot to give away justice; and he cannot be guilty of the impiety of\nasking the judge to break his oath, when he is himself being tried for\nimpiety.',
]
embeddings = model.encode(sentences)
print(embeddings.shape)
# [3, 1024]

# Get the similarity scores for the embeddings
similarities = model.similarity(embeddings, embeddings)
print(similarities.shape)
# [3, 3]

Evaluation

Metrics

Information Retrieval

• Evaluated with InformationRetrievalEvaluator

Metric	Value
cosine_accuracy@1	0.5729
cosine_accuracy@3	0.7271
cosine_accuracy@5	0.7792
cosine_accuracy@10	0.8417
cosine_precision@1	0.5729
cosine_precision@3	0.2424
cosine_precision@5	0.1558
cosine_precision@10	0.0842
cosine_recall@1	0.5729
cosine_recall@3	0.7271
cosine_recall@5	0.7792
cosine_recall@10	0.8417
cosine_ndcg@10	0.7066
cosine_mrr@10	0.6635
cosine_map@100	0.6715

Information Retrieval

• Evaluated with InformationRetrievalEvaluator

Metric	Value
cosine_accuracy@1	0.5458
cosine_accuracy@3	0.725
cosine_accuracy@5	0.7833
cosine_accuracy@10	0.8667
cosine_precision@1	0.5458
cosine_precision@3	0.2417
cosine_precision@5	0.1567
cosine_precision@10	0.0867
cosine_recall@1	0.5458
cosine_recall@3	0.725
cosine_recall@5	0.7833
cosine_recall@10	0.8667
cosine_ndcg@10	0.7036
cosine_mrr@10	0.6518
cosine_map@100	0.6594

Training Details

Training Dataset

Unnamed Dataset

• Size: 3,562 training samples
• Columns: sentence_0 and sentence_1
• Approximate statistics based on the first 1000 samples:

  	sentence_0	sentence_1
  type	string	string
  details	min: 9 tokens mean: 18.47 tokens max: 47 tokens	min: 3 tokens mean: 125.56 tokens max: 220 tokens

• Samples:

  sentence_0	sentence_1
  What causes Kay to burst into tears in the passage?	"Roses bloom and fade away, But we the Christ-child see alway." Then Kay burst into tears. He wept so that the splinter of glass swam out of his eye. Then he recognized Gerda and said joyfully, "Gerda, dear little Gerda, where have you been all this time, and where have I been?" And he looked all around him and said, "How cold it is, and how large and empty it all looks," and he clung to Gerda, and she laughed and wept for joy.
  How does Kay react upon recognizing Gerda?	"Roses bloom and fade away, But we the Christ-child see alway." Then Kay burst into tears. He wept so that the splinter of glass swam out of his eye. Then he recognized Gerda and said joyfully, "Gerda, dear little Gerda, where have you been all this time, and where have I been?" And he looked all around him and said, "How cold it is, and how large and empty it all looks," and he clung to Gerda, and she laughed and wept for joy.
  What editions were compared to prepare the text mentioned in the context?	The text itself has been prepared by a comparison of the editions of 1634 and 1635. It should be borne in mind that Casaubon's is often rather a paraphrase than a close translation; and it did not seem worth while to notice every variation or amplification of the original. In the original editions all that Casaubon conceives as understood, but not expressed, is enclosed in square brackets. These brackets are here omitted, as they interfere with the comfort of the reader; and so have some of the alternative renderings suggested by the translator. In a few cases, Latin words in the text have been replaced by English.

• Loss: MatryoshkaLoss with these parameters:

  {
      "loss": "MultipleNegativesRankingLoss",
      "matryoshka_dims": [
          768,
          512,
          256,
          128,
          64
      ],
      "matryoshka_weights": [
          1,
          1,
          1,
          1,
          1
      ],
      "n_dims_per_step": -1
  }
  

Training Hyperparameters

Non-Default Hyperparameters

• eval_strategy: steps
• num_train_epochs: 10
• multi_dataset_batch_sampler: round_robin

All Hyperparameters

Click to expand

• overwrite_output_dir: False
• do_predict: False
• eval_strategy: steps
• prediction_loss_only: True
• per_device_train_batch_size: 8
• per_device_eval_batch_size: 8
• per_gpu_train_batch_size: None
• per_gpu_eval_batch_size: None
• gradient_accumulation_steps: 1
• eval_accumulation_steps: None
• torch_empty_cache_steps: None
• learning_rate: 5e-05
• weight_decay: 0.0
• adam_beta1: 0.9
• adam_beta2: 0.999
• adam_epsilon: 1e-08
• max_grad_norm: 1
• num_train_epochs: 10
• max_steps: -1
• lr_scheduler_type: linear
• lr_scheduler_kwargs: {}
• warmup_ratio: 0.0
• warmup_steps: 0
• log_level: passive
• log_level_replica: warning
• log_on_each_node: True
• logging_nan_inf_filter: True
• save_safetensors: True
• save_on_each_node: False
• save_only_model: False
• restore_callback_states_from_checkpoint: False
• no_cuda: False
• use_cpu: False
• use_mps_device: False
• seed: 42
• data_seed: None
• jit_mode_eval: False
• use_ipex: False
• bf16: False
• fp16: False
• fp16_opt_level: O1
• half_precision_backend: auto
• bf16_full_eval: False
• fp16_full_eval: False
• tf32: None
• local_rank: 0
• ddp_backend: None
• tpu_num_cores: None
• tpu_metrics_debug: False
• debug: []
• dataloader_drop_last: False
• dataloader_num_workers: 0
• dataloader_prefetch_factor: None
• past_index: -1
• disable_tqdm: False
• remove_unused_columns: True
• label_names: None
• load_best_model_at_end: False
• ignore_data_skip: False
• fsdp: []
• fsdp_min_num_params: 0
• fsdp_config: {'min_num_params': 0, 'xla': False, 'xla_fsdp_v2': False, 'xla_fsdp_grad_ckpt': False}
• tp_size: 0
• fsdp_transformer_layer_cls_to_wrap: None
• accelerator_config: {'split_batches': False, 'dispatch_batches': None, 'even_batches': True, 'use_seedable_sampler': True, 'non_blocking': False, 'gradient_accumulation_kwargs': None}
• deepspeed: None
• label_smoothing_factor: 0.0
• optim: adamw_torch
• optim_args: None
• adafactor: False
• group_by_length: False
• length_column_name: length
• ddp_find_unused_parameters: None
• ddp_bucket_cap_mb: None
• ddp_broadcast_buffers: False
• dataloader_pin_memory: True
• dataloader_persistent_workers: False
• skip_memory_metrics: True
• use_legacy_prediction_loop: False
• push_to_hub: False
• resume_from_checkpoint: None
• hub_model_id: None
• hub_strategy: every_save
• hub_private_repo: None
• hub_always_push: False
• gradient_checkpointing: False
• gradient_checkpointing_kwargs: None
• include_inputs_for_metrics: False
• include_for_metrics: []
• eval_do_concat_batches: True
• fp16_backend: auto
• push_to_hub_model_id: None
• push_to_hub_organization: None
• mp_parameters:
• auto_find_batch_size: False
• full_determinism: False
• torchdynamo: None
• ray_scope: last
• ddp_timeout: 1800
• torch_compile: False
• torch_compile_backend: None
• torch_compile_mode: None
• include_tokens_per_second: False
• include_num_input_tokens_seen: False
• neftune_noise_alpha: None
• optim_target_modules: None
• batch_eval_metrics: False
• eval_on_start: False
• use_liger_kernel: False
• eval_use_gather_object: False
• average_tokens_across_devices: False
• prompts: None
• batch_sampler: batch_sampler
• multi_dataset_batch_sampler: round_robin

Training Logs

Click to expand

Epoch	Step	Training Loss	cosine_ndcg@10
0.0561	50	-	0.5573
0.1122	100	-	0.6540
0.1684	150	-	0.6854
0.2245	200	-	0.7066
0.1121	50	-	0.7155
0.2242	100	-	0.7171
0.3363	150	-	0.7278
0.4484	200	-	0.7270
0.5605	250	-	0.7112
0.6726	300	-	0.7238
0.7848	350	-	0.7255
0.8969	400	-	0.7120
1.0	446	-	0.7164
1.0090	450	-	0.7170
1.1211	500	0.4113	0.7293
1.2332	550	-	0.7032
1.3453	600	-	0.7130
1.4574	650	-	0.6967
1.5695	700	-	0.7132
1.6816	750	-	0.7089
1.7937	800	-	0.7106
1.9058	850	-	0.7148
2.0	892	-	0.7118
2.0179	900	-	0.7072
2.1300	950	-	0.7111
2.2422	1000	0.2493	0.7178
2.3543	1050	-	0.7102
2.4664	1100	-	0.7090
2.5785	1150	-	0.7098
2.6906	1200	-	0.7136
2.8027	1250	-	0.7059
2.9148	1300	-	0.7131
3.0	1338	-	0.7166
3.0269	1350	-	0.7076
3.1390	1400	-	0.7101
3.2511	1450	-	0.7095
3.3632	1500	0.1173	0.6990
3.4753	1550	-	0.7016
3.5874	1600	-	0.6802
3.6996	1650	-	0.6917
3.8117	1700	-	0.7028
3.9238	1750	-	0.6976
4.0	1784	-	0.7015
4.0359	1800	-	0.6949
4.1480	1850	-	0.6894
4.2601	1900	-	0.6926
4.3722	1950	-	0.6929
4.4843	2000	0.052	0.6984
4.5964	2050	-	0.7068
4.7085	2100	-	0.7101
4.8206	2150	-	0.6962
4.9327	2200	-	0.6968
5.0	2230	-	0.7083
5.0448	2250	-	0.7026
5.1570	2300	-	0.7023
5.2691	2350	-	0.7032
5.3812	2400	-	0.7087
5.4933	2450	-	0.7171
5.6054	2500	0.0291	0.7056
5.7175	2550	-	0.6969
5.8296	2600	-	0.7000
5.9417	2650	-	0.7012
6.0	2676	-	0.7068
6.0538	2700	-	0.7084
6.1659	2750	-	0.7002
6.2780	2800	-	0.7015
6.3901	2850	-	0.6974
6.5022	2900	-	0.6903
6.6143	2950	-	0.6943
6.7265	3000	0.0304	0.6938
6.8386	3050	-	0.6988
6.9507	3100	-	0.7001
7.0	3122	-	0.7004
7.0628	3150	-	0.7004
7.1749	3200	-	0.6950
7.2870	3250	-	0.6975
7.3991	3300	-	0.6927
7.5112	3350	-	0.6974
7.6233	3400	-	0.6928
7.7354	3450	-	0.6963
7.8475	3500	0.0207	0.6930
7.9596	3550	-	0.6938
8.0	3568	-	0.6952
8.0717	3600	-	0.6936
8.1839	3650	-	0.7045
8.2960	3700	-	0.6964
8.4081	3750	-	0.6940
8.5202	3800	-	0.6989
8.6323	3850	-	0.6991
8.7444	3900	-	0.6992
8.8565	3950	-	0.6997
8.9686	4000	0.0228	0.7004
9.0	4014	-	0.6993
9.0807	4050	-	0.7030
9.1928	4100	-	0.7040
9.3049	4150	-	0.7071
9.4170	4200	-	0.7059
9.5291	4250	-	0.7049
9.6413	4300	-	0.7037
9.7534	4350	-	0.7037
9.8655	4400	-	0.7038
9.9776	4450	-	0.7036
10.0	4460	-	0.7036

Framework Versions

• Python: 3.11.12
• Sentence Transformers: 4.1.0
• Transformers: 4.51.3
• PyTorch: 2.6.0+cu124
• Accelerate: 1.6.0
• Datasets: 3.6.0
• Tokenizers: 0.21.1

Citation

BibTeX

Sentence Transformers

@inproceedings{reimers-2019-sentence-bert,
    title = "Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks",
    author = "Reimers, Nils and Gurevych, Iryna",
    booktitle = "Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing",
    month = "11",
    year = "2019",
    publisher = "Association for Computational Linguistics",
    url = "https://arxiv.org/abs/1908.10084",
}

MatryoshkaLoss

@misc{kusupati2024matryoshka,
    title={Matryoshka Representation Learning},
    author={Aditya Kusupati and Gantavya Bhatt and Aniket Rege and Matthew Wallingford and Aditya Sinha and Vivek Ramanujan and William Howard-Snyder and Kaifeng Chen and Sham Kakade and Prateek Jain and Ali Farhadi},
    year={2024},
    eprint={2205.13147},
    archivePrefix={arXiv},
    primaryClass={cs.LG}
}

MultipleNegativesRankingLoss

@misc{henderson2017efficient,
    title={Efficient Natural Language Response Suggestion for Smart Reply},
    author={Matthew Henderson and Rami Al-Rfou and Brian Strope and Yun-hsuan Sung and Laszlo Lukacs and Ruiqi Guo and Sanjiv Kumar and Balint Miklos and Ray Kurzweil},
    year={2017},
    eprint={1705.00652},
    archivePrefix={arXiv},
    primaryClass={cs.CL}
}