Hugging Face's logo

vab46
/
nomic-embed-text-v1.5_Clinical-Trials_Matryoshka

Sentence Similarity
sentence-transformers
Safetensors
English
nomic_bert
feature-extraction
dense
Generated from Trainer
dataset_size:7193
loss:MatryoshkaLoss
loss:MultipleNegativesRankingLoss
custom_code
Eval Results (legacy)
text-embeddings-inference
Model card Files
xet
 Community

nomic-embed text-v1.5 Clinical-Trials Matryoshka

This is a sentence-transformers model finetuned from nomic-ai/nomic-embed-text-v1.5 on the dataset from Clinical_trials_anchor-positive-pairs_EmbeddingModel-data. It maps sentences & paragraphs to a 768-dimensional dense vector space and can be used for semantic textual similarity, semantic search, paraphrase mining, classification, clustering, and more.

Model Details

Model Description

Model Sources

Full Model Architecture 

SentenceTransformer(
  (0): Transformer({'transformer_task': 'feature-extraction', 'modality_config': {'text': {'method': 'forward', 'method_output_name': 'last_hidden_state'}}, 'module_output_name': 'token_embeddings', 'architecture': 'NomicBertModel'})
  (1): Pooling({'embedding_dimension': 768, 'pooling_mode': 'mean', 'include_prompt': True})
)

Usage

Direct Usage (Sentence Transformers)

First install the Sentence Transformers library:

pip install -U sentence-transformers

Then you can load this model and run inference.

from sentence_transformers import SentenceTransformer

# Download from the 🤗 Hub
model = SentenceTransformer("vab46/nomic-embed-text-v1.5_Clinical-Trials_Matryoshka")
# Run inference
documents = [
    "TITLE: Exergames-acceptance and Commitment Therapy(e-ACT) for Breast Cancer Depression and Anxiety: A Randomized Controlled Trial The Efficacy of Exergames-acceptance and Commitment Therapy Program for Treatment of Depression and Other Psychological Complications in Breast Cancer: Comparison With Acceptance and Commitment Therapy Alone and Treatment-as-usual in a Randomized Controlled Trial\nSUMMARY: The goal of this three arm, double-blind, randomized controlled trial is to learn if an exergame-acceptance and commitment therapy (e-ACT) program can treat depression and anxiety in breast cancer patients, compared to acceptance and commitment therapy (ACT) alone and treatment-as-usual. The main question it aims to answer is:\nIs there a difference in the effectiveness of the e-ACT program in reducing depressive and anxiety symptoms, cancer-related fatigue, experiential avoidance, and serum IL-6 levels, while increasing BDNF and improving posttraumatic growth and quality of life among breast cancer patients, compared to ACT alone and treatment-as-usual, measured at baseline, 8 weeks (post-intervention), and 12 weeks after the intervention (follow-up)?\nResearchers will compare the e-ACT group, the ACT-alone group, and the treatment-as-usual group to see if the e-ACT program yields superior outcomes in reducing psychological distress and improving well-being.\nParticipants will:\nBe randomly assigned to one of three groups: (1) e-ACT (exergame + ACT), (2) ACT alone, or (3) treatment-as-usual (general patient education).\nAttend an 8-week program (one session per week) if in the e-ACT or ACT group; the control group continues their usual care.\nComplete questionnaires at three time points (baseline, 8 weeks, and 20 weeks) to assess depression, anxiety, quality of life, posttraumatic growth, valued living, experiential avoidance, and cancer-related fatigue.\nProvide blood samples at pre-intervention and post-intervention (week 8) for analysis of interleukin-6 (IL-6) and brain-derived neurotrophic factor (BDNF) biomarkers.\nThe INCLUSION CRITERIA include:\n1. Newly diagnosed breast cancer patients and patients with recurrent breast cancer confirmed by histopathological report, regardless of the stage of cancer.\n2. Those with HADS score of 8 or higher in both Depression and Anxiety sub-scales of the HADS.\n3. Patients who had been treated with surgery or were undergoing the standard regime of clinical anti-tumor treatment (chemotherapy, radiotherapy, immunotherapy, targeted therapy, etc.).\n4. Age 18 years old and above.\n5. Patients who were able to read and understand written Chinese.\nThe EXCLUSION CRITERIA are:\n1. Pregnant women，considered for the following reasons: Pregnancy involves significant hormonal and physical changes that could affect the participant's response to the e-ACT; the safety of e-ACT to the unborn child was of concern.\n2. Those who have current and lifetime history of engaging in any psychotherapy\n3. Those who consumed alcohol and illicit drugs .\n4. Those who has current and lifetime history of other psychiatric illnesses, such as psychotic disorders (schizophrenia, schizophreniform disorder, schizoaffective disorders, brief psychotic disorder, and delusional disorder), bipolar mood disorder, obsessive compulsive disorder, posttraumatic stress disorder, and attention deficit hyperactive disorder, and autism spectrum disorder\n5. Those who are on medications that can induce psychiatric symptoms, such as cardiovascular agents (clonidine, guanethidine, methyldopa, reserpine, beta blockers), dermatologic agents (isotretinoin), anticonvulsants (levetiracetam), antimigraine medications (triptans), hormonal agents (corticosteroids, oral contraceptives, gonadotropin-releasing hormone agonists, tamoxifen), varenicline, immunological agents (interferons), and levodopa.Or those who ccurrently using any psychotropic medication.\n6. Patient who has suicidal tendency.\n7. those who are physically unfit to answer questionnaires. (those are bed-bound or too weak to answer the questionnaire).",
]
queries = [
    'Could a patient with a history of depression and anxiety, who has been diagnosed with breast cancer, qualify for this study?',
    'I have obesity and high blood pressure, can I participate in this study and what will happen if I join?',
    'How old do you need to be to be eligible for this clinical trial?',
]
query_embeddings = model.encode_query(queries)
document_embeddings = model.encode_document(documents)
print(query_embeddings.shape, document_embeddings.shape)
# [1, 768] [3, 768]

# Get the similarity scores for the embeddings
similarities = model.similarity(query_embeddings, document_embeddings)
print(similarities)
# tensor([[0.7282, 0.1019, 0.2957]])

Evaluation

Metrics

Information Retrieval

Metric dim 756 dim 512 dim 256 dim 128 dim 64
cosine_accuracy@1 0.3887 0.3787 0.3725 0.345 0.2963
cosine_accuracy@3 0.5138 0.5138 0.4988 0.47 0.43
cosine_accuracy@5 0.5625 0.5575 0.5375 0.525 0.4813
cosine_accuracy@10 0.6062 0.5988 0.585 0.5675 0.5288
cosine_precision@1 0.3887 0.3787 0.3725 0.345 0.2963
cosine_precision@3 0.1713 0.1713 0.1663 0.1567 0.1433
cosine_precision@5 0.1125 0.1115 0.1075 0.105 0.0963
cosine_precision@10 0.0606 0.0599 0.0585 0.0567 0.0529
cosine_recall@1 0.3887 0.3787 0.3725 0.345 0.2963
cosine_recall@3 0.5138 0.5138 0.4988 0.47 0.43
cosine_recall@5 0.5625 0.5575 0.5375 0.525 0.4813
cosine_recall@10 0.6062 0.5988 0.585 0.5675 0.5288
cosine_ndcg@10 **0.4972 0.4891 0.4777 0.455 0.41**
cosine_mrr@10 0.4623 0.4538 0.4434 0.419 0.3722
cosine_map@100 0.467 0.4589 0.449 0.425 0.3788

Comparison with the baseline model performance on same data

metric dimensions base_model_value FT_model_value diff %change
accuracy@1 768 0.315 0.41375 0.09875 31.34920635
accuracy@1 512 0.30125 0.415 0.11375 37.7593361
accuracy@1 256 0.28625 0.3925 0.10625 37.11790393
accuracy@1 128 0.26125 0.35875 0.0975 37.32057416
accuracy@1 64 0.21125 0.3025 0.09125 43.19526627
accuracy@10 64 0.39875 0.51125 0.1125 28.21316614
accuracy@10 256 0.46875 0.58625 0.1175 25.06666667
accuracy@10 128 0.4375 0.57375 0.13625 31.14285714
accuracy@10 768 0.50375 0.59875 0.095 18.85856079
accuracy@10 512 0.5 0.59625 0.09625 19.25
accuracy@3 768 0.42 0.52625 0.10625 25.29761905
accuracy@3 512 0.40375 0.52 0.11625 28.79256966
accuracy@3 256 0.38875 0.50375 0.115 29.58199357
accuracy@3 128 0.365 0.475 0.11 30.1369863
accuracy@3 64 0.2975 0.42375 0.12625 42.43697479
accuracy@5 64 0.345 0.46625 0.12125 35.14492754
accuracy@5 256 0.43 0.54 0.11 25.58139535
accuracy@5 128 0.39375 0.515 0.12125 30.79365079
accuracy@5 768 0.4575 0.55 0.0925 20.21857923
accuracy@5 512 0.44875 0.55375 0.105 23.39832869
map@100 768 0.382026465 0.48036608 0.098339615 25.74157148
map@100 512 0.370630917 0.478453773 0.107822857 29.09170609
map@100 256 0.351904934 0.459528498 0.107623564 30.58313571
map@100 128 0.324395831 0.431321436 0.106925606 32.96146109
map@100 64 0.272953217 0.376392287 0.103439069 37.89626304
mrr@10 256 0.346236111 0.454637897 0.108401786 31.3086308
mrr@10 64 0.266976687 0.369970734 0.102994048 38.57791816
mrr@10 128 0.318508433 0.426855655 0.108347222 34.01706553
mrr@10 512 0.364892857 0.473862103 0.108969246 29.8633541
mrr@10 768 0.376446429 0.475852679 0.09940625 26.40647977
ndcg@10 768 0.407159105 0.505456639 0.098297535 24.14229076
ndcg@10 512 0.397313669 0.503315103 0.106001433 26.67953348
ndcg@10 256 0.37577036 0.48623639 0.110466031 29.39721772
ndcg@10 128 0.347224832 0.462075633 0.1148508 33.07678184
ndcg@10 64 0.298362231 0.404066399 0.105704168 35.42813311
precision@1 64 0.21125 0.3025 0.09125 43.19526627
precision@1 128 0.26125 0.35875 0.0975 37.32057416
precision@1 256 0.28625 0.3925 0.10625 37.11790393
precision@1 512 0.30125 0.415 0.11375 37.7593361
precision@1 768 0.315 0.41375 0.09875 31.34920635
precision@10 768 0.050375 0.059875 0.0095 18.85856079
precision@10 512 0.05 0.059625 0.009625 19.25
precision@10 256 0.046875 0.058625 0.01175 25.06666667
precision@10 128 0.04375 0.057375 0.013625 31.14285714
precision@10 64 0.039875 0.051125 0.01125 28.21316614
precision@3 64 0.099166667 0.14125 0.042083333 42.43697479
precision@3 128 0.121666667 0.158333333 0.036666667 30.1369863
precision@3 256 0.129583333 0.167916667 0.038333333 29.58199357
precision@3 512 0.134583333 0.173333333 0.03875 28.79256966
precision@3 768 0.14 0.175416667 0.035416667 25.29761905
precision@5 768 0.0915 0.11 0.0185 20.21857923
precision@5 512 0.08975 0.11075 0.021 23.39832869
precision@5 256 0.086 0.108 0.022 25.58139535
precision@5 128 0.07875 0.103 0.02425 30.79365079
precision@5 64 0.069 0.09325 0.02425 35.14492754
recall@1 64 0.21125 0.3025 0.09125 43.19526627
recall@1 128 0.26125 0.35875 0.0975 37.32057416
recall@1 512 0.30125 0.415 0.11375 37.7593361
recall@1 768 0.315 0.41375 0.09875 31.34920635
recall@1 256 0.28625 0.3925 0.10625 37.11790393
recall@10 768 0.50375 0.59875 0.095 18.85856079
recall@10 512 0.5 0.59625 0.09625 19.25
recall@10 256 0.46875 0.58625 0.1175 25.06666667
recall@10 128 0.4375 0.57375 0.13625 31.14285714
recall@10 64 0.39875 0.51125 0.1125 28.21316614
recall@3 512 0.40375 0.52 0.11625 28.79256966
recall@3 768 0.42 0.52625 0.10625 25.29761905
recall@3 256 0.38875 0.50375 0.115 29.58199357
recall@3 128 0.365 0.475 0.11 30.1369863
recall@3 64 0.2975 0.42375 0.12625 42.43697479
recall@5 256 0.43 0.54 0.11 25.58139535
recall@5 128 0.39375 0.515 0.12125 30.79365079
recall@5 768 0.4575 0.55 0.0925 20.21857923
recall@5 512 0.44875 0.55375 0.105 23.39832869
recall@5 64 0.345 0.46625 0.12125 35.14492754
AVERAGE 30.22854974
VARIANCE 43.17916576
ST-DEV 6.571085584

Training Details

Training Dataset

json

  • Dataset: json
  • Size: 7,193 training samples
  • Columns: positive and anchor
  • Approximate statistics based on the first 100 samples:
    positive anchor
    type string string
    modality text text
    details
    • min: 271 tokens
    • mean: 671.29 tokens
    • max: 2110 tokens
    • min: 10 tokens
    • mean: 20.11 tokens
    • max: 35 tokens
  • Samples:
    positive anchor
    TITLE: Comparison of Postoperative Analgesic Efficacy of Two Different Blocks in Modified Radical Mastectomy Surgery Comparison of Postoperative Analgesic Efficacy of Superior Posterior Serratus Intercostal Plane Block (SPSIP) and Anterior Serratus Plane Block in Modified Radical Mastectomy Surgery
    SUMMARY: Postoperative pain following modified radical mastectomy remains a significant clinical challenge and may delay recovery, impair patient comfort, and increase opioid consumption. Ultrasound-guided fascial plane blocks have become an important component of multimodal analgesia for breast surgery. The serratus anterior plane block (SAPB) is an established regional anesthesia technique that provides effective postoperative analgesia. The serratus posterior superior intercostal plane block (SPSIPB) is a recently described interfascial block with promising analgesic effects, but evidence comparing its efficacy with SAPB in breast surgery remains limited.
    This prospective, randomized, dou...    		 Is this trial open to male patients?
    TITLE: Masticatory Efficiency, Bite Force, and Patient-Reported Outcomes in Patients Rehabilitated With Complete and Partial Dentures Supported by Teeth, Mini-Implants, or Conventional Implants Masticatory Efficiency, Bite Force, and Patient-Reported Outcomes in Patients Rehabilitated With Conventional Complete Dentures, Conventional Removable Partial Dentures, Mini-Implant-Retained Overdentures, Conventional Implant-Retained Overdentures, and Implant-Assisted Removable Partial Dentures: A Cross-Sectional and Longitudinal Clinical Study
    SUMMARY: This study aims to evaluate masticatory efficiency, bite force, and patient-reported outcomes in edentulous and partially edentulous patients rehabilitated with different removable prosthodontic treatment modalities. The study includes both cross-sectional and longitudinal components. The cross-sectional component will compare patients wearing conventional complete dentures, conventional removable partial dentures, mini-implant-retained overden...    		 Could I qualify for this trial if I have a partial denture?
    TITLE: ctDNA-driven Adaptive Proton Craniospinal Irradiation in Non-Small Cell Lung Cancer With Leptomeningeal Metastasis After Resistance to Third-Generation TKIs in the Consolidation Phase Dynamic Adaptive Radiotherapy for Non-Small Cell Lung Cancer With Leptomeningeal Metastasis After Resistance to Third-Generation TKIs in the Consolidation Phase: A Multicenter Randomized Controlled Trial Comparing Outcomes Between Proton Craniospinal Irradiation and Intrathecal Pemetrexed (DART-LM)
    SUMMARY: **Brief Summary (English)**
    The goal of this clinical trial is to learn if a risk-adaptive consolidation therapy, guided by cerebrospinal fluid (CSF) circulating tumor DNA (ctDNA) clearance kinetics after induction intrathecal pemetrexed, can improve intracranial progression-free survival (iPFS) compared to standard intrathecal pemetrexed consolidation in patients with leptomeningeal metastasis (LM) from EGFR-mutant non-small cell lung cancer (NSCLC) that has progressed on third-generation E...    		 I have lung cancer that has spread to my brain, can I take part in a trial that uses a special radiation treatment?
  • Loss: MatryoshkaLoss with these parameters:
    {
    "loss": "MultipleNegativesRankingLoss",
    "matryoshka_dims": [
        768,
        512,
        256,
        128,
        64
    ],
    "matryoshka_weights": [
        1,
        1,
        1,
        1,
        1
    ],
    "n_dims_per_step": -1
}

Training Hyperparameters

Non-Default Hyperparameters

  • per_device_train_batch_size: 4
  • num_train_epochs: 4
  • learning_rate: 2e-05
  • lr_scheduler_type: cosine
  • warmup_steps: 0.1
  • gradient_accumulation_steps: 8
  • fp16: True
  • per_device_eval_batch_size: 32
  • load_best_model_at_end: True
  • batch_sampler: no_duplicates

All Hyperparameters

Click to expand
  • per_device_train_batch_size: 4
  • num_train_epochs: 4
  • max_steps: -1
  • learning_rate: 2e-05
  • lr_scheduler_type: cosine
  • lr_scheduler_kwargs: None
  • warmup_steps: 0.1
  • optim: adamw_torch_fused
  • optim_args: None
  • weight_decay: 0.0
  • adam_beta1: 0.9
  • adam_beta2: 0.999
  • adam_epsilon: 1e-08
  • optim_target_modules: None
  • gradient_accumulation_steps: 8
  • average_tokens_across_devices: True
  • max_grad_norm: 1.0
  • label_smoothing_factor: 0.0
  • bf16: False
  • fp16: True
  • bf16_full_eval: False
  • fp16_full_eval: False
  • tf32: None
  • gradient_checkpointing: False
  • gradient_checkpointing_kwargs: None
  • torch_compile: False
  • torch_compile_backend: None
  • torch_compile_mode: None
  • use_liger_kernel: False
  • liger_kernel_config: None
  • use_cache: False
  • neftune_noise_alpha: None
  • torch_empty_cache_steps: None
  • auto_find_batch_size: False
  • log_on_each_node: True
  • logging_nan_inf_filter: True
  • include_num_input_tokens_seen: no
  • log_level: passive
  • log_level_replica: warning
  • disable_tqdm: False
  • project: huggingface
  • trackio_space_id: None
  • trackio_bucket_id: None
  • trackio_static_space_id: None
  • per_device_eval_batch_size: 32
  • prediction_loss_only: True
  • eval_on_start: False
  • eval_do_concat_batches: True
  • eval_use_gather_object: False
  • eval_accumulation_steps: None
  • include_for_metrics: []
  • batch_eval_metrics: False
  • save_only_model: False
  • save_on_each_node: False
  • enable_jit_checkpoint: False
  • push_to_hub: False
  • hub_private_repo: None
  • hub_model_id: None
  • hub_strategy: every_save
  • hub_always_push: False
  • hub_revision: None
  • load_best_model_at_end: True
  • ignore_data_skip: False
  • restore_callback_states_from_checkpoint: False
  • full_determinism: False
  • seed: 42
  • data_seed: None
  • use_cpu: False
  • accelerator_config: {'split_batches': False, 'dispatch_batches': None, 'even_batches': True, 'use_seedable_sampler': True, 'non_blocking': False, 'gradient_accumulation_kwargs': None}
  • parallelism_config: None
  • dataloader_drop_last: False
  • dataloader_num_workers: 0
  • dataloader_pin_memory: True
  • dataloader_persistent_workers: False
  • dataloader_prefetch_factor: None
  • dataloader_multiprocessing_context: None
  • dataloader_in_order: True
  • remove_unused_columns: True
  • label_names: None
  • train_sampling_strategy: random
  • length_column_name: length
  • ddp_find_unused_parameters: None
  • ddp_bucket_cap_mb: None
  • ddp_broadcast_buffers: False
  • ddp_static_graph: None
  • ddp_backend: None
  • ddp_timeout: 1800
  • fsdp: None
  • fsdp_config: None
  • deepspeed: None
  • debug: []
  • skip_memory_metrics: True
  • do_predict: False
  • resume_from_checkpoint: None
  • local_rank: -1
  • prompts: None
  • batch_sampler: no_duplicates
  • multi_dataset_batch_sampler: proportional
  • router_mapping: {}
  • learning_rate_mapping: {}
  • warmup_ratio: None

Training Logs

Epoch Step Training Loss dim_768_cosine_ndcg@10 dim_512_cosine_ndcg@10 dim_256_cosine_ndcg@10 dim_128_cosine_ndcg@10 dim_64_cosine_ndcg@10
0.0445 10 4.1210 - - - - -
0.0889 20 3.0847 - - - - -
0.1334 30 2.5468 - - - - -
0.1779 40 2.4293 - - - - -
0.2223 50 2.0351 - - - - -
0.2668 60 1.9562 - - - - -
0.3113 70 1.8560 - - - - -
0.3558 80 1.7936 - - - - -
0.4002 90 1.4392 - - - - -
0.4447 100 1.8119 0.5052 0.5024 0.4855 0.4616 0.4034
0.4892 110 1.9161 - - - - -
0.5336 120 1.5390 - - - - -
0.5781 130 1.2008 - - - - -
0.6226 140 1.5791 - - - - -
0.6670 150 1.7636 - - - - -
0.7115 160 1.8731 - - - - -
0.7560 170 1.6094 - - - - -
0.8004 180 1.4552 - - - - -
0.8449 190 1.3247 - - - - -
0.8894 200 1.6684 0.4950 0.4951 0.4729 0.4431 0.3967
0.9339 210 1.1478 - - - - -
0.9783 220 1.5489 - - - - -
1.0222 230 1.1574 - - - - -
1.0667 240 1.1790 - - - - -
1.1112 250 1.0418 - - - - -
1.1556 260 0.9808 - - - - -
1.2001 270 1.3452 - - - - -
1.2446 280 1.4245 - - - - -
1.2890 290 1.1960 - - - - -
1.3335 300 1.2613 0.4888 0.4840 0.4702 0.4500 0.4001
1.3780 310 1.2790 - - - - -
1.4225 320 1.0427 - - - - -
1.4669 330 1.4931 - - - - -
1.5114 340 0.9111 - - - - -
1.5559 350 1.2460 - - - - -
1.6003 360 1.2518 - - - - -
1.6448 370 1.5406 - - - - -
1.6893 380 1.1824 - - - - -
1.7337 390 0.8772 - - - - -
1.7782 400 1.3137 0.4884 0.4824 0.4687 0.4401 0.3959
1.8227 410 1.4667 - - - - -
1.8671 420 1.2337 - - - - -
1.9116 430 1.3271 - - - - -
1.9561 440 1.1768 - - - - -
2.0 450 1.2846 - - - - -
2.0445 460 0.9551 - - - - -
2.0889 470 0.8428 - - - - -
2.1334 480 0.8362 - - - - -
2.1779 490 0.6628 - - - - -
2.2223 500 0.7811 0.4884 0.4798 0.4622 0.4487 0.3936
2.2668 510 1.1356 - - - - -
2.3113 520 0.7177 - - - - -
2.3558 530 0.9575 - - - - -
2.4002 540 0.7495 - - - - -
2.4447 550 0.9872 - - - - -
2.4892 560 0.9362 - - - - -
2.5336 570 0.7413 - - - - -
2.5781 580 0.9984 - - - - -
2.6226 590 0.7781 - - - - -
2.6670 600 0.9545 0.4887 0.4860 0.4685 0.4507 0.4027
2.7115 610 0.7938 - - - - -
2.7560 620 0.7651 - - - - -
2.8004 630 1.1165 - - - - -
2.8449 640 0.9484 - - - - -
2.8894 650 0.9166 - - - - -
2.9339 660 0.8746 - - - - -
2.9783 670 1.1498 - - - - -
3.0222 680 0.7173 - - - - -
3.0667 690 1.0119 - - - - -
3.1112 700 0.7311 0.4970 0.4910 0.4771 0.4534 0.4073
3.1556 710 0.7264 - - - - -
3.2001 720 0.6853 - - - - -
3.2446 730 0.6763 - - - - -
3.2890 740 0.8079 - - - - -
3.3335 750 0.6731 - - - - -
3.3780 760 0.9650 - - - - -
3.4225 770 0.7704 - - - - -
3.4669 780 1.0366 - - - - -
3.5114 790 0.9004 - - - - -
3.5559 800 0.5977 0.4963 0.4885 0.4771 0.4544 0.4090
3.6003 810 0.6151 - - - - -
3.6448 820 0.8600 - - - - -
3.6893 830 0.5171 - - - - -
3.7337 840 0.8487 - - - - -
3.7782 850 0.5347 - - - - -
3.8227 860 0.7020 - - - - -
3.8671 870 0.4907 - - - - -
3.9116 880 0.5780 - - - - -
3.9561 890 0.7386 - - - - -
4.0 900 0.8223 0.4972 0.4891 0.4777 0.4550 0.4100
  • The bold row denotes the saved checkpoint.

Training Time

  • Training: 1.6 hours

Framework Versions

  • Python: 3.12.13
  • Sentence Transformers: 5.7.0
  • Transformers: 5.15.0
  • PyTorch: 2.11.0+cu128
  • Accelerate: 1.14.0
  • Datasets: 4.0.0
  • Tokenizers: 0.22.2

Additional Resources

Citation

BibTeX

Sentence Transformers 

@inproceedings{reimers-2019-sentence-bert,
    title = "Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks",
    author = "Reimers, Nils and Gurevych, Iryna",
    booktitle = "Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing",
    month = "11",
    year = "2019",
    publisher = "Association for Computational Linguistics",
    url = "https://arxiv.org/abs/1908.10084",
}

MatryoshkaLoss 

@misc{kusupati2024matryoshka,
    title={Matryoshka Representation Learning},
    author={Aditya Kusupati and Gantavya Bhatt and Aniket Rege and Matthew Wallingford and Aditya Sinha and Vivek Ramanujan and William Howard-Snyder and Kaifeng Chen and Sham Kakade and Prateek Jain and Ali Farhadi},
    year={2024},
    eprint={2205.13147},
    archivePrefix={arXiv},
    primaryClass={cs.LG}
}

MultipleNegativesRankingLoss 

@misc{oord2019representationlearningcontrastivepredictive,
      title={Representation Learning with Contrastive Predictive Coding},
      author={Aaron van den Oord and Yazhe Li and Oriol Vinyals},
      year={2019},
      eprint={1807.03748},
      archivePrefix={arXiv},
      primaryClass={cs.LG},
      url={https://arxiv.org/abs/1807.03748},
}
Downloads last month
53
Safetensors
Model size
0.1B params
Tensor type
F32
·
Inference Providers NEW
Sentence Similarity
This model isn't deployed by any Inference Provider. 🙋 Ask for provider support

Model tree for vab46/nomic-embed-text-v1.5_Clinical-Trials_Matryoshka

Base model

nomic-ai/nomic-embed-text-v1.5
Finetuned
(38)
this model

Dataset used to train vab46/nomic-embed-text-v1.5_Clinical-Trials_Matryoshka

Papers for vab46/nomic-embed-text-v1.5_Clinical-Trials_Matryoshka

Evaluation results