---
language: []
library_name: sentence-transformers
tags:
- sentence-transformers
- sentence-similarity
- feature-extraction
- dataset_size:1M<n<10M
- loss:CachedGISTEmbedLoss
base_model: nomic-ai/nomic-embed-text-v1.5
widget:
- source_sentence: 'search_query: ipad'
  sentences:
  - 'search_query: ipad newest model'
  - 'search_query: epyc amd cpu'
  - 'search_query: 35x29 mens jeans'
- source_sentence: 'search_query: 掲示板'
  sentences:
  - 'search_query: 寝袋 冬'
  - 'search_query: lg ゲーミングモニター'
  - 'search_query: vga to dvi-d'
- source_sentence: 'search_query: ダイアン'
  sentences:
  - 'search_query: ゴアテックス レインウェア'
  - 'search_query: roy vickery'
  - 'search_query: cbd oil not hemp'
- source_sentence: 'search_query: psp'
  sentences:
  - 'search_query: petechtool'
  - 'search_query: heavy cream'
  - 'search_query: sweaters for women'
- source_sentence: 'search_query: 龍師の翼'
  sentences:
  - 'search_query: 烏龍茶 2l'
  - 'search_query: 筆箱プラスチックケース'
  - 'search_query: fitbit for kids'
pipeline_tag: sentence-similarity
---

# SentenceTransformer based on nomic-ai/nomic-embed-text-v1.5

This is a [sentence-transformers](https://www.SBERT.net) model finetuned from [nomic-ai/nomic-embed-text-v1.5](https://huggingface.co/nomic-ai/nomic-embed-text-v1.5). It maps sentences & paragraphs to a 768-dimensional dense vector space and can be used for semantic textual similarity, semantic search, paraphrase mining, text classification, clustering, and more.

## Model Details

### Model Description
- **Model Type:** Sentence Transformer
- **Base model:** [nomic-ai/nomic-embed-text-v1.5](https://huggingface.co/nomic-ai/nomic-embed-text-v1.5) <!-- at revision 91d2d6bfdddf0b0da840f901b533e99bae30d757 -->
- **Maximum Sequence Length:** 8192 tokens
- **Output Dimensionality:** 768 tokens
- **Similarity Function:** Cosine Similarity
<!-- - **Training Dataset:** Unknown -->
<!-- - **Language:** Unknown -->
<!-- - **License:** Unknown -->

### Model Sources

- **Documentation:** [Sentence Transformers Documentation](https://sbert.net)
- **Repository:** [Sentence Transformers on GitHub](https://github.com/UKPLab/sentence-transformers)
- **Hugging Face:** [Sentence Transformers on Hugging Face](https://huggingface.co/models?library=sentence-transformers)

### Full Model Architecture

```
SentenceTransformer(
  (0): Transformer({'max_seq_length': 8192, 'do_lower_case': False}) with Transformer model: NomicBertModel 
  (1): Pooling({'word_embedding_dimension': 768, 'pooling_mode_cls_token': False, 'pooling_mode_mean_tokens': True, 'pooling_mode_max_tokens': False, 'pooling_mode_mean_sqrt_len_tokens': False, 'pooling_mode_weightedmean_tokens': False, 'pooling_mode_lasttoken': False, 'include_prompt': True})
)
```

## Usage

### Direct Usage (Sentence Transformers)

First install the Sentence Transformers library:

```bash
pip install -U sentence-transformers
```

Then you can load this model and run inference.
```python
from sentence_transformers import SentenceTransformer

# Download from the 🤗 Hub
model = SentenceTransformer("sentence_transformers_model_id")
# Run inference
sentences = [
    'search_query: 龍師の翼',
    'search_query: 烏龍茶 2l',
    'search_query: 筆箱プラスチックケース',
]
embeddings = model.encode(sentences)
print(embeddings.shape)
# [3, 768]

# Get the similarity scores for the embeddings
similarities = model.similarity(embeddings, embeddings)
print(similarities.shape)
# [3, 3]
```

<!--
### Direct Usage (Transformers)

<details><summary>Click to see the direct usage in Transformers</summary>

</details>
-->

<!--
### Downstream Usage (Sentence Transformers)

You can finetune this model on your own dataset.

<details><summary>Click to expand</summary>

</details>
-->

<!--
### Out-of-Scope Use

*List how the model may foreseeably be misused and address what users ought not to do with the model.*
-->

<!--
## Bias, Risks and Limitations

*What are the known or foreseeable issues stemming from this model? You could also flag here known failure cases or weaknesses of the model.*
-->

<!--
### Recommendations

*What are recommendations with respect to the foreseeable issues? For example, filtering explicit content.*
-->

## Training Details

### Training Dataset

#### Unnamed Dataset


* Size: 1,767,572 training samples
* Columns: <code>anchor</code>, <code>positive</code>, and <code>negative</code>
* Approximate statistics based on the first 1000 samples:
  |         | anchor                                                                            | positive                                                                           | negative                                                                          |
  |:--------|:----------------------------------------------------------------------------------|:-----------------------------------------------------------------------------------|:----------------------------------------------------------------------------------|
  | type    | string                                                                            | string                                                                             | string                                                                            |
  | details | <ul><li>min: 7 tokens</li><li>mean: 12.26 tokens</li><li>max: 59 tokens</li></ul> | <ul><li>min: 7 tokens</li><li>mean: 31.93 tokens</li><li>max: 140 tokens</li></ul> | <ul><li>min: 7 tokens</li><li>mean: 34.3 tokens</li><li>max: 157 tokens</li></ul> |
* Samples:
  | anchor                                                   | positive                                                                                                                                      | negative                                                                                                                                        |
  |:---------------------------------------------------------|:----------------------------------------------------------------------------------------------------------------------------------------------|:------------------------------------------------------------------------------------------------------------------------------------------------|
  | <code>search_query: plus tops for women</code>           | <code>search_document:Just My Size Women's Plus-Size Graphic Short Sleeve V-Neck T-Shirt, White-Y07188, 5X, JUST MY SIZE, White-y07188</code> | <code>search_document:Calvin Klein Women's Regular Modern Cotton Bralette, Nymph's Thigh, S, Calvin Klein, Nymph's Thigh</code>                 |
  | <code>search_query: mens black wallet trifold</code>     | <code>search_document:Stealth Mode Trifold RFID Blocking Leather Wallet for Men (Black), Stealth Mode, Black</code>                           | <code>search_document:RFID Trifold Canvas Outdoor Sports Wallet for Kids - Front Pocket Wallet with Magic Sticker (Black), AI-DEE, Black</code> |
  | <code>search_query: ipad pro reacondicionado 12,9</code> | <code>search_document:Apple iPad Pro (12.9 Pouces, Wi-FI, 64Go) 2018 - Gray (Renewed), Apple, Gris Espacial</code>                            | <code>search_document:Apple iPad Pro 3rd Generation (11-Inch, Wi-FI Only 64GB) - Space Gray (Renewed), Apple, Gris Espacial</code>              |
* Loss: [<code>CachedGISTEmbedLoss</code>](https://sbert.net/docs/package_reference/sentence_transformer/losses.html#cachedgistembedloss) with these parameters:
  ```json
  {'guide': SentenceTransformer(
    (0): Transformer({'max_seq_length': 256, 'do_lower_case': False}) with Transformer model: BertModel 
    (1): Pooling({'word_embedding_dimension': 384, 'pooling_mode_cls_token': False, 'pooling_mode_mean_tokens': True, 'pooling_mode_max_tokens': False, 'pooling_mode_mean_sqrt_len_tokens': False, 'pooling_mode_weightedmean_tokens': False, 'pooling_mode_lasttoken': False, 'include_prompt': True})
    (2): Normalize()
  ), 'temperature': 0.01}
  ```

### Training Hyperparameters
#### Non-Default Hyperparameters

- `per_device_eval_batch_size`: 2
- `gradient_accumulation_steps`: 2
- `learning_rate`: 1e-05
- `lr_scheduler_type`: cosine_with_restarts
- `warmup_ratio`: 0.1
- `fp16`: True
- `batch_sampler`: no_duplicates

#### All Hyperparameters
<details><summary>Click to expand</summary>

- `overwrite_output_dir`: False
- `do_predict`: False
- `prediction_loss_only`: True
- `per_device_train_batch_size`: 8
- `per_device_eval_batch_size`: 2
- `per_gpu_train_batch_size`: None
- `per_gpu_eval_batch_size`: None
- `gradient_accumulation_steps`: 2
- `eval_accumulation_steps`: None
- `learning_rate`: 1e-05
- `weight_decay`: 0.0
- `adam_beta1`: 0.9
- `adam_beta2`: 0.999
- `adam_epsilon`: 1e-08
- `max_grad_norm`: 1.0
- `num_train_epochs`: 3
- `max_steps`: -1
- `lr_scheduler_type`: cosine_with_restarts
- `lr_scheduler_kwargs`: {}
- `warmup_ratio`: 0.1
- `warmup_steps`: 0
- `log_level`: passive
- `log_level_replica`: warning
- `log_on_each_node`: True
- `logging_nan_inf_filter`: True
- `save_safetensors`: True
- `save_on_each_node`: False
- `save_only_model`: False
- `no_cuda`: False
- `use_cpu`: False
- `use_mps_device`: False
- `seed`: 42
- `data_seed`: None
- `jit_mode_eval`: False
- `use_ipex`: False
- `bf16`: False
- `fp16`: True
- `fp16_opt_level`: O1
- `half_precision_backend`: auto
- `bf16_full_eval`: False
- `fp16_full_eval`: False
- `tf32`: None
- `local_rank`: 0
- `ddp_backend`: None
- `tpu_num_cores`: None
- `tpu_metrics_debug`: False
- `debug`: []
- `dataloader_drop_last`: False
- `dataloader_num_workers`: 0
- `dataloader_prefetch_factor`: None
- `past_index`: -1
- `disable_tqdm`: False
- `remove_unused_columns`: True
- `label_names`: None
- `load_best_model_at_end`: False
- `ignore_data_skip`: False
- `fsdp`: []
- `fsdp_min_num_params`: 0
- `fsdp_config`: {'min_num_params': 0, 'xla': False, 'xla_fsdp_v2': False, 'xla_fsdp_grad_ckpt': False}
- `fsdp_transformer_layer_cls_to_wrap`: None
- `accelerator_config`: {'split_batches': False, 'dispatch_batches': None, 'even_batches': True, 'use_seedable_sampler': True}
- `deepspeed`: None
- `label_smoothing_factor`: 0.0
- `optim`: adamw_torch
- `optim_args`: None
- `adafactor`: False
- `group_by_length`: False
- `length_column_name`: length
- `ddp_find_unused_parameters`: None
- `ddp_bucket_cap_mb`: None
- `ddp_broadcast_buffers`: False
- `dataloader_pin_memory`: True
- `dataloader_persistent_workers`: False
- `skip_memory_metrics`: True
- `use_legacy_prediction_loop`: False
- `push_to_hub`: False
- `resume_from_checkpoint`: None
- `hub_model_id`: None
- `hub_strategy`: every_save
- `hub_private_repo`: False
- `hub_always_push`: False
- `gradient_checkpointing`: False
- `gradient_checkpointing_kwargs`: None
- `include_inputs_for_metrics`: False
- `fp16_backend`: auto
- `push_to_hub_model_id`: None
- `push_to_hub_organization`: None
- `mp_parameters`: 
- `auto_find_batch_size`: False
- `full_determinism`: False
- `torchdynamo`: None
- `ray_scope`: last
- `ddp_timeout`: 1800
- `torch_compile`: False
- `torch_compile_backend`: None
- `torch_compile_mode`: None
- `dispatch_batches`: None
- `split_batches`: None
- `include_tokens_per_second`: False
- `include_num_input_tokens_seen`: False
- `neftune_noise_alpha`: None
- `batch_sampler`: no_duplicates
- `multi_dataset_batch_sampler`: proportional

</details>

### Training Logs
<details><summary>Click to expand</summary>

| Epoch  | Step  | Training Loss |
|:------:|:-----:|:-------------:|
| 0.0009 | 100   | 3.7009        |
| 0.0018 | 200   | 3.3796        |
| 0.0027 | 300   | 2.8348        |
| 0.0036 | 400   | 2.1803        |
| 0.0045 | 500   | 1.8272        |
| 0.0054 | 600   | 1.4715        |
| 0.0063 | 700   | 1.0056        |
| 0.0072 | 800   | 0.7984        |
| 0.0081 | 900   | 0.6925        |
| 0.0091 | 1000  | 0.6552        |
| 0.0100 | 1100  | 0.6054        |
| 0.0109 | 1200  | 0.5874        |
| 0.0118 | 1300  | 0.5641        |
| 0.0127 | 1400  | 0.528         |
| 0.0136 | 1500  | 0.5285        |
| 0.0145 | 1600  | 0.5032        |
| 0.0154 | 1700  | 0.5238        |
| 0.0163 | 1800  | 0.4565        |
| 0.0172 | 1900  | 0.4739        |
| 0.0181 | 2000  | 0.4614        |
| 0.0190 | 2100  | 0.4334        |
| 0.0199 | 2200  | 0.4217        |
| 0.0208 | 2300  | 0.3931        |
| 0.0217 | 2400  | 0.4102        |
| 0.0226 | 2500  | 0.3936        |
| 0.0235 | 2600  | 0.415         |
| 0.0244 | 2700  | 0.4462        |
| 0.0253 | 2800  | 0.3886        |
| 0.0263 | 2900  | 0.3887        |
| 0.0272 | 3000  | 0.3629        |
| 0.0281 | 3100  | 0.37          |
| 0.0290 | 3200  | 0.3861        |
| 0.0299 | 3300  | 0.3813        |
| 0.0308 | 3400  | 0.3348        |
| 0.0317 | 3500  | 0.3566        |
| 0.0326 | 3600  | 0.3674        |
| 0.0335 | 3700  | 0.3421        |
| 0.0344 | 3800  | 0.3225        |
| 0.0353 | 3900  | 0.406         |
| 0.0362 | 4000  | 0.3975        |
| 0.0371 | 4100  | 0.368         |
| 0.0380 | 4200  | 0.3481        |
| 0.0389 | 4300  | 0.3405        |
| 0.0398 | 4400  | 0.3529        |
| 0.0407 | 4500  | 0.3968        |
| 0.0416 | 4600  | 0.3634        |
| 0.0425 | 4700  | 0.3518        |
| 0.0434 | 4800  | 0.383         |
| 0.0444 | 4900  | 0.3261        |
| 0.0453 | 5000  | 0.323         |
| 0.0462 | 5100  | 0.3372        |
| 0.0471 | 5200  | 0.358         |
| 0.0480 | 5300  | 0.3207        |
| 0.0489 | 5400  | 0.341         |
| 0.0498 | 5500  | 0.3146        |
| 0.0507 | 5600  | 0.3065        |
| 0.0516 | 5700  | 0.3597        |
| 0.0525 | 5800  | 0.3352        |
| 0.0534 | 5900  | 0.3212        |
| 0.0543 | 6000  | 0.316         |
| 0.0552 | 6100  | 0.3405        |
| 0.0561 | 6200  | 0.3416        |
| 0.0570 | 6300  | 0.3124        |
| 0.0579 | 6400  | 0.3146        |
| 0.0588 | 6500  | 0.3043        |
| 0.0597 | 6600  | 0.3687        |
| 0.0606 | 6700  | 0.3359        |
| 0.0616 | 6800  | 0.3414        |
| 0.0625 | 6900  | 0.3161        |
| 0.0634 | 7000  | 0.3266        |
| 0.0643 | 7100  | 0.3465        |
| 0.0652 | 7200  | 0.3017        |
| 0.0661 | 7300  | 0.33          |
| 0.0670 | 7400  | 0.3324        |
| 0.0679 | 7500  | 0.2829        |
| 0.0688 | 7600  | 0.2691        |
| 0.0697 | 7700  | 0.2939        |
| 0.0706 | 7800  | 0.2769        |
| 0.0715 | 7900  | 0.3092        |
| 0.0724 | 8000  | 0.2761        |
| 0.0733 | 8100  | 0.2858        |
| 0.0742 | 8200  | 0.3667        |
| 0.0751 | 8300  | 0.304         |
| 0.0760 | 8400  | 0.2971        |
| 0.0769 | 8500  | 0.3284        |
| 0.0778 | 8600  | 0.2915        |
| 0.0788 | 8700  | 0.2903        |
| 0.0797 | 8800  | 0.3132        |
| 0.0806 | 8900  | 0.2999        |
| 0.0815 | 9000  | 0.3191        |
| 0.0824 | 9100  | 0.2846        |
| 0.0833 | 9200  | 0.2966        |
| 0.0842 | 9300  | 0.3075        |
| 0.0851 | 9400  | 0.2814        |
| 0.0860 | 9500  | 0.3161        |
| 0.0869 | 9600  | 0.3056        |
| 0.0878 | 9700  | 0.2874        |
| 0.0887 | 9800  | 0.3238        |
| 0.0896 | 9900  | 0.3018        |
| 0.0905 | 10000 | 0.2994        |
| 0.0914 | 10100 | 0.2794        |
| 0.0923 | 10200 | 0.2581        |
| 0.0932 | 10300 | 0.3097        |
| 0.0941 | 10400 | 0.2711        |
| 0.0950 | 10500 | 0.2428        |
| 0.0960 | 10600 | 0.3025        |
| 0.0969 | 10700 | 0.3062        |
| 0.0978 | 10800 | 0.3189        |
| 0.0987 | 10900 | 0.3237        |
| 0.0996 | 11000 | 0.2843        |
| 0.1005 | 11100 | 0.2671        |
| 0.1014 | 11200 | 0.2939        |
| 0.1023 | 11300 | 0.2976        |
| 0.1032 | 11400 | 0.2991        |
| 0.1041 | 11500 | 0.2456        |
| 0.1050 | 11600 | 0.2895        |
| 0.1059 | 11700 | 0.2743        |
| 0.1068 | 11800 | 0.2827        |
| 0.1077 | 11900 | 0.3276        |
| 0.1086 | 12000 | 0.2813        |

</details>

### Framework Versions
- Python: 3.10.12
- Sentence Transformers: 3.0.0
- Transformers: 4.38.2
- PyTorch: 2.1.2+cu121
- Accelerate: 0.27.2
- Datasets: 2.19.1
- Tokenizers: 0.15.2

## Citation

### BibTeX

#### Sentence Transformers
```bibtex
@inproceedings{reimers-2019-sentence-bert,
    title = "Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks",
    author = "Reimers, Nils and Gurevych, Iryna",
    booktitle = "Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing",
    month = "11",
    year = "2019",
    publisher = "Association for Computational Linguistics",
    url = "https://arxiv.org/abs/1908.10084",
}
```

<!--
## Glossary

*Clearly define terms in order to be accessible across audiences.*
-->

<!--
## Model Card Authors

*Lists the people who create the model card, providing recognition and accountability for the detailed work that goes into its construction.*
-->

<!--
## Model Card Contact

*Provides a way for people who have updates to the Model Card, suggestions, or questions, to contact the Model Card authors.*
-->