metadata

language: []
library_name: sentence-transformers
tags:
  - sentence-transformers
  - sentence-similarity
  - feature-extraction
  - dataset_size:100K<n<1M
  - loss:TripletLoss
base_model: nomic-ai/nomic-embed-text-v1.5
metrics:
  - cosine_accuracy
  - dot_accuracy
  - manhattan_accuracy
  - euclidean_accuracy
  - max_accuracy
widget:
  - source_sentence: 'search_query: floral'
    sentences:
      - 'search_query: hair dryer'
      - 'search_query: leporad tumbler'
      - 'search_query: cerrojo sin cerradura'
  - source_sentence: 'search_query: 赤ワイシャツ'
    sentences:
      - 'search_query: sワークス ロードシューズ'
      - 'search_query: ropa astronauta'
      - 'search_query: rosa azul preservada'
  - source_sentence: 'search_query: ギター カポ'
    sentences:
      - 'search_query: カゴバック'
      - 'search_query: midi flowy dress'
      - 'search_query: pesticide sprayer'
  - source_sentence: 'search_query: note 9'
    sentences:
      - 'search_query: samsung s9'
      - 'search_query: wallflower jeans'
      - 'search_query: 12 pomos sin tornillos'
  - source_sentence: 'search_query: 傘 鬼滅の刃'
    sentences:
      - 'search_query: ノースフェイス リュック'
      - 'search_query: お札 を 折ら ない ミニ 財布'
      - 'search_query: buffalo plaid earrings'
pipeline_tag: sentence-similarity
model-index:
  - name: SentenceTransformer based on nomic-ai/nomic-embed-text-v1.5
    results:
      - task:
          type: triplet
          name: Triplet
        dataset:
          name: triplet esci
          type: triplet-esci
        metrics:
          - type: cosine_accuracy
            value: 0.655
            name: Cosine Accuracy
          - type: dot_accuracy
            value: 0.343
            name: Dot Accuracy
          - type: manhattan_accuracy
            value: 0.657
            name: Manhattan Accuracy
          - type: euclidean_accuracy
            value: 0.656
            name: Euclidean Accuracy
          - type: max_accuracy
            value: 0.657
            name: Max Accuracy

SentenceTransformer based on nomic-ai/nomic-embed-text-v1.5

This is a sentence-transformers model finetuned from nomic-ai/nomic-embed-text-v1.5. It maps sentences & paragraphs to a 768-dimensional dense vector space and can be used for semantic textual similarity, semantic search, paraphrase mining, text classification, clustering, and more.

Model Details

Model Description

Model Type: Sentence Transformer
Base model: nomic-ai/nomic-embed-text-v1.5
Maximum Sequence Length: 8192 tokens
Output Dimensionality: 768 tokens
Similarity Function: Cosine Similarity

Model Sources

Documentation: Sentence Transformers Documentation
Repository: Sentence Transformers on GitHub
Hugging Face: Sentence Transformers on Hugging Face

Full Model Architecture

SentenceTransformer(
  (0): Transformer({'max_seq_length': 8192, 'do_lower_case': False}) with Transformer model: NomicBertModel 
  (1): Pooling({'word_embedding_dimension': 768, 'pooling_mode_cls_token': False, 'pooling_mode_mean_tokens': True, 'pooling_mode_max_tokens': False, 'pooling_mode_mean_sqrt_len_tokens': False, 'pooling_mode_weightedmean_tokens': False, 'pooling_mode_lasttoken': False, 'include_prompt': True})
)

Usage

Direct Usage (Sentence Transformers)

First install the Sentence Transformers library:

pip install -U sentence-transformers

Then you can load this model and run inference.

from sentence_transformers import SentenceTransformer

# Download from the 🤗 Hub
model = SentenceTransformer("sentence_transformers_model_id")
# Run inference
sentences = [
    'search_query: 傘 鬼滅の刃',
    'search_query: ノースフェイス リュック',
    'search_query: お札 を 折ら ない ミニ 財布',
]
embeddings = model.encode(sentences)
print(embeddings.shape)
# [3, 768]

# Get the similarity scores for the embeddings
similarities = model.similarity(embeddings, embeddings)
print(similarities.shape)
# [3, 3]

Evaluation

Metrics

Triplet

Dataset: triplet-esci
Evaluated with TripletEvaluator

Metric	Value
cosine_accuracy	0.655
dot_accuracy	0.343
manhattan_accuracy	0.657
euclidean_accuracy	0.656
max_accuracy	0.657

Training Details

Training Dataset

Unnamed Dataset

Size: 100,000 training samples
Columns: anchor, positive, and negative

Approximate statistics based on the first 1000 samples:

	anchor	positive	negative
type	string	string	string
details	min: 7 tokens mean: 12.21 tokens max: 50 tokens	min: 16 tokens mean: 51.18 tokens max: 209 tokens	min: 18 tokens mean: 52.69 tokens max: 175 tokens

Samples:

anchor	positive	negative
`search_query: college cactus backpack`	`search_document: Teecho Waterproof Cute Backpack for Girl Casual Print School Bag Women Laptop Backpack Cactus, Teecho, Cactus`	`search_document: JanSport Huntington Backpack - Lightweight Laptop Bag`
`search_query: yellow laces for sneakers`	`search_document: DELELE Solid Flat Shoelaces Hollow Thick Athletic Shoe Laces Strings Light Yellow 2 Pair 63", DELELE, 05 Light Yellow`	`search_document: Marrywindix 29 Pairs 47" Flat Colourful Athletic Shoe Laces for Sneakers Skate Shoes Boots Sport Shoes (29 Colors), Marrywindix, Colorful`
`search_query: home sign grey`	`search_document: Bigtime Signs Home Sweet Home Sign - 11.75 inch x 9 inch Rigid PVC Signs Decor - Printed Rustic Wood LOOK - Predrilled Hole for Easy Hanging - Family Decoration for Home, Door, Mantle, Porch, Bigtime Signs, Home Sweet Home`	`search_document: Yankario Funny Bathroom Wall Decor Sign, Farmhouse Rustic Bathroom Decorations Wall Art , 12" by 6" Best Seat Wood Plaque, Yankario, grey 1`

Loss: TripletLoss with these parameters:

{
    "distance_metric": "TripletDistanceMetric.EUCLIDEAN",
    "triplet_margin": 5
}

Evaluation Dataset

Unnamed Dataset

Size: 1,000 evaluation samples
Columns: anchor, positive, and negative

Approximate statistics based on the first 1000 samples:

	anchor	positive	negative
type	string	string	string
details	min: 7 tokens mean: 12.24 tokens max: 33 tokens	min: 16 tokens mean: 53.16 tokens max: 173 tokens	min: 13 tokens mean: 53.72 tokens max: 175 tokens

Samples:

anchor	positive	negative
`search_query: black vinyl placemat`	`search_document: Red-A Dining Table Placemats Set of 4 Heat-Resistant Wipeable Table Mats for Kitchen Table Decoration Waterproof Vinyl Placemats Easy to Clean,Black w/Brown, Red-A, Black`	`search_document: Winknowl Placemats, Set of 8 Heat Resistant Stain Resistant Non-Slip Woven Vinyl Insulation Placemats, Washable Durable Elegant Table Mats for Dining (Black), Winknowl, Black`
`search_query: 1 1/2 leather belts without buckle`	`search_document: Vatee's Women's/Men's Real Leather Replacement Belt Strap No Buckle 1 1/2"(38mm) Wide 45" Long Black, Vatee's, 154: Black`	`search_document: Women Skinny Leather Belt Thin Waist Jeans Belt for Pants in Pin Buckle Belt by WHIPPY, Black/Brown, Suit Pants 24-29 Inches, WHIPPY, 2-black+brown`
`search_query: 1x cat 7a conector de red rj45 sin herramientas`	`search_document: deleyCON 3,0m RJ45 Cable Plano Cable de Red de Categoría CAT7 Cable Ethernet U/FTP con Revestimiento Interior de Cobre - Negro, deleyCON, Negro`	`search_document: Conector de Odedo®, 2 unidades, categoría 6, UTP RJ45, para cable de conexión, prolongación de 6,3 mm, AWG 23-26 montaje sin herramientas, contactos dorados, odedo, Weiß`

Loss: TripletLoss with these parameters:

{
    "distance_metric": "TripletDistanceMetric.EUCLIDEAN",
    "triplet_margin": 5
}

Training Hyperparameters

Non-Default Hyperparameters

per_device_train_batch_size: 4
per_device_eval_batch_size: 4
learning_rate: 1e-05
lr_scheduler_type: cosine_with_restarts
warmup_ratio: 0.1
dataloader_drop_last: True
dataloader_num_workers: 4
batch_sampler: no_duplicates

All Hyperparameters

Click to expand

overwrite_output_dir: False
do_predict: False
prediction_loss_only: True
per_device_train_batch_size: 4
per_device_eval_batch_size: 4
per_gpu_train_batch_size: None
per_gpu_eval_batch_size: None
gradient_accumulation_steps: 1
eval_accumulation_steps: None
learning_rate: 1e-05
weight_decay: 0.0
adam_beta1: 0.9
adam_beta2: 0.999
adam_epsilon: 1e-08
max_grad_norm: 1.0
num_train_epochs: 3
max_steps: -1
lr_scheduler_type: cosine_with_restarts
lr_scheduler_kwargs: {}
warmup_ratio: 0.1
warmup_steps: 0
log_level: passive
log_level_replica: warning
log_on_each_node: True
logging_nan_inf_filter: True
save_safetensors: True
save_on_each_node: False
save_only_model: False
no_cuda: False
use_cpu: False
use_mps_device: False
seed: 42
data_seed: None
jit_mode_eval: False
use_ipex: False
bf16: False
fp16: False
fp16_opt_level: O1
half_precision_backend: auto
bf16_full_eval: False
fp16_full_eval: False
tf32: None
local_rank: 0
ddp_backend: None
tpu_num_cores: None
tpu_metrics_debug: False
debug: []
dataloader_drop_last: True
dataloader_num_workers: 4
dataloader_prefetch_factor: None
past_index: -1
disable_tqdm: False
remove_unused_columns: True
label_names: None
load_best_model_at_end: False
ignore_data_skip: False
fsdp: []
fsdp_min_num_params: 0
fsdp_config: {'min_num_params': 0, 'xla': False, 'xla_fsdp_v2': False, 'xla_fsdp_grad_ckpt': False}
fsdp_transformer_layer_cls_to_wrap: None
accelerator_config: {'split_batches': False, 'dispatch_batches': None, 'even_batches': True, 'use_seedable_sampler': True}
deepspeed: None
label_smoothing_factor: 0.0
optim: adamw_torch
optim_args: None
adafactor: False
group_by_length: False
length_column_name: length
ddp_find_unused_parameters: None
ddp_bucket_cap_mb: None
ddp_broadcast_buffers: False
dataloader_pin_memory: True
dataloader_persistent_workers: False
skip_memory_metrics: True
use_legacy_prediction_loop: False
push_to_hub: False
resume_from_checkpoint: None
hub_model_id: None
hub_strategy: every_save
hub_private_repo: False
hub_always_push: False
gradient_checkpointing: False
gradient_checkpointing_kwargs: None
include_inputs_for_metrics: False
fp16_backend: auto
push_to_hub_model_id: None
push_to_hub_organization: None
mp_parameters:
auto_find_batch_size: False
full_determinism: False
torchdynamo: None
ray_scope: last
ddp_timeout: 1800
torch_compile: False
torch_compile_backend: None
torch_compile_mode: None
dispatch_batches: None
split_batches: None
include_tokens_per_second: False
include_num_input_tokens_seen: False
neftune_noise_alpha: None
batch_sampler: no_duplicates
multi_dataset_batch_sampler: proportional

Training Logs

Click to expand

Epoch	Step	Training Loss	loss	triplet-esci_cosine_accuracy
0.008	200	3.9805	-	-
0.016	400	4.0739	-	-
0.024	600	4.0571	-	-
0.032	800	3.8848	-	-
0.04	1000	3.8249	3.9825	0.664
0.048	1200	3.7097	-	-
0.056	1400	3.6869	-	-
0.064	1600	3.4327	-	-
0.072	1800	3.64	-	-
0.08	2000	3.3813	3.8244	0.657
0.088	2200	3.4011	-	-
0.096	2400	3.34	-	-
0.104	2600	3.2488	-	-
0.112	2800	3.5031	-	-
0.12	3000	3.3615	3.7263	0.674
0.128	3200	3.1028	-	-
0.136	3400	3.2969	-	-
0.144	3600	3.0463	-	-
0.152	3800	3.1194	-	-
0.16	4000	3.2372	3.6599	0.673
0.168	4200	3.2954	-	-
0.176	4400	3.2753	-	-
0.184	4600	3.179	-	-
0.192	4800	3.2646	-	-
0.2	5000	3.1295	3.6405	0.677
0.208	5200	3.2211	-	-
0.216	5400	3.2222	-	-
0.224	5600	2.9471	-	-
0.232	5800	3.1564	-	-
0.24	6000	3.1099	3.6138	0.684
0.248	6200	2.9399	-	-
0.256	6400	3.1087	-	-
0.264	6600	3.2675	-	-
0.272	6800	3.2149	-	-
0.28	7000	2.9484	3.6086	0.673
0.288	7200	3.0829	-	-
0.296	7400	3.1864	-	-
0.304	7600	3.1201	-	-
0.312	7800	3.0698	-	-
0.32	8000	2.9968	3.5750	0.668
0.328	8200	3.0636	-	-
0.336	8400	3.1293	-	-
0.344	8600	3.1282	-	-
0.352	8800	3.1415	-	-
0.36	9000	2.7868	3.5211	0.693
0.368	9200	2.9714	-	-
0.376	9400	2.9409	-	-
0.384	9600	2.9071	-	-
0.392	9800	2.9154	-	-
0.4	10000	2.9709	3.5510	0.683
0.408	10200	2.741	-	-
0.416	10400	2.678	-	-
0.424	10600	2.8429	-	-
0.432	10800	2.9782	-	-
0.44	11000	2.9548	3.5369	0.681
0.448	11200	2.8452	-	-
0.456	11400	2.8578	-	-
0.464	11600	2.9211	-	-
0.472	11800	2.897	-	-
0.48	12000	2.7651	3.6031	0.687
0.488	12200	2.9472	-	-
0.496	12400	2.6198	-	-
0.504	12600	2.8444	-	-
0.512	12800	2.7384	-	-
0.52	13000	2.7827	3.6082	0.68
0.528	13200	2.6882	-	-
0.536	13400	2.6722	-	-
0.544	13600	2.7218	-	-
0.552	13800	2.7278	-	-
0.56	14000	2.7156	3.5606	0.677
0.568	14200	2.5748	-	-
0.576	14400	2.5414	-	-
0.584	14600	2.6918	-	-
0.592	14800	2.5429	-	-
0.6	15000	2.5724	3.5178	0.694
0.608	15200	2.7594	-	-
0.616	15400	2.4841	-	-
0.624	15600	2.4667	-	-
0.632	15800	2.6253	-	-
0.64	16000	2.5001	3.5428	0.683
0.648	16200	2.5707	-	-
0.656	16400	2.4924	-	-
0.664	16600	2.5419	-	-
0.672	16800	2.487	-	-
0.68	17000	2.4747	3.5825	0.681
0.688	17200	2.4194	-	-
0.696	17400	2.5714	-	-
0.704	17600	2.4069	-	-
0.712	17800	2.5767	-	-
0.72	18000	2.5952	3.6268	0.684
0.728	18200	2.4023	-	-
0.736	18400	2.3989	-	-
0.744	18600	2.379	-	-
0.752	18800	2.4943	-	-
0.76	19000	2.3762	3.5686	0.701
0.768	19200	2.4825	-	-
0.776	19400	2.4451	-	-
0.784	19600	2.5374	-	-
0.792	19800	2.4569	-	-
0.8	20000	2.2353	3.6429	0.681
0.808	20200	2.3447	-	-
0.816	20400	2.3083	-	-
0.824	20600	2.2126	-	-
0.832	20800	2.3935	-	-
0.84	21000	2.5115	3.6387	0.68
0.848	21200	2.1469	-	-
0.856	21400	2.2717	-	-
0.864	21600	2.2993	-	-
0.872	21800	2.3519	-	-
0.88	22000	2.2947	3.5908	0.671
0.888	22200	2.3249	-	-
0.896	22400	2.2452	-	-
0.904	22600	2.114	-	-
0.912	22800	2.208	-	-
0.92	23000	2.4168	3.6659	0.671
0.928	23200	2.2098	-	-
0.936	23400	2.1805	-	-
0.944	23600	2.122	-	-
0.952	23800	2.1364	-	-
0.96	24000	2.1464	3.6284	0.671
0.968	24200	2.1298	-	-
0.976	24400	2.2657	-	-
0.984	24600	2.304	-	-
0.992	24800	2.175	-	-
1.0	25000	2.1349	3.6532	0.681
1.008	25200	2.0151	-	-
1.016	25400	2.0881	-	-
1.024	25600	1.9897	-	-
1.032	25800	2.1987	-	-
1.04	26000	1.9913	3.6511	0.672
1.048	26200	1.9088	-	-
1.056	26400	1.9555	-	-
1.064	26600	1.6892	-	-
1.072	26800	2.0404	-	-
1.08	27000	1.6976	3.6465	0.671
1.088	27200	1.894	-	-
1.096	27400	1.8056	-	-
1.104	27600	1.6426	-	-
1.112	27800	2.0203	-	-
1.12	28000	1.697	3.6182	0.681
1.1280	28200	1.5562	-	-
1.1360	28400	1.6061	-	-
1.144	28600	1.5201	-	-
1.152	28800	1.4388	-	-
1.16	29000	1.5198	3.5924	0.676
1.168	29200	1.6404	-	-
1.176	29400	1.6058	-	-
1.184	29600	1.6063	-	-
1.192	29800	1.4398	-	-
1.2	30000	1.4952	3.6587	0.665
1.208	30200	1.5077	-	-
1.216	30400	1.3177	-	-
1.224	30600	1.095	-	-
1.232	30800	1.2841	-	-
1.24	31000	1.3544	3.6066	0.684
1.248	31200	1.2188	-	-
1.256	31400	1.1761	-	-
1.264	31600	1.2601	-	-
1.272	31800	1.2057	-	-
1.28	32000	1.0478	3.6371	0.681
1.288	32200	1.0888	-	-
1.296	32400	1.1335	-	-
1.304	32600	1.1297	-	-
1.312	32800	1.0302	-	-
1.32	33000	1.0583	3.6186	0.685
1.328	33200	1.0623	-	-
1.336	33400	0.9047	-	-
1.3440	33600	1.0706	-	-
1.3520	33800	1.0877	-	-
1.3600	34000	0.8205	3.6613	0.653
1.3680	34200	0.9596	-	-
1.376	34400	0.8855	-	-
1.384	34600	0.9186	-	-
1.392	34800	0.8087	-	-
1.4	35000	0.9732	3.6558	0.662
1.408	35200	0.8753	-	-
1.416	35400	0.8257	-	-
1.424	35600	0.8689	-	-
1.432	35800	0.8596	-	-
1.44	36000	0.9202	3.6872	0.66
1.448	36200	0.8993	-	-
1.456	36400	0.8889	-	-
1.464	36600	0.9138	-	-
1.472	36800	0.8212	-	-
1.48	37000	0.7591	3.6708	0.666
1.488	37200	0.769	-	-
1.496	37400	0.8656	-	-
1.504	37600	0.9134	-	-
1.512	37800	0.7212	-	-
1.52	38000	0.8118	3.6249	0.672
1.528	38200	0.7454	-	-
1.536	38400	0.7491	-	-
1.544	38600	0.8148	-	-
1.552	38800	0.6845	-	-
1.56	39000	0.6169	3.6868	0.679
1.568	39200	0.7377	-	-
1.576	39400	0.7296	-	-
1.584	39600	0.7204	-	-
1.592	39800	0.6748	-	-
1.6	40000	0.6494	3.7054	0.673
1.608	40200	0.7435	-	-
1.616	40400	0.6196	-	-
1.624	40600	0.6977	-	-
1.6320	40800	0.7442	-	-
1.6400	41000	0.5824	3.7500	0.66
1.6480	41200	0.6144	-	-
1.6560	41400	0.5909	-	-
1.6640	41600	0.6717	-	-
1.6720	41800	0.6436	-	-
1.6800	42000	0.6161	3.6769	0.676
1.688	42200	0.5282	-	-
1.696	42400	0.6647	-	-
1.704	42600	0.649	-	-
1.712	42800	0.6284	-	-
1.72	43000	0.7055	3.6992	0.671
1.728	43200	0.6249	-	-
1.736	43400	0.5722	-	-
1.744	43600	0.621	-	-
1.752	43800	0.6129	-	-
1.76	44000	0.501	3.7589	0.662
1.768	44200	0.5566	-	-
1.776	44400	0.576	-	-
1.784	44600	0.6428	-	-
1.792	44800	0.5629	-	-
1.8	45000	0.5134	3.7530	0.659
1.808	45200	0.4681	-	-
1.8160	45400	0.6268	-	-
1.8240	45600	0.5877	-	-
1.8320	45800	0.5219	-	-
1.8400	46000	0.545	3.7755	0.658
1.8480	46200	0.4539	-	-
1.8560	46400	0.5255	-	-
1.8640	46600	0.5573	-	-
1.8720	46800	0.5508	-	-
1.88	47000	0.5391	3.7489	0.653
1.888	47200	0.4276	-	-
1.896	47400	0.4906	-	-
1.904	47600	0.3771	-	-
1.912	47800	0.4959	-	-
1.92	48000	0.5377	3.7770	0.658
1.928	48200	0.4807	-	-
1.936	48400	0.5239	-	-
1.944	48600	0.4441	-	-
1.952	48800	0.4536	-	-
1.96	49000	0.5265	3.7507	0.669
1.968	49200	0.3817	-	-
1.976	49400	0.4468	-	-
1.984	49600	0.5766	-	-
1.992	49800	0.4789	-	-
2.0	50000	0.4853	3.7328	0.663
2.008	50200	0.3744	-	-
2.016	50400	0.4662	-	-
2.024	50600	0.394	-	-
2.032	50800	0.3938	-	-
2.04	51000	0.3586	3.8004	0.656
2.048	51200	0.3522	-	-
2.056	51400	0.4173	-	-
2.064	51600	0.3177	-	-
2.072	51800	0.4113	-	-
2.08	52000	0.3027	3.7366	0.665
2.088	52200	0.3693	-	-
2.096	52400	0.4268	-	-
2.104	52600	0.3678	-	-
2.112	52800	0.4192	-	-
2.12	53000	0.3105	3.7831	0.661
2.128	53200	0.3228	-	-
2.136	53400	0.2408	-	-
2.144	53600	0.2804	-	-
2.152	53800	0.3034	-	-
2.16	54000	0.3562	3.7866	0.656
2.168	54200	0.3526	-	-
2.176	54400	0.414	-	-
2.184	54600	0.3678	-	-
2.192	54800	0.2965	-	-
2.2	55000	0.3691	3.8108	0.655

Framework Versions

Python: 3.10.12
Sentence Transformers: 3.0.0
Transformers: 4.38.2
PyTorch: 2.1.2+cu121
Accelerate: 0.27.2
Datasets: 2.19.1
Tokenizers: 0.15.2

Citation

BibTeX

Sentence Transformers

@inproceedings{reimers-2019-sentence-bert,
    title = "Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks",
    author = "Reimers, Nils and Gurevych, Iryna",
    booktitle = "Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing",
    month = "11",
    year = "2019",
    publisher = "Association for Computational Linguistics",
    url = "https://arxiv.org/abs/1908.10084",
}

TripletLoss

@misc{hermans2017defense,
    title={In Defense of the Triplet Loss for Person Re-Identification}, 
    author={Alexander Hermans and Lucas Beyer and Bastian Leibe},
    year={2017},
    eprint={1703.07737},
    archivePrefix={arXiv},
    primaryClass={cs.CV}
}