SentenceTransformer based on liudev/novel-embedding-gpl-base-v1

This is a sentence-transformers model finetuned from liudev/novel-embedding-gpl-base-v1. It maps sentences & paragraphs to a 768-dimensional dense vector space and can be used for semantic textual similarity, semantic search, paraphrase mining, classification, clustering, and more.

Model Details

Model Description

Model Type: Sentence Transformer
Base model: liudev/novel-embedding-gpl-base-v1
Maximum Sequence Length: 256 tokens
Output Dimensionality: 768 dimensions
Similarity Function: Cosine Similarity
Supported Modality: Text

Model Sources

Documentation: Sentence Transformers Documentation
Repository: Sentence Transformers on GitHub
Hugging Face: Sentence Transformers on Hugging Face

Full Model Architecture

SentenceTransformer(
  (0): Transformer({'transformer_task': 'feature-extraction', 'modality_config': {'text': {'method': 'forward', 'method_output_name': 'last_hidden_state'}}, 'module_output_name': 'token_embeddings', 'architecture': 'BertModel'})
  (1): Pooling({'embedding_dimension': 768, 'pooling_mode': 'cls', 'include_prompt': True})
  (2): Normalize({})
)

Usage

Direct Usage (Sentence Transformers)

First install the Sentence Transformers library:

pip install -U sentence-transformers

Then you can load this model and run inference.

from sentence_transformers import SentenceTransformer

# Download from the 🤗 Hub
model = SentenceTransformer("liudev/novel-embedding-gpl-base-v2")
# Run inference
sentences = [
    '在众多好友和大众的见证下举行盛大婚礼',
    '升官后搬进官舍',
    '长辈私下质疑晚辈文章的真实出处',
    '获得强力群攻技能后快速练级',
]
embeddings = model.encode(sentences)
print(embeddings.shape)
# [4, 768]

# Get the similarity scores for the embeddings
similarities = model.similarity(embeddings, embeddings)
print(similarities.shape)
# [4, 4]

Training Details

Training Dataset

Unnamed Dataset

Size: 435,262 training samples
Columns: anchor, sentence1, sentence2, and label

Approximate statistics based on the first 100 samples:

	anchor	sentence1	sentence2	label
type	string	string	string	float
modality	text	text	text
details	min: 10 tokens mean: 16.4 tokens max: 24 tokens	min: 48 tokens mean: 73.12 tokens max: 102 tokens	min: 15 tokens mean: 72.16 tokens max: 97 tokens	min: 0.0 mean: 0.3 max: 0.8

Samples:

anchor	sentence1	sentence2	label
`同伴意外获得逆天机缘修为暴涨`	`林辰开启系统新手大礼包获得逆天机缘，在利用生命泉水恢复伤势后，通过融合洪荒先天道体本源，不仅大幅提升了资质，更将修为一举推至蜕凡境四重，开启了低调发展家族的崛起之路。`	`周通拜师蜀山派遭羞辱驱逐，在对修仙绝望之际，意外撞见一名蜀山弟子被魔道中人追杀，身怀“偷天手”外挂的他意识到逆天改命的契机到了。`	`0.10039120339592345`
`选择留在原处暗中积蓄力量`	`洛兰希尔在枫叶林中通过改良作物与手工劳作充实生活，在权衡利弊后，她决定暂时搁置与外界接触的计划，继续保持低调隐居以沉淀实力。`	`夏炎在面对白雨陌被迫离去及太虚仙宫企图通过傀儡婚姻掌控皇权的阴谋时，深夜在湖心施展禁忌的“纸人控灵术”凝聚百鬼之力，开启了暗中积蓄力量的反抗之路。`	`0.021916571743932134`
`获得大笔钱后豪掷千金买豪车送给别人`	`李鲤在拒绝拜金女的挑衅后，前往保时捷4S店豪掷重金购入多款跑车，并为兄弟赵峰送上巨额婚庆贺礼，尽显其暴富后的快意生活与重义性格。`	`农村女孩因持旧币无法购票而当众“洗钱”，在引发楚卓然的疑虑与好奇后，最终因其天然朴素的举动获得对方同情，并受邀搭乘其豪车前往CMFU学园。`	`0.626574703727987`

Loss: MarginMSELoss with these parameters:

{
    "similarity_fct": "pairwise_dot_score"
}

Training Hyperparameters

Non-Default Hyperparameters

per_device_train_batch_size: 32
num_train_epochs: 80
learning_rate: 1e-05
warmup_steps: 0.05
fp16: True
push_to_hub: True
hub_model_id: liudev/novel-embedding-gpl-base-v2
hub_strategy: checkpoint
dataloader_num_workers: 4

All Hyperparameters

Click to expand

per_device_train_batch_size: 32
num_train_epochs: 80
max_steps: -1
learning_rate: 1e-05
lr_scheduler_type: linear
lr_scheduler_kwargs: None
warmup_steps: 0.05
optim: adamw_torch_fused
optim_args: None
weight_decay: 0.0
adam_beta1: 0.9
adam_beta2: 0.999
adam_epsilon: 1e-08
optim_target_modules: None
gradient_accumulation_steps: 1
average_tokens_across_devices: True
max_grad_norm: 1.0
label_smoothing_factor: 0.0
bf16: False
fp16: True
bf16_full_eval: False
fp16_full_eval: False
tf32: None
gradient_checkpointing: False
gradient_checkpointing_kwargs: None
torch_compile: False
torch_compile_backend: None
torch_compile_mode: None
use_liger_kernel: False
liger_kernel_config: None
use_cache: False
neftune_noise_alpha: None
torch_empty_cache_steps: None
auto_find_batch_size: False
log_on_each_node: True
logging_nan_inf_filter: True
include_num_input_tokens_seen: no
log_level: passive
log_level_replica: warning
disable_tqdm: False
project: huggingface
trackio_space_id: None
trackio_bucket_id: None
trackio_static_space_id: None
per_device_eval_batch_size: 8
prediction_loss_only: True
eval_on_start: False
eval_do_concat_batches: True
eval_use_gather_object: False
eval_accumulation_steps: None
include_for_metrics: []
batch_eval_metrics: False
save_only_model: False
save_on_each_node: False
enable_jit_checkpoint: False
push_to_hub: True
hub_private_repo: None
hub_model_id: liudev/novel-embedding-gpl-base-v2
hub_strategy: checkpoint
hub_always_push: False
hub_revision: None
load_best_model_at_end: False
ignore_data_skip: False
restore_callback_states_from_checkpoint: False
full_determinism: False
seed: 42
data_seed: None
use_cpu: False
accelerator_config: {'split_batches': False, 'dispatch_batches': None, 'even_batches': True, 'use_seedable_sampler': True, 'non_blocking': False, 'gradient_accumulation_kwargs': None}
parallelism_config: None
dataloader_drop_last: True
dataloader_num_workers: 4
dataloader_pin_memory: True
dataloader_persistent_workers: False
dataloader_prefetch_factor: None
dataloader_multiprocessing_context: None
dataloader_in_order: True
remove_unused_columns: True
label_names: None
train_sampling_strategy: random
length_column_name: length
ddp_find_unused_parameters: None
ddp_bucket_cap_mb: None
ddp_broadcast_buffers: False
ddp_static_graph: None
ddp_backend: None
ddp_timeout: 1800
fsdp: None
fsdp_config: None
deepspeed: None
debug: []
skip_memory_metrics: True
do_predict: False
resume_from_checkpoint: None
local_rank: -1
prompts: None
batch_sampler: batch_sampler
multi_dataset_batch_sampler: proportional
router_mapping: {}
learning_rate_mapping: {}
warmup_ratio: None

Training Logs

Click to expand

Epoch	Step	Training Loss
72.0662	490050	0.0091
72.0735	490100	0.0085
72.0809	490150	0.0091
72.0882	490200	0.0091
72.0956	490250	0.0094
72.1029	490300	0.0087
72.1103	490350	0.0091
72.1176	490400	0.0094
72.125	490450	0.0089
72.1324	490500	0.0089
72.1397	490550	0.0088
72.1471	490600	0.0089
72.1544	490650	0.0092
72.1618	490700	0.0089
72.1691	490750	0.0087
72.1765	490800	0.0089
72.1838	490850	0.0088
72.1912	490900	0.0091
72.1985	490950	0.0086
72.2059	491000	0.0095
72.2132	491050	0.0088
72.2206	491100	0.0091
72.2279	491150	0.0092
72.2353	491200	0.0088
72.2426	491250	0.0088
72.25	491300	0.0088
72.2574	491350	0.0092
72.2647	491400	0.0092
72.2721	491450	0.0087
72.2794	491500	0.0088
72.2868	491550	0.0085
72.2941	491600	0.0089
72.3015	491650	0.0090
72.3088	491700	0.0091
72.3162	491750	0.0091
72.3235	491800	0.0091
72.3309	491850	0.0092
72.3382	491900	0.0087
72.3456	491950	0.0090
72.3529	492000	0.0091
72.3603	492050	0.0085
72.3676	492100	0.0091
72.375	492150	0.0092
72.3824	492200	0.0090
72.3897	492250	0.0092
72.3971	492300	0.0095
72.4044	492350	0.0090
72.4118	492400	0.0094
72.4191	492450	0.0095
72.4265	492500	0.0091
72.4338	492550	0.0089
72.4412	492600	0.0087
72.4485	492650	0.0093
72.4559	492700	0.0090
72.4632	492750	0.0093
72.4706	492800	0.0085
72.4779	492850	0.0091
72.4853	492900	0.0088
72.4926	492950	0.0089
72.5	493000	0.0089
72.5074	493050	0.0091
72.5147	493100	0.0090
72.5221	493150	0.0091
72.5294	493200	0.0090
72.5368	493250	0.0091
72.5441	493300	0.0094
72.5515	493350	0.0087
72.5588	493400	0.0090
72.5662	493450	0.0090
72.5735	493500	0.0086
72.5809	493550	0.0085
72.5882	493600	0.0090
72.5956	493650	0.0090
72.6029	493700	0.0093
72.6103	493750	0.0089
72.6176	493800	0.0093
72.625	493850	0.0092
72.6324	493900	0.0096
72.6397	493950	0.0090
72.6471	494000	0.0087
72.6544	494050	0.0092
72.6618	494100	0.0087
72.6691	494150	0.0093
72.6765	494200	0.0088
72.6838	494250	0.0095
72.6912	494300	0.0093
72.6985	494350	0.0090
72.7059	494400	0.0091
72.7132	494450	0.0090
72.7206	494500	0.0092
72.7279	494550	0.0091
72.7353	494600	0.0092
72.7426	494650	0.0086
72.75	494700	0.0088
72.7574	494750	0.0092
72.7647	494800	0.0095
72.7721	494850	0.0090
72.7794	494900	0.0090
72.7868	494950	0.0086
72.7941	495000	0.0090
72.8015	495050	0.0091
72.8088	495100	0.0094
72.8162	495150	0.0092
72.8235	495200	0.0092
72.8309	495250	0.0096
72.8382	495300	0.0085
72.8456	495350	0.0093
72.8529	495400	0.0096
72.8603	495450	0.0087
72.8676	495500	0.0089
72.875	495550	0.0089
72.8824	495600	0.0094
72.8897	495650	0.0087
72.8971	495700	0.0094
72.9044	495750	0.0090
72.9118	495800	0.0093
72.9191	495850	0.0088
72.9265	495900	0.0090
72.9338	495950	0.0091
72.9412	496000	0.0085
72.9485	496050	0.0089
72.9559	496100	0.0092
72.9632	496150	0.0088
72.9706	496200	0.0095
72.9779	496250	0.0095
72.9853	496300	0.0087
72.9926	496350	0.0093
73.0	496400	0.0093
73.0074	496450	0.0092
73.0147	496500	0.0084
73.0221	496550	0.0092
73.0294	496600	0.0088
73.0368	496650	0.0091
73.0441	496700	0.0085
73.0515	496750	0.0093
73.0588	496800	0.0090
73.0662	496850	0.0091
73.0735	496900	0.0089
73.0809	496950	0.0090
73.0882	497000	0.0086
73.0956	497050	0.0092
73.1029	497100	0.0088
73.1103	497150	0.0086
73.1176	497200	0.0090
73.125	497250	0.0091
73.1324	497300	0.0091
73.1397	497350	0.0093
73.1471	497400	0.0087
73.1544	497450	0.0091
73.1618	497500	0.0094
73.1691	497550	0.0089
73.1765	497600	0.0087
73.1838	497650	0.0096
73.1912	497700	0.0089
73.1985	497750	0.0095
73.2059	497800	0.0091
73.2132	497850	0.0089
73.2206	497900	0.0091
73.2279	497950	0.0087
73.2353	498000	0.0088
73.2426	498050	0.0092
73.25	498100	0.0083
73.2574	498150	0.0093
73.2647	498200	0.0091
73.2721	498250	0.0091
73.2794	498300	0.0091
73.2868	498350	0.0089
73.2941	498400	0.0089
73.3015	498450	0.0093
73.3088	498500	0.0089
73.3162	498550	0.0093
73.3235	498600	0.0088
73.3309	498650	0.0091
73.3382	498700	0.0092
73.3456	498750	0.0090
73.3529	498800	0.0087
73.3603	498850	0.0089
73.3676	498900	0.0086
73.375	498950	0.0090
73.3824	499000	0.0089
73.3897	499050	0.0090
73.3971	499100	0.0091
73.4044	499150	0.0090
73.4118	499200	0.0085
73.4191	499250	0.0090
73.4265	499300	0.0088
73.4338	499350	0.0092
73.4412	499400	0.0091
73.4485	499450	0.0092
73.4559	499500	0.0092
73.4632	499550	0.0088
73.4706	499600	0.0089
73.4779	499650	0.0088
73.4853	499700	0.0089
73.4926	499750	0.0084
73.5	499800	0.0090
73.5074	499850	0.0091
73.5147	499900	0.0086
73.5221	499950	0.0090
73.5294	500000	0.0091
73.5368	500050	0.0092
73.5441	500100	0.0091
73.5515	500150	0.0091
73.5588	500200	0.0091
73.5662	500250	0.0093
73.5735	500300	0.0091
73.5809	500350	0.0093
73.5882	500400	0.0092
73.5956	500450	0.0094
73.6029	500500	0.0093
73.6103	500550	0.0090
73.6176	500600	0.0088
73.625	500650	0.0093
73.6324	500700	0.0088
73.6397	500750	0.0089
73.6471	500800	0.0090
73.6544	500850	0.0089
73.6618	500900	0.0092
73.6691	500950	0.0089
73.6765	501000	0.0092
73.6838	501050	0.0094
73.6912	501100	0.0090
73.6985	501150	0.0091
73.7059	501200	0.0091
73.7132	501250	0.0085
73.7206	501300	0.0089
73.7279	501350	0.0091
73.7353	501400	0.0089
73.7426	501450	0.0092
73.75	501500	0.0093
73.7574	501550	0.0089
73.7647	501600	0.0087
73.7721	501650	0.0090
73.7794	501700	0.0090
73.7868	501750	0.0086
73.7941	501800	0.0089
73.8015	501850	0.0095
73.8088	501900	0.0091
73.8162	501950	0.0091
73.8235	502000	0.0091
73.8309	502050	0.0094
73.8382	502100	0.0087
73.8456	502150	0.0094
73.8529	502200	0.0091
73.8603	502250	0.0090
73.8676	502300	0.0091
73.875	502350	0.0088
73.8824	502400	0.0085
73.8897	502450	0.0089
73.8971	502500	0.0092
73.9044	502550	0.0090
73.9118	502600	0.0091
73.9191	502650	0.0087
73.9265	502700	0.0086
73.9338	502750	0.0087
73.9412	502800	0.0089
73.9485	502850	0.0086
73.9559	502900	0.0088
73.9632	502950	0.0090
73.9706	503000	0.0090
73.9779	503050	0.0091
73.9853	503100	0.0091
73.9926	503150	0.0091
74.0	503200	0.0088
74.0074	503250	0.0092
74.0147	503300	0.0089
74.0221	503350	0.0091
74.0294	503400	0.0093
74.0368	503450	0.0088
74.0441	503500	0.0089
74.0515	503550	0.0087
74.0588	503600	0.0094
74.0662	503650	0.0089
74.0735	503700	0.0090
74.0809	503750	0.0089
74.0882	503800	0.0088
74.0956	503850	0.0088
74.1029	503900	0.0086
74.1103	503950	0.0088
74.1176	504000	0.0089
74.125	504050	0.0094
74.1324	504100	0.0089
74.1397	504150	0.0090
74.1471	504200	0.0088
74.1544	504250	0.0086
74.1618	504300	0.0088
74.1691	504350	0.0090
74.1765	504400	0.0089
74.1838	504450	0.0092
74.1912	504500	0.0090
74.1985	504550	0.0086
74.2059	504600	0.0087
74.2132	504650	0.0096
74.2206	504700	0.0088
74.2279	504750	0.0094
74.2353	504800	0.0087
74.2426	504850	0.0092
74.25	504900	0.0088
74.2574	504950	0.0087
74.2647	505000	0.0090
74.2721	505050	0.0091
74.2794	505100	0.0090
74.2868	505150	0.0092
74.2941	505200	0.0090
74.3015	505250	0.0089
74.3088	505300	0.0089
74.3162	505350	0.0088
74.3235	505400	0.0088
74.3309	505450	0.0087
74.3382	505500	0.0090
74.3456	505550	0.0086
74.3529	505600	0.0091
74.3603	505650	0.0090
74.3676	505700	0.0086
74.375	505750	0.0090
74.3824	505800	0.0088
74.3897	505850	0.0086
74.3971	505900	0.0094
74.4044	505950	0.0091
74.4118	506000	0.0093
74.4191	506050	0.0089
74.4265	506100	0.0088
74.4338	506150	0.0094
74.4412	506200	0.0089
74.4485	506250	0.0089
74.4559	506300	0.0083
74.4632	506350	0.0093
74.4706	506400	0.0085
74.4779	506450	0.0092
74.4853	506500	0.0091
74.4926	506550	0.0088
74.5	506600	0.0088
74.5074	506650	0.0095
74.5147	506700	0.0090
74.5221	506750	0.0087
74.5294	506800	0.0092
74.5368	506850	0.0088
74.5441	506900	0.0088
74.5515	506950	0.0089
74.5588	507000	0.0091
74.5662	507050	0.0088
74.5735	507100	0.0089
74.5809	507150	0.0090
74.5882	507200	0.0086
74.5956	507250	0.0087
74.6029	507300	0.0090
74.6103	507350	0.0089
74.6176	507400	0.0093
74.625	507450	0.0092
74.6324	507500	0.0086
74.6397	507550	0.0088
74.6471	507600	0.0090
74.6544	507650	0.0089
74.6618	507700	0.0088
74.6691	507750	0.0094
74.6765	507800	0.0089
74.6838	507850	0.0093
74.6912	507900	0.0093
74.6985	507950	0.0092
74.7059	508000	0.0083
74.7132	508050	0.0089
74.7206	508100	0.0089
74.7279	508150	0.0092
74.7353	508200	0.0094
74.7426	508250	0.0088
74.75	508300	0.0090
74.7574	508350	0.0087
74.7647	508400	0.0093
74.7721	508450	0.0090
74.7794	508500	0.0090
74.7868	508550	0.0094
74.7941	508600	0.0092
74.8015	508650	0.0092
74.8088	508700	0.0090
74.8162	508750	0.0094
74.8235	508800	0.0093
74.8309	508850	0.0087
74.8382	508900	0.0089
74.8456	508950	0.0086
74.8529	509000	0.0092
74.8603	509050	0.0092
74.8676	509100	0.0089
74.875	509150	0.0088
74.8824	509200	0.0100
74.8897	509250	0.0090
74.8971	509300	0.0093
74.9044	509350	0.0093
74.9118	509400	0.0087
74.9191	509450	0.0083
74.9265	509500	0.0089
74.9338	509550	0.0091
74.9412	509600	0.0090
74.9485	509650	0.0085
74.9559	509700	0.0093
74.9632	509750	0.0091
74.9706	509800	0.0092
74.9779	509850	0.0086
74.9853	509900	0.0089
74.9926	509950	0.0094
75.0	510000	0.0088
75.0074	510050	0.0087
75.0147	510100	0.0085
75.0221	510150	0.0088
75.0294	510200	0.0089
75.0368	510250	0.0090
75.0441	510300	0.0085
75.0515	510350	0.0095
75.0588	510400	0.0087
75.0662	510450	0.0095
75.0735	510500	0.0090
75.0809	510550	0.0092
75.0882	510600	0.0088
75.0956	510650	0.0094
75.1029	510700	0.0087
75.1103	510750	0.0091
75.1176	510800	0.0089
75.125	510850	0.0088
75.1324	510900	0.0088
75.1397	510950	0.0089
75.1471	511000	0.0091
75.1544	511050	0.0090
75.1618	511100	0.0092
75.1691	511150	0.0091
75.1765	511200	0.0092
75.1838	511250	0.0089
75.1912	511300	0.0091
75.1985	511350	0.0087
75.2059	511400	0.0091
75.2132	511450	0.0091
75.2206	511500	0.0090
75.2279	511550	0.0091
75.2353	511600	0.0088
75.2426	511650	0.0087
75.25	511700	0.0094
75.2574	511750	0.0089
75.2647	511800	0.0094
75.2721	511850	0.0094
75.2794	511900	0.0090
75.2868	511950	0.0085
75.2941	512000	0.0092
75.3015	512050	0.0095
75.3088	512100	0.0092
75.3162	512150	0.0093
75.3235	512200	0.0092
75.3309	512250	0.0091
75.3382	512300	0.0088
75.3456	512350	0.0088
75.3529	512400	0.0086
75.3603	512450	0.0091
75.3676	512500	0.0094
75.375	512550	0.0094
75.3824	512600	0.0090
75.3897	512650	0.0089
75.3971	512700	0.0089
75.4044	512750	0.0089
75.4118	512800	0.0092
75.4191	512850	0.0086
75.4265	512900	0.0094
75.4338	512950	0.0091
75.4412	513000	0.0089
75.4485	513050	0.0085
75.4559	513100	0.0088
75.4632	513150	0.0084
75.4706	513200	0.0091
75.4779	513250	0.0086
75.4853	513300	0.0088
75.4926	513350	0.0093
75.5	513400	0.0089
75.5074	513450	0.0090
75.5147	513500	0.0089
75.5221	513550	0.0089
75.5294	513600	0.0092
75.5368	513650	0.0090
75.5441	513700	0.0089
75.5515	513750	0.0097
75.5588	513800	0.0097
75.5662	513850	0.0093
75.5735	513900	0.0091
75.5809	513950	0.0089
75.5882	514000	0.0085
75.5956	514050	0.0086
75.6029	514100	0.0084
75.6103	514150	0.0088
75.6176	514200	0.0090
75.625	514250	0.0088
75.6324	514300	0.0092
75.6397	514350	0.0092
75.6471	514400	0.0088
75.6544	514450	0.0089
75.6618	514500	0.0090
75.6691	514550	0.0087
75.6765	514600	0.0093
75.6838	514650	0.0092
75.6912	514700	0.0086
75.6985	514750	0.0089
75.7059	514800	0.0090
75.7132	514850	0.0089
75.7206	514900	0.0093
75.7279	514950	0.0091
75.7353	515000	0.0094
75.7426	515050	0.0089
75.75	515100	0.0089
75.7574	515150	0.0089
75.7647	515200	0.0088
75.7721	515250	0.0088
75.7794	515300	0.0093
75.7868	515350	0.0096
75.7941	515400	0.0094
75.8015	515450	0.0088
75.8088	515500	0.0092
75.8162	515550	0.0093
75.8235	515600	0.0088
75.8309	515650	0.0090
75.8382	515700	0.0088
75.8456	515750	0.0089
75.8529	515800	0.0090
75.8603	515850	0.0091
75.8676	515900	0.0093
75.875	515950	0.0091
75.8824	516000	0.0084
75.8897	516050	0.0089
75.8971	516100	0.0085
75.9044	516150	0.0092
75.9118	516200	0.0089
75.9191	516250	0.0090
75.9265	516300	0.0093
75.9338	516350	0.0089
75.9412	516400	0.0094
75.9485	516450	0.0088
75.9559	516500	0.0092
75.9632	516550	0.0090
75.9706	516600	0.0091
75.9779	516650	0.0084
75.9853	516700	0.0088
75.9926	516750	0.0089
76.0	516800	0.0088
76.0074	516850	0.0092
76.0147	516900	0.0089
76.0221	516950	0.0086
76.0294	517000	0.0086
76.0368	517050	0.0092
76.0441	517100	0.0090
76.0515	517150	0.0094
76.0588	517200	0.0094
76.0662	517250	0.0089
76.0735	517300	0.0090
76.0809	517350	0.0092
76.0882	517400	0.0082
76.0956	517450	0.0086
76.1029	517500	0.0089
76.1103	517550	0.0090
76.1176	517600	0.0083
76.125	517650	0.0094
76.1324	517700	0.0095
76.1397	517750	0.0086
76.1471	517800	0.0092
76.1544	517850	0.0095
76.1618	517900	0.0088
76.1691	517950	0.0093
76.1765	518000	0.0087
76.1838	518050	0.0093
76.1912	518100	0.0091
76.1985	518150	0.0083
76.2059	518200	0.0091
76.2132	518250	0.0087
76.2206	518300	0.0093
76.2279	518350	0.0090
76.2353	518400	0.0091
76.2426	518450	0.0085
76.25	518500	0.0089
76.2574	518550	0.0093
76.2647	518600	0.0090
76.2721	518650	0.0090
76.2794	518700	0.0093
76.2868	518750	0.0089
76.2941	518800	0.0090
76.3015	518850	0.0088
76.3088	518900	0.0087
76.3162	518950	0.0087
76.3235	519000	0.0092
76.3309	519050	0.0090
76.3382	519100	0.0093
76.3456	519150	0.0088
76.3529	519200	0.0092
76.3603	519250	0.0085
76.3676	519300	0.0088
76.375	519350	0.0088
76.3824	519400	0.0090
76.3897	519450	0.0091
76.3971	519500	0.0085
76.4044	519550	0.0089
76.4118	519600	0.0087
76.4191	519650	0.0091
76.4265	519700	0.0085
76.4338	519750	0.0090
76.4412	519800	0.0089
76.4485	519850	0.0084
76.4559	519900	0.0088
76.4632	519950	0.0091
76.4706	520000	0.0087
76.4779	520050	0.0086
76.4853	520100	0.0093
76.4926	520150	0.0088
76.5	520200	0.0096
76.5074	520250	0.0086
76.5147	520300	0.0089
76.5221	520350	0.0083
76.5294	520400	0.0089
76.5368	520450	0.0090
76.5441	520500	0.0086
76.5515	520550	0.0092
76.5588	520600	0.0084
76.5662	520650	0.0094
76.5735	520700	0.0093
76.5809	520750	0.0092
76.5882	520800	0.0090
76.5956	520850	0.0085
76.6029	520900	0.0093
76.6103	520950	0.0086
76.6176	521000	0.0091
76.625	521050	0.0086
76.6324	521100	0.0089
76.6397	521150	0.0089
76.6471	521200	0.0087
76.6544	521250	0.0096
76.6618	521300	0.0090
76.6691	521350	0.0090
76.6765	521400	0.0089
76.6838	521450	0.0093
76.6912	521500	0.0089
76.6985	521550	0.0091
76.7059	521600	0.0093
76.7132	521650	0.0088
76.7206	521700	0.0089
76.7279	521750	0.0088
76.7353	521800	0.0092
76.7426	521850	0.0090
76.75	521900	0.0083
76.7574	521950	0.0089
76.7647	522000	0.0091
76.7721	522050	0.0090
76.7794	522100	0.0091
76.7868	522150	0.0090
76.7941	522200	0.0087
76.8015	522250	0.0094
76.8088	522300	0.0091
76.8162	522350	0.0091
76.8235	522400	0.0087
76.8309	522450	0.0092
76.8382	522500	0.0091
76.8456	522550	0.0089
76.8529	522600	0.0091
76.8603	522650	0.0084
76.8676	522700	0.0092
76.875	522750	0.0089
76.8824	522800	0.0094
76.8897	522850	0.0090
76.8971	522900	0.0088
76.9044	522950	0.0087
76.9118	523000	0.0089
76.9191	523050	0.0094
76.9265	523100	0.0094
76.9338	523150	0.0093
76.9412	523200	0.0095
76.9485	523250	0.0090
76.9559	523300	0.0087
76.9632	523350	0.0089
76.9706	523400	0.0095
76.9779	523450	0.0088
76.9853	523500	0.0092
76.9926	523550	0.0091
77.0	523600	0.0086
77.0074	523650	0.0086
77.0147	523700	0.0090
77.0221	523750	0.0094
77.0294	523800	0.0094
77.0368	523850	0.0089
77.0441	523900	0.0091
77.0515	523950	0.0087
77.0588	524000	0.0090
77.0662	524050	0.0091
77.0735	524100	0.0095
77.0809	524150	0.0088
77.0882	524200	0.0091
77.0956	524250	0.0088
77.1029	524300	0.0096
77.1103	524350	0.0094
77.1176	524400	0.0094
77.125	524450	0.0087
77.1324	524500	0.0085
77.1397	524550	0.0088
77.1471	524600	0.0091
77.1544	524650	0.0089
77.1618	524700	0.0093
77.1691	524750	0.0084
77.1765	524800	0.0088
77.1838	524850	0.0087
77.1912	524900	0.0089
77.1985	524950	0.0093
77.2059	525000	0.0089
77.2132	525050	0.0092
77.2206	525100	0.0087
77.2279	525150	0.0091
77.2353	525200	0.0089
77.2426	525250	0.0092
77.25	525300	0.0087
77.2574	525350	0.0092
77.2647	525400	0.0088
77.2721	525450	0.0091
77.2794	525500	0.0088
77.2868	525550	0.0091
77.2941	525600	0.0087
77.3015	525650	0.0086
77.3088	525700	0.0087
77.3162	525750	0.0087
77.3235	525800	0.0091
77.3309	525850	0.0090
77.3382	525900	0.0095
77.3456	525950	0.0088
77.3529	526000	0.0087
77.3603	526050	0.0091
77.3676	526100	0.0090
77.375	526150	0.0090
77.3824	526200	0.0087
77.3897	526250	0.0090
77.3971	526300	0.0088
77.4044	526350	0.0091
77.4118	526400	0.0090
77.4191	526450	0.0084
77.4265	526500	0.0091
77.4338	526550	0.0089
77.4412	526600	0.0087
77.4485	526650	0.0087
77.4559	526700	0.0091
77.4632	526750	0.0090
77.4706	526800	0.0091
77.4779	526850	0.0088
77.4853	526900	0.0091
77.4926	526950	0.0090
77.5	527000	0.0087
77.5074	527050	0.0091
77.5147	527100	0.0084
77.5221	527150	0.0090
77.5294	527200	0.0093
77.5368	527250	0.0086
77.5441	527300	0.0089
77.5515	527350	0.0089
77.5588	527400	0.0091
77.5662	527450	0.0094
77.5735	527500	0.0091
77.5809	527550	0.0089
77.5882	527600	0.0088
77.5956	527650	0.0090
77.6029	527700	0.0087
77.6103	527750	0.0092
77.6176	527800	0.0094
77.625	527850	0.0091
77.6324	527900	0.0092
77.6397	527950	0.0092
77.6471	528000	0.0088
77.6544	528050	0.0086
77.6618	528100	0.0092
77.6691	528150	0.0095
77.6765	528200	0.0095
77.6838	528250	0.0090
77.6912	528300	0.0088
77.6985	528350	0.0091
77.7059	528400	0.0088
77.7132	528450	0.0086
77.7206	528500	0.0087
77.7279	528550	0.0088
77.7353	528600	0.0088
77.7426	528650	0.0090
77.75	528700	0.0092
77.7574	528750	0.0091
77.7647	528800	0.0092
77.7721	528850	0.0090
77.7794	528900	0.0089
77.7868	528950	0.0091
77.7941	529000	0.0088
77.8015	529050	0.0090
77.8088	529100	0.0091
77.8162	529150	0.0088
77.8235	529200	0.0091
77.8309	529250	0.0087
77.8382	529300	0.0091
77.8456	529350	0.0087
77.8529	529400	0.0090
77.8603	529450	0.0094
77.8676	529500	0.0093
77.875	529550	0.0086
77.8824	529600	0.0090
77.8897	529650	0.0092
77.8971	529700	0.0094
77.9044	529750	0.0088
77.9118	529800	0.0088
77.9191	529850	0.0086
77.9265	529900	0.0087
77.9338	529950	0.0092
77.9412	530000	0.0090

Training Time

Training: 5.8 hours

Framework Versions

Python: 3.13.15
Sentence Transformers: 5.7.0
Transformers: 5.16.1
PyTorch: 2.11.0+cu128
Accelerate: 1.14.0
Datasets: 4.8.5
Tokenizers: 0.23.1

Additional Resources

Training and Finetuning Embedding Models with Sentence Transformers: the end-to-end guide for training or finetuning Sentence Transformer models.
Introduction to Matryoshka Embedding Models: variable-size embeddings that can be truncated with minimal quality loss.
Binary and Scalar Embedding Quantization for Significantly Faster & Cheaper Retrieval: post-training compression of embedding vectors.
Multimodal Embedding & Reranker Models with Sentence Transformers: use text, image, audio, and video models through the same API.
Training and Finetuning Multimodal Embedding & Reranker Models with Sentence Transformers: train multimodal embedding models, with a Visual Document Retrieval walkthrough.

Citation

BibTeX

Sentence Transformers

@inproceedings{reimers-2019-sentence-bert,
    title = "Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks",
    author = "Reimers, Nils and Gurevych, Iryna",
    booktitle = "Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing",
    month = "11",
    year = "2019",
    publisher = "Association for Computational Linguistics",
    url = "https://arxiv.org/abs/1908.10084",
}

MarginMSELoss

@misc{hofstätter2021improving,
    title={Improving Efficient Neural Ranking Models with Cross-Architecture Knowledge Distillation},
    author={Sebastian Hofstätter and Sophia Althammer and Michael Schröder and Mete Sertkan and Allan Hanbury},
    year={2021},
    eprint={2010.02666},
    archivePrefix={arXiv},
    primaryClass={cs.IR}
}