SentenceTransformer based on BAAI/bge-base-zh-v1.5

This is a sentence-transformers model finetuned from BAAI/bge-base-zh-v1.5. It maps sentences & paragraphs to a 768-dimensional dense vector space and can be used for semantic textual similarity, semantic search, paraphrase mining, classification, clustering, and more.

Model Details

Model Description

Model Type: Sentence Transformer
Base model: BAAI/bge-base-zh-v1.5
Maximum Sequence Length: 256 tokens
Output Dimensionality: 768 dimensions
Similarity Function: Cosine Similarity
Supported Modality: Text

Model Sources

Documentation: Sentence Transformers Documentation
Repository: Sentence Transformers on GitHub
Hugging Face: Sentence Transformers on Hugging Face

Full Model Architecture

SentenceTransformer(
  (0): Transformer({'transformer_task': 'feature-extraction', 'modality_config': {'text': {'method': 'forward', 'method_output_name': 'last_hidden_state'}}, 'module_output_name': 'token_embeddings', 'architecture': 'BertModel'})
  (1): Pooling({'embedding_dimension': 768, 'pooling_mode': 'cls', 'include_prompt': True})
  (2): Normalize({})
)

Usage

Direct Usage (Sentence Transformers)

First install the Sentence Transformers library:

pip install -U sentence-transformers

Then you can load this model and run inference.

from sentence_transformers import SentenceTransformer

# Download from the 🤗 Hub
model = SentenceTransformer("liudev/novel-embedding-gpl-base-v1")
# Run inference
sentences = [
    '在异能觉醒爆发时安抚对方并带走宝物',
    '宴席上有人当众示爱引发全场震惊',
    '凶暴巨兽冲破防线引发骚乱',
    '山贼讲述当年的恐惧往事，结果再次遭遇强敌被瞬杀',
]
embeddings = model.encode(sentences)
print(embeddings.shape)
# [4, 768]

# Get the similarity scores for the embeddings
similarities = model.similarity(embeddings, embeddings)
print(similarities.shape)
# [4, 4]

Training Details

Training Dataset

Unnamed Dataset

Size: 435,262 training samples
Columns: anchor, sentence1, sentence2, and label

Approximate statistics based on the first 100 samples:

	anchor	sentence1	sentence2	label
type	string	string	string	float
modality	text	text	text
details	min: 8 tokens mean: 15.5 tokens max: 25 tokens	min: 47 tokens mean: 73.19 tokens max: 256 tokens	min: 29 tokens mean: 69.23 tokens max: 92 tokens	min: 0.02 mean: 0.37 max: 0.84

Samples:

anchor	sentence1	sentence2	label
`看破敌人的隐身手段`	`周昂在围攻强敌时敏锐察觉到异样，通过观察灵气流动识破并破除隐身法术，最终果断出手将藏匿在墙头的施法者格杀。`	`李破在马场以赏罚分明的手段收拢人心并大摆酒宴后，在李碧的私下召见中，意外得知其欲随军出征辽东的计划并被征询意见。`	`0.759356061900818`
`在仓库意外得到防身宝物`	`柳渊在医馆仓库意外获得防御法器佛珠及攻击真言，随后通过定星罗盘锁定义务星邪，并凭借缜密的推理引导葛老与辛捕头共同前往后院枯井探查。`	`沈河在烂尾楼工地通过系统指引接连开启宝箱，获得了具备强控效果的“大力飞砖”与防御性的“安全员安全帽”，在确认工地无剩余宝箱后，随即根据新坐标向十公里外进发。`	`0.09672797019024215`
`进宫面圣前，太监好心提醒忌讳规矩`	`宦官郭穰在权衡利弊后决定拉拢入觐的天子待诏秀才张毅，不仅对其礼遇有加，更在入宫途中悉心传授面圣禁忌与生存经验，双方达成了初步的盟友共识。`	`在正月初一太子加冠的重大典礼上，南风费尽心思哄劝赖床的太子完成繁琐仪式并接手庞大的东宫官僚体系，同时担忧自己微服出游时的糗事在稍后的公开亮相中被路人认出。`	`0.5887360227245302`

Loss: MarginMSELoss with these parameters:

{
    "similarity_fct": "pairwise_dot_score"
}

Training Hyperparameters

Non-Default Hyperparameters

per_device_train_batch_size: 32
num_train_epochs: 150
learning_rate: 2e-05
warmup_steps: 0.02
fp16: True
push_to_hub: True
hub_model_id: liudev/novel-embedding-gpl-base-v1
hub_strategy: checkpoint
dataloader_num_workers: 4

All Hyperparameters

Click to expand

per_device_train_batch_size: 32
num_train_epochs: 150
max_steps: -1
learning_rate: 2e-05
lr_scheduler_type: linear
lr_scheduler_kwargs: None
warmup_steps: 0.02
optim: adamw_torch_fused
optim_args: None
weight_decay: 0.0
adam_beta1: 0.9
adam_beta2: 0.999
adam_epsilon: 1e-08
optim_target_modules: None
gradient_accumulation_steps: 1
average_tokens_across_devices: True
max_grad_norm: 1.0
label_smoothing_factor: 0.0
bf16: False
fp16: True
bf16_full_eval: False
fp16_full_eval: False
tf32: None
gradient_checkpointing: False
gradient_checkpointing_kwargs: None
torch_compile: False
torch_compile_backend: None
torch_compile_mode: None
use_liger_kernel: False
liger_kernel_config: None
use_cache: False
neftune_noise_alpha: None
torch_empty_cache_steps: None
auto_find_batch_size: False
log_on_each_node: True
logging_nan_inf_filter: True
include_num_input_tokens_seen: no
log_level: passive
log_level_replica: warning
disable_tqdm: False
project: huggingface
trackio_space_id: None
trackio_bucket_id: None
trackio_static_space_id: None
per_device_eval_batch_size: 8
prediction_loss_only: True
eval_on_start: False
eval_do_concat_batches: True
eval_use_gather_object: False
eval_accumulation_steps: None
include_for_metrics: []
batch_eval_metrics: False
save_only_model: False
save_on_each_node: False
enable_jit_checkpoint: False
push_to_hub: True
hub_private_repo: None
hub_model_id: liudev/novel-embedding-gpl-base-v1
hub_strategy: checkpoint
hub_always_push: False
hub_revision: None
load_best_model_at_end: False
ignore_data_skip: False
restore_callback_states_from_checkpoint: False
full_determinism: False
seed: 42
data_seed: None
use_cpu: False
accelerator_config: {'split_batches': False, 'dispatch_batches': None, 'even_batches': True, 'use_seedable_sampler': True, 'non_blocking': False, 'gradient_accumulation_kwargs': None}
parallelism_config: None
dataloader_drop_last: True
dataloader_num_workers: 4
dataloader_pin_memory: True
dataloader_persistent_workers: False
dataloader_prefetch_factor: None
dataloader_multiprocessing_context: None
dataloader_in_order: True
remove_unused_columns: True
label_names: None
train_sampling_strategy: random
length_column_name: length
ddp_find_unused_parameters: None
ddp_bucket_cap_mb: None
ddp_broadcast_buffers: False
ddp_static_graph: None
ddp_backend: None
ddp_timeout: 1800
fsdp: None
fsdp_config: None
deepspeed: None
debug: []
skip_memory_metrics: True
do_predict: False
resume_from_checkpoint: None
local_rank: -1
prompts: None
batch_sampler: batch_sampler
multi_dataset_batch_sampler: proportional
router_mapping: {}
learning_rate_mapping: {}
warmup_ratio: None

Training Logs

Click to expand

Epoch	Step	Training Loss
117.6544	800050	0.0077
117.6618	800100	0.0081
117.6691	800150	0.0084
117.6765	800200	0.0086
117.6838	800250	0.0084
117.6912	800300	0.0085
117.6985	800350	0.0087
117.7059	800400	0.0080
117.7132	800450	0.0080
117.7206	800500	0.0080
117.7279	800550	0.0083
117.7353	800600	0.0080
117.7426	800650	0.0079
117.75	800700	0.0082
117.7574	800750	0.0078
117.7647	800800	0.0079
117.7721	800850	0.0082
117.7794	800900	0.0078
117.7868	800950	0.0081
117.7941	801000	0.0083
117.8015	801050	0.0082
117.8088	801100	0.0082
117.8162	801150	0.0084
117.8235	801200	0.0083
117.8309	801250	0.0084
117.8382	801300	0.0082
117.8456	801350	0.0084
117.8529	801400	0.0084
117.8603	801450	0.0077
117.8676	801500	0.0084
117.875	801550	0.0080
117.8824	801600	0.0081
117.8897	801650	0.0089
117.8971	801700	0.0084
117.9044	801750	0.0083
117.9118	801800	0.0080
117.9191	801850	0.0083
117.9265	801900	0.0079
117.9338	801950	0.0084
117.9412	802000	0.0081
117.9485	802050	0.0082
117.9559	802100	0.0082
117.9632	802150	0.0081
117.9706	802200	0.0081
117.9779	802250	0.0084
117.9853	802300	0.0080
117.9926	802350	0.0081
118.0	802400	0.0083
118.0074	802450	0.0082
118.0147	802500	0.0084
118.0221	802550	0.0081
118.0294	802600	0.0081
118.0368	802650	0.0082
118.0441	802700	0.0079
118.0515	802750	0.0082
118.0588	802800	0.0081
118.0662	802850	0.0080
118.0735	802900	0.0081
118.0809	802950	0.0083
118.0882	803000	0.0077
118.0956	803050	0.0083
118.1029	803100	0.0080
118.1103	803150	0.0081
118.1176	803200	0.0083
118.125	803250	0.0081
118.1324	803300	0.0080
118.1397	803350	0.0079
118.1471	803400	0.0083
118.1544	803450	0.0080
118.1618	803500	0.0084
118.1691	803550	0.0084
118.1765	803600	0.0084
118.1838	803650	0.0084
118.1912	803700	0.0079
118.1985	803750	0.0084
118.2059	803800	0.0083
118.2132	803850	0.0080
118.2206	803900	0.0082
118.2279	803950	0.0083
118.2353	804000	0.0081
118.2426	804050	0.0082
118.25	804100	0.0085
118.2574	804150	0.0084
118.2647	804200	0.0084
118.2721	804250	0.0081
118.2794	804300	0.0081
118.2868	804350	0.0081
118.2941	804400	0.0080
118.3015	804450	0.0081
118.3088	804500	0.0079
118.3162	804550	0.0085
118.3235	804600	0.0081
118.3309	804650	0.0083
118.3382	804700	0.0082
118.3456	804750	0.0078
118.3529	804800	0.0080
118.3603	804850	0.0083
118.3676	804900	0.0081
118.375	804950	0.0083
118.3824	805000	0.0078
118.3897	805050	0.0082
118.3971	805100	0.0082
118.4044	805150	0.0080
118.4118	805200	0.0084
118.4191	805250	0.0082
118.4265	805300	0.0081
118.4338	805350	0.0084
118.4412	805400	0.0077
118.4485	805450	0.0086
118.4559	805500	0.0078
118.4632	805550	0.0086
118.4706	805600	0.0085
118.4779	805650	0.0085
118.4853	805700	0.0085
118.4926	805750	0.0086
118.5	805800	0.0081
118.5074	805850	0.0082
118.5147	805900	0.0079
118.5221	805950	0.0082
118.5294	806000	0.0082
118.5368	806050	0.0080
118.5441	806100	0.0081
118.5515	806150	0.0081
118.5588	806200	0.0082
118.5662	806250	0.0086
118.5735	806300	0.0084
118.5809	806350	0.0082
118.5882	806400	0.0082
118.5956	806450	0.0082
118.6029	806500	0.0084
118.6103	806550	0.0089
118.6176	806600	0.0081
118.625	806650	0.0081
118.6324	806700	0.0082
118.6397	806750	0.0084
118.6471	806800	0.0080
118.6544	806850	0.0083
118.6618	806900	0.0086
118.6691	806950	0.0083
118.6765	807000	0.0080
118.6838	807050	0.0083
118.6912	807100	0.0077
118.6985	807150	0.0085
118.7059	807200	0.0080
118.7132	807250	0.0082
118.7206	807300	0.0084
118.7279	807350	0.0082
118.7353	807400	0.0080
118.7426	807450	0.0081
118.75	807500	0.0083
118.7574	807550	0.0083
118.7647	807600	0.0080
118.7721	807650	0.0081
118.7794	807700	0.0079
118.7868	807750	0.0079
118.7941	807800	0.0081
118.8015	807850	0.0081
118.8088	807900	0.0082
118.8162	807950	0.0080
118.8235	808000	0.0084
118.8309	808050	0.0080
118.8382	808100	0.0077
118.8456	808150	0.0083
118.8529	808200	0.0083
118.8603	808250	0.0081
118.8676	808300	0.0081
118.875	808350	0.0078
118.8824	808400	0.0076
118.8897	808450	0.0082
118.8971	808500	0.0079
118.9044	808550	0.0078
118.9118	808600	0.0082
118.9191	808650	0.0081
118.9265	808700	0.0080
118.9338	808750	0.0086
118.9412	808800	0.0083
118.9485	808850	0.0083
118.9559	808900	0.0084
118.9632	808950	0.0082
118.9706	809000	0.0078
118.9779	809050	0.0081
118.9853	809100	0.0080
118.9926	809150	0.0082
119.0	809200	0.0085
119.0074	809250	0.0080
119.0147	809300	0.0082
119.0221	809350	0.0083
119.0294	809400	0.0081
119.0368	809450	0.0079
119.0441	809500	0.0079
119.0515	809550	0.0083
119.0588	809600	0.0079
119.0662	809650	0.0083
119.0735	809700	0.0082
119.0809	809750	0.0081
119.0882	809800	0.0082
119.0956	809850	0.0087
119.1029	809900	0.0081
119.1103	809950	0.0082
119.1176	810000	0.0079
119.125	810050	0.0078
119.1324	810100	0.0082
119.1397	810150	0.0081
119.1471	810200	0.0084
119.1544	810250	0.0085
119.1618	810300	0.0080
119.1691	810350	0.0083
119.1765	810400	0.0079
119.1838	810450	0.0082
119.1912	810500	0.0081
119.1985	810550	0.0082
119.2059	810600	0.0080
119.2132	810650	0.0082
119.2206	810700	0.0080
119.2279	810750	0.0082
119.2353	810800	0.0080
119.2426	810850	0.0083
119.25	810900	0.0080
119.2574	810950	0.0083
119.2647	811000	0.0080
119.2721	811050	0.0082
119.2794	811100	0.0083
119.2868	811150	0.0084
119.2941	811200	0.0083
119.3015	811250	0.0080
119.3088	811300	0.0078
119.3162	811350	0.0077
119.3235	811400	0.0080
119.3309	811450	0.0082
119.3382	811500	0.0078
119.3456	811550	0.0080
119.3529	811600	0.0084
119.3603	811650	0.0080
119.3676	811700	0.0076
119.375	811750	0.0082
119.3824	811800	0.0081
119.3897	811850	0.0081
119.3971	811900	0.0080
119.4044	811950	0.0083
119.4118	812000	0.0081
119.4191	812050	0.0083
119.4265	812100	0.0080
119.4338	812150	0.0083
119.4412	812200	0.0082
119.4485	812250	0.0082
119.4559	812300	0.0085
119.4632	812350	0.0081
119.4706	812400	0.0081
119.4779	812450	0.0088
119.4853	812500	0.0080
119.4926	812550	0.0079
119.5	812600	0.0079
119.5074	812650	0.0081
119.5147	812700	0.0078
119.5221	812750	0.0077
119.5294	812800	0.0083
119.5368	812850	0.0080
119.5441	812900	0.0081
119.5515	812950	0.0079
119.5588	813000	0.0080
119.5662	813050	0.0083
119.5735	813100	0.0086
119.5809	813150	0.0081
119.5882	813200	0.0082
119.5956	813250	0.0080
119.6029	813300	0.0083
119.6103	813350	0.0079
119.6176	813400	0.0082
119.625	813450	0.0081
119.6324	813500	0.0083
119.6397	813550	0.0081
119.6471	813600	0.0083
119.6544	813650	0.0081
119.6618	813700	0.0082
119.6691	813750	0.0082
119.6765	813800	0.0082
119.6838	813850	0.0078
119.6912	813900	0.0082
119.6985	813950	0.0085
119.7059	814000	0.0084
119.7132	814050	0.0082
119.7206	814100	0.0078
119.7279	814150	0.0083
119.7353	814200	0.0077
119.7426	814250	0.0083
119.75	814300	0.0083
119.7574	814350	0.0084
119.7647	814400	0.0081
119.7721	814450	0.0084
119.7794	814500	0.0085
119.7868	814550	0.0079
119.7941	814600	0.0077
119.8015	814650	0.0083
119.8088	814700	0.0082
119.8162	814750	0.0082
119.8235	814800	0.0079
119.8309	814850	0.0079
119.8382	814900	0.0085
119.8456	814950	0.0081
119.8529	815000	0.0082
119.8603	815050	0.0082
119.8676	815100	0.0085
119.875	815150	0.0082
119.8824	815200	0.0081
119.8897	815250	0.0084
119.8971	815300	0.0084
119.9044	815350	0.0082
119.9118	815400	0.0078
119.9191	815450	0.0088
119.9265	815500	0.0082
119.9338	815550	0.0081
119.9412	815600	0.0079
119.9485	815650	0.0084
119.9559	815700	0.0084
119.9632	815750	0.0086
119.9706	815800	0.0081
119.9779	815850	0.0083
119.9853	815900	0.0082
119.9926	815950	0.0079
120.0	816000	0.0080
120.0074	816050	0.0078
120.0147	816100	0.0080
120.0221	816150	0.0078
120.0294	816200	0.0077
120.0368	816250	0.0084
120.0441	816300	0.0080
120.0515	816350	0.0079
120.0588	816400	0.0086
120.0662	816450	0.0082
120.0735	816500	0.0079
120.0809	816550	0.0084
120.0882	816600	0.0080
120.0956	816650	0.0082
120.1029	816700	0.0080
120.1103	816750	0.0080
120.1176	816800	0.0084
120.125	816850	0.0085
120.1324	816900	0.0080
120.1397	816950	0.0084
120.1471	817000	0.0079
120.1544	817050	0.0081
120.1618	817100	0.0080
120.1691	817150	0.0084
120.1765	817200	0.0086
120.1838	817250	0.0080
120.1912	817300	0.0078
120.1985	817350	0.0085
120.2059	817400	0.0077
120.2132	817450	0.0084
120.2206	817500	0.0080
120.2279	817550	0.0081
120.2353	817600	0.0081
120.2426	817650	0.0076
120.25	817700	0.0077
120.2574	817750	0.0081
120.2647	817800	0.0081
120.2721	817850	0.0080
120.2794	817900	0.0083
120.2868	817950	0.0086
120.2941	818000	0.0082
120.3015	818050	0.0081
120.3088	818100	0.0081
120.3162	818150	0.0080
120.3235	818200	0.0081
120.3309	818250	0.0082
120.3382	818300	0.0081
120.3456	818350	0.0081
120.3529	818400	0.0078
120.3603	818450	0.0081
120.3676	818500	0.0079
120.375	818550	0.0080
120.3824	818600	0.0082
120.3897	818650	0.0078
120.3971	818700	0.0077
120.4044	818750	0.0082
120.4118	818800	0.0084
120.4191	818850	0.0079
120.4265	818900	0.0081
120.4338	818950	0.0086
120.4412	819000	0.0084
120.4485	819050	0.0081
120.4559	819100	0.0083
120.4632	819150	0.0083
120.4706	819200	0.0087
120.4779	819250	0.0080
120.4853	819300	0.0083
120.4926	819350	0.0081
120.5	819400	0.0080
120.5074	819450	0.0079
120.5147	819500	0.0083
120.5221	819550	0.0082
120.5294	819600	0.0082
120.5368	819650	0.0082
120.5441	819700	0.0080
120.5515	819750	0.0080
120.5588	819800	0.0079
120.5662	819850	0.0083
120.5735	819900	0.0082
120.5809	819950	0.0079
120.5882	820000	0.0080
120.5956	820050	0.0081
120.6029	820100	0.0080
120.6103	820150	0.0082
120.6176	820200	0.0079
120.625	820250	0.0083
120.6324	820300	0.0079
120.6397	820350	0.0082
120.6471	820400	0.0083
120.6544	820450	0.0081
120.6618	820500	0.0082
120.6691	820550	0.0078
120.6765	820600	0.0082
120.6838	820650	0.0079
120.6912	820700	0.0079
120.6985	820750	0.0083
120.7059	820800	0.0083
120.7132	820850	0.0076
120.7206	820900	0.0083
120.7279	820950	0.0084
120.7353	821000	0.0080
120.7426	821050	0.0077
120.75	821100	0.0080
120.7574	821150	0.0080
120.7647	821200	0.0081
120.7721	821250	0.0084
120.7794	821300	0.0084
120.7868	821350	0.0080
120.7941	821400	0.0081
120.8015	821450	0.0082
120.8088	821500	0.0080
120.8162	821550	0.0079
120.8235	821600	0.0083
120.8309	821650	0.0079
120.8382	821700	0.0081
120.8456	821750	0.0080
120.8529	821800	0.0080
120.8603	821850	0.0083
120.8676	821900	0.0084
120.875	821950	0.0081
120.8824	822000	0.0077
120.8897	822050	0.0081
120.8971	822100	0.0081
120.9044	822150	0.0080
120.9118	822200	0.0082
120.9191	822250	0.0082
120.9265	822300	0.0083
120.9338	822350	0.0087
120.9412	822400	0.0082
120.9485	822450	0.0082
120.9559	822500	0.0078
120.9632	822550	0.0085
120.9706	822600	0.0081
120.9779	822650	0.0080
120.9853	822700	0.0081
120.9926	822750	0.0082
121.0	822800	0.0079
121.0074	822850	0.0081
121.0147	822900	0.0079
121.0221	822950	0.0082
121.0294	823000	0.0080
121.0368	823050	0.0085
121.0441	823100	0.0082
121.0515	823150	0.0086
121.0588	823200	0.0083
121.0662	823250	0.0082
121.0735	823300	0.0082
121.0809	823350	0.0082
121.0882	823400	0.0079
121.0956	823450	0.0081
121.1029	823500	0.0081
121.1103	823550	0.0081
121.1176	823600	0.0080
121.125	823650	0.0084
121.1324	823700	0.0080
121.1397	823750	0.0082
121.1471	823800	0.0080
121.1544	823850	0.0080
121.1618	823900	0.0083
121.1691	823950	0.0081
121.1765	824000	0.0079
121.1838	824050	0.0083
121.1912	824100	0.0085
121.1985	824150	0.0083
121.2059	824200	0.0085
121.2132	824250	0.0080
121.2206	824300	0.0078
121.2279	824350	0.0080
121.2353	824400	0.0080
121.2426	824450	0.0083
121.25	824500	0.0080
121.2574	824550	0.0085
121.2647	824600	0.0080
121.2721	824650	0.0078
121.2794	824700	0.0077
121.2868	824750	0.0081
121.2941	824800	0.0083
121.3015	824850	0.0081
121.3088	824900	0.0076
121.3162	824950	0.0082
121.3235	825000	0.0080
121.3309	825050	0.0079
121.3382	825100	0.0081
121.3456	825150	0.0081
121.3529	825200	0.0078
121.3603	825250	0.0083
121.3676	825300	0.0084
121.375	825350	0.0084
121.3824	825400	0.0085
121.3897	825450	0.0078
121.3971	825500	0.0085
121.4044	825550	0.0083
121.4118	825600	0.0083
121.4191	825650	0.0082
121.4265	825700	0.0082
121.4338	825750	0.0080
121.4412	825800	0.0080
121.4485	825850	0.0084
121.4559	825900	0.0082
121.4632	825950	0.0079
121.4706	826000	0.0083
121.4779	826050	0.0080
121.4853	826100	0.0079
121.4926	826150	0.0081
121.5	826200	0.0082
121.5074	826250	0.0082
121.5147	826300	0.0081
121.5221	826350	0.0082
121.5294	826400	0.0079
121.5368	826450	0.0084
121.5441	826500	0.0080
121.5515	826550	0.0080
121.5588	826600	0.0083
121.5662	826650	0.0079
121.5735	826700	0.0083
121.5809	826750	0.0082
121.5882	826800	0.0082
121.5956	826850	0.0080
121.6029	826900	0.0083
121.6103	826950	0.0080
121.6176	827000	0.0082
121.625	827050	0.0084
121.6324	827100	0.0084
121.6397	827150	0.0084
121.6471	827200	0.0081
121.6544	827250	0.0081
121.6618	827300	0.0079
121.6691	827350	0.0080
121.6765	827400	0.0079
121.6838	827450	0.0082
121.6912	827500	0.0077
121.6985	827550	0.0075
121.7059	827600	0.0081
121.7132	827650	0.0082
121.7206	827700	0.0082
121.7279	827750	0.0079
121.7353	827800	0.0086
121.7426	827850	0.0080
121.75	827900	0.0078
121.7574	827950	0.0076
121.7647	828000	0.0082
121.7721	828050	0.0078
121.7794	828100	0.0080
121.7868	828150	0.0084
121.7941	828200	0.0083
121.8015	828250	0.0080
121.8088	828300	0.0081
121.8162	828350	0.0080
121.8235	828400	0.0080
121.8309	828450	0.0087
121.8382	828500	0.0084
121.8456	828550	0.0082
121.8529	828600	0.0078
121.8603	828650	0.0081
121.8676	828700	0.0079
121.875	828750	0.0083
121.8824	828800	0.0081
121.8897	828850	0.0080
121.8971	828900	0.0085
121.9044	828950	0.0082
121.9118	829000	0.0080
121.9191	829050	0.0084
121.9265	829100	0.0080
121.9338	829150	0.0084
121.9412	829200	0.0079
121.9485	829250	0.0082
121.9559	829300	0.0080
121.9632	829350	0.0081
121.9706	829400	0.0083
121.9779	829450	0.0084
121.9853	829500	0.0079
121.9926	829550	0.0079
122.0	829600	0.0083
122.0074	829650	0.0079
122.0147	829700	0.0080
122.0221	829750	0.0080
122.0294	829800	0.0081
122.0368	829850	0.0078
122.0441	829900	0.0080
122.0515	829950	0.0079
122.0588	830000	0.0085
122.0662	830050	0.0082
122.0735	830100	0.0081
122.0809	830150	0.0077
122.0882	830200	0.0077
122.0956	830250	0.0080
122.1029	830300	0.0080
122.1103	830350	0.0076
122.1176	830400	0.0084
122.125	830450	0.0080
122.1324	830500	0.0084
122.1397	830550	0.0081
122.1471	830600	0.0083
122.1544	830650	0.0080
122.1618	830700	0.0078
122.1691	830750	0.0077
122.1765	830800	0.0079
122.1838	830850	0.0084
122.1912	830900	0.0079
122.1985	830950	0.0079
122.2059	831000	0.0079
122.2132	831050	0.0082
122.2206	831100	0.0080
122.2279	831150	0.0081
122.2353	831200	0.0078
122.2426	831250	0.0084
122.25	831300	0.0078
122.2574	831350	0.0080
122.2647	831400	0.0080
122.2721	831450	0.0084
122.2794	831500	0.0077
122.2868	831550	0.0076
122.2941	831600	0.0081
122.3015	831650	0.0083
122.3088	831700	0.0080
122.3162	831750	0.0081
122.3235	831800	0.0083
122.3309	831850	0.0082
122.3382	831900	0.0078
122.3456	831950	0.0080
122.3529	832000	0.0082
122.3603	832050	0.0083
122.3676	832100	0.0080
122.375	832150	0.0087
122.3824	832200	0.0081
122.3897	832250	0.0081
122.3971	832300	0.0081
122.4044	832350	0.0079
122.4118	832400	0.0077
122.4191	832450	0.0076
122.4265	832500	0.0079
122.4338	832550	0.0077
122.4412	832600	0.0083
122.4485	832650	0.0082
122.4559	832700	0.0083
122.4632	832750	0.0082
122.4706	832800	0.0084
122.4779	832850	0.0078
122.4853	832900	0.0073
122.4926	832950	0.0082
122.5	833000	0.0082
122.5074	833050	0.0085
122.5147	833100	0.0082
122.5221	833150	0.0080
122.5294	833200	0.0080
122.5368	833250	0.0080
122.5441	833300	0.0078
122.5515	833350	0.0079
122.5588	833400	0.0081
122.5662	833450	0.0082
122.5735	833500	0.0077
122.5809	833550	0.0081
122.5882	833600	0.0080
122.5956	833650	0.0080
122.6029	833700	0.0080
122.6103	833750	0.0081
122.6176	833800	0.0079
122.625	833850	0.0083
122.6324	833900	0.0080
122.6397	833950	0.0080
122.6471	834000	0.0079
122.6544	834050	0.0086
122.6618	834100	0.0082
122.6691	834150	0.0081
122.6765	834200	0.0078
122.6838	834250	0.0078
122.6912	834300	0.0081
122.6985	834350	0.0078
122.7059	834400	0.0077
122.7132	834450	0.0083
122.7206	834500	0.0080
122.7279	834550	0.0078
122.7353	834600	0.0084
122.7426	834650	0.0080
122.75	834700	0.0080
122.7574	834750	0.0083
122.7647	834800	0.0084
122.7721	834850	0.0084
122.7794	834900	0.0081
122.7868	834950	0.0081
122.7941	835000	0.0079
122.8015	835050	0.0083
122.8088	835100	0.0079
122.8162	835150	0.0082
122.8235	835200	0.0078
122.8309	835250	0.0079
122.8382	835300	0.0080
122.8456	835350	0.0080
122.8529	835400	0.0079
122.8603	835450	0.0081
122.8676	835500	0.0084
122.875	835550	0.0082
122.8824	835600	0.0088
122.8897	835650	0.0085
122.8971	835700	0.0081
122.9044	835750	0.0079
122.9118	835800	0.0081
122.9191	835850	0.0082
122.9265	835900	0.0083
122.9338	835950	0.0082
122.9412	836000	0.0080
122.9485	836050	0.0080
122.9559	836100	0.0086
122.9632	836150	0.0082
122.9706	836200	0.0079
122.9779	836250	0.0081
122.9853	836300	0.0080
122.9926	836350	0.0081
123.0	836400	0.0084
123.0074	836450	0.0080
123.0147	836500	0.0081
123.0221	836550	0.0079
123.0294	836600	0.0079
123.0368	836650	0.0079
123.0441	836700	0.0083
123.0515	836750	0.0080
123.0588	836800	0.0079
123.0662	836850	0.0076
123.0735	836900	0.0079
123.0809	836950	0.0078
123.0882	837000	0.0077
123.0956	837050	0.0079
123.1029	837100	0.0079
123.1103	837150	0.0079
123.1176	837200	0.0084
123.125	837250	0.0083
123.1324	837300	0.0079
123.1397	837350	0.0082
123.1471	837400	0.0079
123.1544	837450	0.0085
123.1618	837500	0.0079
123.1691	837550	0.0079
123.1765	837600	0.0079
123.1838	837650	0.0080
123.1912	837700	0.0080
123.1985	837750	0.0081
123.2059	837800	0.0083
123.2132	837850	0.0079
123.2206	837900	0.0077
123.2279	837950	0.0080
123.2353	838000	0.0082
123.2426	838050	0.0082
123.25	838100	0.0080
123.2574	838150	0.0078
123.2647	838200	0.0081
123.2721	838250	0.0081
123.2794	838300	0.0084
123.2868	838350	0.0082
123.2941	838400	0.0082
123.3015	838450	0.0079
123.3088	838500	0.0078
123.3162	838550	0.0079
123.3235	838600	0.0076
123.3309	838650	0.0078
123.3382	838700	0.0085
123.3456	838750	0.0083
123.3529	838800	0.0080
123.3603	838850	0.0079
123.3676	838900	0.0077
123.375	838950	0.0081
123.3824	839000	0.0080
123.3897	839050	0.0077
123.3971	839100	0.0080
123.4044	839150	0.0081
123.4118	839200	0.0081
123.4191	839250	0.0076
123.4265	839300	0.0079
123.4338	839350	0.0079
123.4412	839400	0.0083
123.4485	839450	0.0083
123.4559	839500	0.0082
123.4632	839550	0.0081
123.4706	839600	0.0082
123.4779	839650	0.0078
123.4853	839700	0.0076
123.4926	839750	0.0077
123.5	839800	0.0079
123.5074	839850	0.0083
123.5147	839900	0.0085
123.5221	839950	0.0080
123.5294	840000	0.0079
123.5368	840050	0.0083
123.5441	840100	0.0080
123.5515	840150	0.0085
123.5588	840200	0.0077
123.5662	840250	0.0080
123.5735	840300	0.0080
123.5809	840350	0.0081
123.5882	840400	0.0080
123.5956	840450	0.0084
123.6029	840500	0.0080
123.6103	840550	0.0077
123.6176	840600	0.0083
123.625	840650	0.0078
123.6324	840700	0.0080
123.6397	840750	0.0082
123.6471	840800	0.0081
123.6544	840850	0.0079
123.6618	840900	0.0081
123.6691	840950	0.0081
123.6765	841000	0.0084
123.6838	841050	0.0080
123.6912	841100	0.0079
123.6985	841150	0.0079
123.7059	841200	0.0079
123.7132	841250	0.0083
123.7206	841300	0.0080
123.7279	841350	0.0080
123.7353	841400	0.0085
123.7426	841450	0.0081
123.75	841500	0.0078
123.7574	841550	0.0080
123.7647	841600	0.0083
123.7721	841650	0.0077
123.7794	841700	0.0079
123.7868	841750	0.0077
123.7941	841800	0.0080
123.8015	841850	0.0084
123.8088	841900	0.0083
123.8162	841950	0.0080
123.8235	842000	0.0085
123.8309	842050	0.0083
123.8382	842100	0.0081
123.8456	842150	0.0082
123.8529	842200	0.0081
123.8603	842250	0.0076
123.8676	842300	0.0077
123.875	842350	0.0083
123.8824	842400	0.0083
123.8897	842450	0.0080
123.8971	842500	0.0079
123.9044	842550	0.0081
123.9118	842600	0.0078
123.9191	842650	0.0077
123.9265	842700	0.0080
123.9338	842750	0.0081
123.9412	842800	0.0076
123.9485	842850	0.0079
123.9559	842900	0.0078
123.9632	842950	0.0084
123.9706	843000	0.0080
123.9779	843050	0.0077
123.9853	843100	0.0084
123.9926	843150	0.0080
124.0	843200	0.0081
124.0074	843250	0.0078
124.0147	843300	0.0081
124.0221	843350	0.0081
124.0294	843400	0.0082
124.0368	843450	0.0080
124.0441	843500	0.0077
124.0515	843550	0.0078
124.0588	843600	0.0079
124.0662	843650	0.0084
124.0735	843700	0.0082
124.0809	843750	0.0076
124.0882	843800	0.0083
124.0956	843850	0.0081
124.1029	843900	0.0076
124.1103	843950	0.0076
124.1176	844000	0.0082
124.125	844050	0.0081
124.1324	844100	0.0076
124.1397	844150	0.0078
124.1471	844200	0.0077
124.1544	844250	0.0079
124.1618	844300	0.0081
124.1691	844350	0.0082
124.1765	844400	0.0081
124.1838	844450	0.0080
124.1912	844500	0.0076
124.1985	844550	0.0079
124.2059	844600	0.0080
124.2132	844650	0.0081
124.2206	844700	0.0079
124.2279	844750	0.0079
124.2353	844800	0.0084
124.2426	844850	0.0081
124.25	844900	0.0079
124.2574	844950	0.0081
124.2647	845000	0.0081
124.2721	845050	0.0081
124.2794	845100	0.0079
124.2868	845150	0.0080
124.2941	845200	0.0082
124.3015	845250	0.0081
124.3088	845300	0.0078
124.3162	845350	0.0077
124.3235	845400	0.0080
124.3309	845450	0.0079
124.3382	845500	0.0076
124.3456	845550	0.0080
124.3529	845600	0.0083
124.3603	845650	0.0080
124.3676	845700	0.0077
124.375	845750	0.0082
124.3824	845800	0.0077
124.3897	845850	0.0079
124.3971	845900	0.0083
124.4044	845950	0.0079
124.4118	846000	0.0077
124.4191	846050	0.0082
124.4265	846100	0.0080
124.4338	846150	0.0080
124.4412	846200	0.0078
124.4485	846250	0.0075
124.4559	846300	0.0085
124.4632	846350	0.0080
124.4706	846400	0.0078
124.4779	846450	0.0080
124.4853	846500	0.0079
124.4926	846550	0.0078
124.5	846600	0.0080
124.5074	846650	0.0081
124.5147	846700	0.0079
124.5221	846750	0.0081
124.5294	846800	0.0080
124.5368	846850	0.0080
124.5441	846900	0.0082
124.5515	846950	0.0079
124.5588	847000	0.0081
124.5662	847050	0.0083
124.5735	847100	0.0083
124.5809	847150	0.0078
124.5882	847200	0.0081
124.5956	847250	0.0077
124.6029	847300	0.0079
124.6103	847350	0.0080
124.6176	847400	0.0079
124.625	847450	0.0078
124.6324	847500	0.0081
124.6397	847550	0.0077
124.6471	847600	0.0081
124.6544	847650	0.0080
124.6618	847700	0.0082
124.6691	847750	0.0083
124.6765	847800	0.0079
124.6838	847850	0.0082
124.6912	847900	0.0081
124.6985	847950	0.0082
124.7059	848000	0.0080
124.7132	848050	0.0078
124.7206	848100	0.0081
124.7279	848150	0.0079
124.7353	848200	0.0082
124.7426	848250	0.0080
124.75	848300	0.0079
124.7574	848350	0.0080
124.7647	848400	0.0078
124.7721	848450	0.0084
124.7794	848500	0.0081
124.7868	848550	0.0079
124.7941	848600	0.0077
124.8015	848650	0.0079
124.8088	848700	0.0081
124.8162	848750	0.0081
124.8235	848800	0.0082
124.8309	848850	0.0081
124.8382	848900	0.0078
124.8456	848950	0.0082
124.8529	849000	0.0078
124.8603	849050	0.0081
124.8676	849100	0.0083
124.875	849150	0.0079
124.8824	849200	0.0080
124.8897	849250	0.0078
124.8971	849300	0.0078
124.9044	849350	0.0079
124.9118	849400	0.0080
124.9191	849450	0.0081
124.9265	849500	0.0079
124.9338	849550	0.0083
124.9412	849600	0.0081
124.9485	849650	0.0079
124.9559	849700	0.0085
124.9632	849750	0.0077
124.9706	849800	0.0083
124.9779	849850	0.0078
124.9853	849900	0.0081
124.9926	849950	0.0081
125.0	850000	0.0080

Training Time

Training: 7.0 hours

Framework Versions

Python: 3.13.15
Sentence Transformers: 5.7.0
Transformers: 5.16.1
PyTorch: 2.11.0+cu128
Accelerate: 1.14.0
Datasets: 4.8.5
Tokenizers: 0.23.1

Additional Resources

Training and Finetuning Embedding Models with Sentence Transformers: the end-to-end guide for training or finetuning Sentence Transformer models.
Introduction to Matryoshka Embedding Models: variable-size embeddings that can be truncated with minimal quality loss.
Binary and Scalar Embedding Quantization for Significantly Faster & Cheaper Retrieval: post-training compression of embedding vectors.
Multimodal Embedding & Reranker Models with Sentence Transformers: use text, image, audio, and video models through the same API.
Training and Finetuning Multimodal Embedding & Reranker Models with Sentence Transformers: train multimodal embedding models, with a Visual Document Retrieval walkthrough.

Citation

BibTeX

Sentence Transformers

@inproceedings{reimers-2019-sentence-bert,
    title = "Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks",
    author = "Reimers, Nils and Gurevych, Iryna",
    booktitle = "Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing",
    month = "11",
    year = "2019",
    publisher = "Association for Computational Linguistics",
    url = "https://arxiv.org/abs/1908.10084",
}

MarginMSELoss

@misc{hofstätter2021improving,
    title={Improving Efficient Neural Ranking Models with Cross-Architecture Knowledge Distillation},
    author={Sebastian Hofstätter and Sophia Althammer and Michael Schröder and Mete Sertkan and Allan Hanbury},
    year={2021},
    eprint={2010.02666},
    archivePrefix={arXiv},
    primaryClass={cs.IR}
}