Hugging Face's logo

johnyy212
/
moe-girl-test

Sentence Similarity
sentence-transformers
Safetensors
bert
feature-extraction
Generated from Trainer
dataset_size:30620
loss:MultipleNegativesRankingLoss
text-embeddings-inference
Model card Files
xet
 Community

SentenceTransformer based on BAAI/bge-small-zh-v1.5

This is a sentence-transformers model finetuned from BAAI/bge-small-zh-v1.5 on the json dataset. It maps sentences & paragraphs to a 512-dimensional dense vector space and can be used for retrieval.

Model Details

Model Description

  • Model Type: Sentence Transformer
  • Base model: BAAI/bge-small-zh-v1.5
  • Maximum Sequence Length: 512 tokens
  • Output Dimensionality: 512 dimensions
  • Similarity Function: Cosine Similarity
  • Supported Modality: Text
  • Training Dataset:
    • json

Model Sources

Full Model Architecture 

SentenceTransformer(
  (0): Transformer({'transformer_task': 'feature-extraction', 'modality_config': {'text': {'method': 'forward', 'method_output_name': 'last_hidden_state'}}, 'module_output_name': 'token_embeddings', 'architecture': 'BertModel'})
  (1): Pooling({'embedding_dimension': 512, 'pooling_mode': 'cls', 'include_prompt': True})
  (2): Normalize({})
)

Usage

Direct Usage (Sentence Transformers)

First install the Sentence Transformers library:

pip install -U sentence-transformers

Then you can load this model and run inference.

from sentence_transformers import SentenceTransformer

# Download from the 🤗 Hub
model = SentenceTransformer("johnyy212/moe-girl-test")
# Run inference
sentences = [
    '黑发、眼镜、外柔内刚、宅男、黑瞳、弱气',
    '角色：德怀特·费菲尔德\n本名：Dwight Fairfield\n别名：汪涵、领导、光头\n声优：Ian Chuprun\n发色：黑\n瞳色：黑\n身高：169\n萌点：眼镜、弱气、外柔内刚、上班族、宅男',
    '德怀特·费菲尔德',
]
embeddings = model.encode(sentences)
print(embeddings.shape)
# [3, 512]

# Get the similarity scores for the embeddings
similarities = model.similarity(embeddings, embeddings)
print(similarities)
# tensor([[ 1.0000,  0.7818, -0.2186],
#         [ 0.7818,  1.0000, -0.0236],
#         [-0.2186, -0.0236,  1.0000]])

Training Details

Training Dataset

json

  • Dataset: json
  • Size: 30,620 training samples
  • Columns: anchor, positive, and char_name
  • Approximate statistics based on the first 1000 samples:
    anchor positive char_name
    type string string string
    details
    • min: 9 tokens
    • mean: 15.89 tokens
    • max: 30 tokens
    • min: 29 tokens
    • mean: 80.66 tokens
    • max: 347 tokens
    • min: 3 tokens
    • mean: 7.39 tokens
    • max: 20 tokens
  • Samples:
    anchor positive char_name
    学姐、辣妹、绿瞳、及膝袜 角色：白石奈奈
    本名：白石奈々
    别名：学生会长、绚濑绘里、早坂爱、金发学姐
    声优：M.A.O
    发色：金
    瞳色：绿
    生日：5月30日
    星座：双子
    血型：O
    萌点：辣妹、白丝、学姐    		 白石奈奈
    挑染、酒保、紫瞳 角色：陶陶
    本名：陶陶
    别名：陶宝
    发色：紫
    瞳色：紫
    萌点：大学生、酒保、巨乳    		 陶陶
    紫瞳、粉发、乳袋 角色：美少女花骑士:蔓炎花
    本名：マネッチア
    别名：蔓炎花、火焰草、糖果玉米藤
    发色：粉
    瞳色：紫
    萌点：魔、御姐、乳袋、船形帽、过膝袜、好酒、烧毁    		 美少女花骑士:蔓炎花
  • Loss: MultipleNegativesRankingLoss with these parameters:
    {
    "scale": 20.0,
    "similarity_fct": "cos_sim",
    "gather_across_devices": false,
    "directions": [
        "query_to_doc"
    ],
    "partition_mode": "joint",
    "hardness_mode": null,
    "hardness_strength": 0.0
}

Evaluation Dataset

json

  • Dataset: json
  • Size: 3,403 evaluation samples
  • Columns: anchor, positive, and char_name
  • Approximate statistics based on the first 1000 samples:
    anchor positive char_name
    type string string string
    details
    • min: 9 tokens
    • mean: 15.89 tokens
    • max: 31 tokens
    • min: 26 tokens
    • mean: 79.83 tokens
    • max: 276 tokens
    • min: 3 tokens
    • mean: 7.21 tokens
    • max: 22 tokens
  • Samples:
    anchor positive char_name
    御姐、蓝瞳、蓝发 角色：卡露娜
    本名：カルナ
    声优：柚木凉香
    发色：蓝
    瞳色：蓝
    萌点：御姐、马尾    		 卡露娜
    露肩装、绿瞳、长发、卷发、科学家 角色：冷周六
    发色：青
    瞳色：浅绿
    生日：12月23日
    星座：摩羯
    萌点：科学家、长发、卷发、露肩装、中靴    		 冷周六
    枪械、爸爸、A型、转轮手枪 角色：巴瑞·波顿
    本名：バリー・バートン
    别名：巴瑞
    声优：Barry Gjerde、Ed Smaron、Jamieson K. Price、Michael McConnohie、屋良有作
    发色：棕
    瞳色：棕
    年龄：38
    血型：A
    萌点：大叔、爸爸、手枪、转轮手枪、飞行员、特警、战术背心    		 巴瑞·波顿
  • Loss: MultipleNegativesRankingLoss with these parameters:
    {
    "scale": 20.0,
    "similarity_fct": "cos_sim",
    "gather_across_devices": false,
    "directions": [
        "query_to_doc"
    ],
    "partition_mode": "joint",
    "hardness_mode": null,
    "hardness_strength": 0.0
}

Training Hyperparameters

Non-Default Hyperparameters

  • per_device_train_batch_size: 64
  • learning_rate: 2e-05
  • warmup_steps: 0.1
  • fp16: True
  • load_best_model_at_end: True
  • batch_sampler: no_duplicates

All Hyperparameters

Click to expand
  • do_predict: False
  • prediction_loss_only: True
  • per_device_train_batch_size: 64
  • per_device_eval_batch_size: 8
  • gradient_accumulation_steps: 1
  • eval_accumulation_steps: None
  • torch_empty_cache_steps: None
  • learning_rate: 2e-05
  • weight_decay: 0.0
  • adam_beta1: 0.9
  • adam_beta2: 0.999
  • adam_epsilon: 1e-08
  • max_grad_norm: 1.0
  • num_train_epochs: 3
  • max_steps: -1
  • lr_scheduler_type: linear
  • lr_scheduler_kwargs: None
  • warmup_ratio: None
  • warmup_steps: 0.1
  • log_level: passive
  • log_level_replica: warning
  • log_on_each_node: True
  • logging_nan_inf_filter: True
  • enable_jit_checkpoint: False
  • save_on_each_node: False
  • save_only_model: False
  • restore_callback_states_from_checkpoint: False
  • use_cpu: False
  • seed: 42
  • data_seed: None
  • bf16: False
  • fp16: True
  • bf16_full_eval: False
  • fp16_full_eval: False
  • tf32: None
  • local_rank: -1
  • ddp_backend: None
  • debug: []
  • dataloader_drop_last: False
  • dataloader_num_workers: 0
  • dataloader_prefetch_factor: None
  • disable_tqdm: False
  • remove_unused_columns: True
  • label_names: None
  • load_best_model_at_end: True
  • ignore_data_skip: False
  • fsdp: []
  • fsdp_config: {'min_num_params': 0, 'xla': False, 'xla_fsdp_v2': False, 'xla_fsdp_grad_ckpt': False}
  • accelerator_config: {'split_batches': False, 'dispatch_batches': None, 'even_batches': True, 'use_seedable_sampler': True, 'non_blocking': False, 'gradient_accumulation_kwargs': None}
  • parallelism_config: None
  • deepspeed: None
  • label_smoothing_factor: 0.0
  • optim: adamw_torch_fused
  • optim_args: None
  • group_by_length: False
  • length_column_name: length
  • project: huggingface
  • trackio_space_id: trackio
  • ddp_find_unused_parameters: None
  • ddp_bucket_cap_mb: None
  • ddp_broadcast_buffers: False
  • dataloader_pin_memory: True
  • dataloader_persistent_workers: False
  • skip_memory_metrics: True
  • push_to_hub: False
  • resume_from_checkpoint: None
  • hub_model_id: None
  • hub_strategy: every_save
  • hub_private_repo: None
  • hub_always_push: False
  • hub_revision: None
  • gradient_checkpointing: False
  • gradient_checkpointing_kwargs: None
  • include_for_metrics: []
  • eval_do_concat_batches: True
  • auto_find_batch_size: False
  • full_determinism: False
  • ddp_timeout: 1800
  • torch_compile: False
  • torch_compile_backend: None
  • torch_compile_mode: None
  • include_num_input_tokens_seen: no
  • neftune_noise_alpha: None
  • optim_target_modules: None
  • batch_eval_metrics: False
  • eval_on_start: False
  • use_liger_kernel: False
  • liger_kernel_config: None
  • eval_use_gather_object: False
  • average_tokens_across_devices: True
  • use_cache: False
  • prompts: None
  • batch_sampler: no_duplicates
  • multi_dataset_batch_sampler: proportional
  • router_mapping: {}
  • learning_rate_mapping: {}

Training Logs

Epoch Step Training Loss Validation Loss
0.2088 100 1.8291 -
0.4175 200 0.7744 -
0.6263 300 0.6619 -
0.8351 400 0.5894 -
1.0 479 - 0.0947
1.0438 500 0.5654 -
1.2526 600 0.4878 -
1.4614 700 0.4823 -
1.6701 800 0.4479 -
1.8789 900 0.4625 -
2.0 958 - 0.0796
2.0877 1000 0.4401 -
2.2965 1100 0.4072 -
2.5052 1200 0.3933 -
2.7140 1300 0.4178 -
2.9228 1400 0.4235 -
3.0 1437 - 0.075
  • The bold row denotes the saved checkpoint.

Training Time

  • Training: 13.9 minutes

Framework Versions

  • Python: 3.12.13
  • Sentence Transformers: 5.4.1
  • Transformers: 5.0.0
  • PyTorch: 2.10.0+cu128
  • Accelerate: 1.13.0
  • Datasets: 4.8.5
  • Tokenizers: 0.22.2

Citation

BibTeX

Sentence Transformers 

@inproceedings{reimers-2019-sentence-bert,
    title = "Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks",
    author = "Reimers, Nils and Gurevych, Iryna",
    booktitle = "Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing",
    month = "11",
    year = "2019",
    publisher = "Association for Computational Linguistics",
    url = "https://arxiv.org/abs/1908.10084",
}

MultipleNegativesRankingLoss 

@misc{oord2019representationlearningcontrastivepredictive,
      title={Representation Learning with Contrastive Predictive Coding},
      author={Aaron van den Oord and Yazhe Li and Oriol Vinyals},
      year={2019},
      eprint={1807.03748},
      archivePrefix={arXiv},
      primaryClass={cs.LG},
      url={https://arxiv.org/abs/1807.03748},
}
Downloads last month
77
Safetensors
Model size
24M params
Tensor type
F32
·
Inference Providers NEW
Sentence Similarity
This model isn't deployed by any Inference Provider. 🙋 Ask for provider support

Model tree for johnyy212/moe-girl-test

Base model

BAAI/bge-small-zh-v1.5
Finetuned
(9)
this model

Papers for johnyy212/moe-girl-test