metadata
language: []
library_name: sentence-transformers
tags:
- sentence-transformers
- sentence-similarity
- feature-extraction
- generated_from_trainer
- dataset_size:36755
- loss:MatryoshkaLoss
- loss:MultipleNegativesRankingLoss
base_model: intfloat/multilingual-e5-base
datasets: []
metrics:
- cosine_accuracy@1
- cosine_accuracy@3
- cosine_accuracy@5
- cosine_accuracy@10
- cosine_precision@1
- cosine_precision@3
- cosine_precision@5
- cosine_precision@10
- cosine_recall@1
- cosine_recall@3
- cosine_recall@5
- cosine_recall@10
- cosine_ndcg@10
- cosine_mrr@10
- cosine_map@100
widget:
- source_sentence: >-
Sexual assault by bodily force at industrial and construction area While
engaged in other types of work
sentences:
- >-
การถูกทำร้ายทางเพศด้วยกำลังกาย
พื้นที่อุตสาหกรรมและก่อสร้างขณะทำงานประเภทอื่น
- ยาต้านมาเลเรียและยาที่ออกฤทธิ์ต่อโปรโตซัวอื่นในเลือด
- >-
อุบัติเหตุยานพาหนะทางน้ำทำให้จมน้ำตายและจมน้ำ
จากยานพาหนะทางน้ำอื่นที่ไม่มีเครื่องยนต์ขณะทำงานประเภทอื่น
- source_sentence: Peripheral nerves of head, face and neck malignant neoplasm
sentences:
- การสัมผัสปัจจัยอื่นที่ระบุรายละเอียด ที่พักอาศัยรวมขณะทำกิจกรรมกีฬา
- เนื้องอกร้ายของเส้นประสาทส่วนปลายของศีรษะ หน้า และคอ
- การใส่เครื่องดึงสูญญากาศและคีมช่วยคลอดไม่สำเร็จ ไม่ระบุรายละเอียด
- source_sentence: >-
Diving or jumping into water causing injury other than drowning or
submersion at school,other institution and public administrative area
sentences:
- >-
กระแทกกับวัตถุที่ถูกขว้าง โยน หรือหล่นใส่
พื้นที่การค้าและบริการขณะพักผ่อนหลับรับประทานอาหารหรือประกอบกิจกรรมในชีวิตประจำวัน
- >-
การดำหรือกระโดดลงไปในน้ำทำให้บาดเจ็บอย่างอื่นนอกเหนือจากการจมน้ำตายหรือจมน้ำ
ที่โรงเรียน สถาบันอื่นและพื้นที่สาธารณะ
- >-
การเป็นพิษโดยอุบัติเหตุจากยาต้านลมชักยาระงับประสาท-ยานอนหลับ
ยาต้านพาร์คินสัน และยาที่ออกฤทธิ์ต่อจิตใจ มิได้จำแนกไว้ที่ใด
ที่สนามกีฬาและพื้นที่เล่นกีฬาขณะทำงานประเภทอื่น
- source_sentence: >-
Bus occupant injured in collision with two- or three motor vehicle at
Driver injured in nontraffic accident During unspecified activity
sentences:
- เนื้องอกไม่ร้ายขององคชาต
- >-
ผู้ใช้รถโดยสารบาดเจ็บเพราะชนกับยานยนต์สองหรือสามล้อ
ผู้ขับขี่บาดเจ็บในอุบัติเหตุที่ไม่ใช่อุบัติเหตุจราจรขณะทำกิจกรรมอื่นที่ไม่ระบุรายละเอียด
- >-
ผลที่ตามมาของการถูกความร้อนและสารกัดกร่อนที่จำแนกเฉพาะตามปริมาณพื้นที่ผิวกายที่เป็นแผล
- source_sentence: >-
Contact with other powered hand tools and household machinery at
residential institution While engaged in leisure activity
sentences:
- >-
การถูกทำร้ายด้วยยาฆ่าศัตรูพืชและสัตว์
ที่พักอาศัยรวมขณะทำกิจกรรมอื่นที่ระบุรายละเอียด
- >-
สัมผัสกับเครื่องมือที่มีเครื่องยนต์และเครื่องจักรกลอื่นในบ้าน
ที่พักอาศัยรวมขณะทำกิจกรรมยามว่าง
- หลอดเลือดอักเสบรูมาตอยด์
pipeline_tag: sentence-similarity
model-index:
- name: SentenceTransformer based on intfloat/multilingual-e5-base
results:
- task:
type: information-retrieval
name: Information Retrieval
dataset:
name: dim 768
type: dim_768
metrics:
- type: cosine_accuracy@1
value: 0.876101860920666
name: Cosine Accuracy@1
- type: cosine_accuracy@3
value: 0.9407443682664055
name: Cosine Accuracy@3
- type: cosine_accuracy@5
value: 0.9569049951028403
name: Cosine Accuracy@5
- type: cosine_accuracy@10
value: 0.9718413320274241
name: Cosine Accuracy@10
- type: cosine_precision@1
value: 0.876101860920666
name: Cosine Precision@1
- type: cosine_precision@3
value: 0.3135814560888018
name: Cosine Precision@3
- type: cosine_precision@5
value: 0.1913809990205681
name: Cosine Precision@5
- type: cosine_precision@10
value: 0.09718413320274243
name: Cosine Precision@10
- type: cosine_recall@1
value: 0.876101860920666
name: Cosine Recall@1
- type: cosine_recall@3
value: 0.9407443682664055
name: Cosine Recall@3
- type: cosine_recall@5
value: 0.9569049951028403
name: Cosine Recall@5
- type: cosine_recall@10
value: 0.9718413320274241
name: Cosine Recall@10
- type: cosine_ndcg@10
value: 0.9263587749780547
name: Cosine Ndcg@10
- type: cosine_mrr@10
value: 0.9115257178614189
name: Cosine Mrr@10
- type: cosine_map@100
value: 0.9126820654456553
name: Cosine Map@100
- task:
type: information-retrieval
name: Information Retrieval
dataset:
name: dim 512
type: dim_512
metrics:
- type: cosine_accuracy@1
value: 0.8768364348677767
name: Cosine Accuracy@1
- type: cosine_accuracy@3
value: 0.9400097943192948
name: Cosine Accuracy@3
- type: cosine_accuracy@5
value: 0.9576395690499511
name: Cosine Accuracy@5
- type: cosine_accuracy@10
value: 0.9715964740450539
name: Cosine Accuracy@10
- type: cosine_precision@1
value: 0.8768364348677767
name: Cosine Precision@1
- type: cosine_precision@3
value: 0.31333659810643155
name: Cosine Precision@3
- type: cosine_precision@5
value: 0.19152791380999024
name: Cosine Precision@5
- type: cosine_precision@10
value: 0.0971596474045054
name: Cosine Precision@10
- type: cosine_recall@1
value: 0.8768364348677767
name: Cosine Recall@1
- type: cosine_recall@3
value: 0.9400097943192948
name: Cosine Recall@3
- type: cosine_recall@5
value: 0.9576395690499511
name: Cosine Recall@5
- type: cosine_recall@10
value: 0.9715964740450539
name: Cosine Recall@10
- type: cosine_ndcg@10
value: 0.9265125019493139
name: Cosine Ndcg@10
- type: cosine_mrr@10
value: 0.9118172154594785
name: Cosine Mrr@10
- type: cosine_map@100
value: 0.9130070367510391
name: Cosine Map@100
- task:
type: information-retrieval
name: Information Retrieval
dataset:
name: dim 256
type: dim_256
metrics:
- type: cosine_accuracy@1
value: 0.876101860920666
name: Cosine Accuracy@1
- type: cosine_accuracy@3
value: 0.9375612144955926
name: Cosine Accuracy@3
- type: cosine_accuracy@5
value: 0.9556807051909892
name: Cosine Accuracy@5
- type: cosine_accuracy@10
value: 0.9713516160626836
name: Cosine Accuracy@10
- type: cosine_precision@1
value: 0.876101860920666
name: Cosine Precision@1
- type: cosine_precision@3
value: 0.3125204048318641
name: Cosine Precision@3
- type: cosine_precision@5
value: 0.19113614103819787
name: Cosine Precision@5
- type: cosine_precision@10
value: 0.09713516160626838
name: Cosine Precision@10
- type: cosine_recall@1
value: 0.876101860920666
name: Cosine Recall@1
- type: cosine_recall@3
value: 0.9375612144955926
name: Cosine Recall@3
- type: cosine_recall@5
value: 0.9556807051909892
name: Cosine Recall@5
- type: cosine_recall@10
value: 0.9713516160626836
name: Cosine Recall@10
- type: cosine_ndcg@10
value: 0.9256666603512674
name: Cosine Ndcg@10
- type: cosine_mrr@10
value: 0.9108620749965022
name: Cosine Mrr@10
- type: cosine_map@100
value: 0.9120751966168019
name: Cosine Map@100
- task:
type: information-retrieval
name: Information Retrieval
dataset:
name: dim 128
type: dim_128
metrics:
- type: cosine_accuracy@1
value: 0.8704701273261508
name: Cosine Accuracy@1
- type: cosine_accuracy@3
value: 0.9365817825661117
name: Cosine Accuracy@3
- type: cosine_accuracy@5
value: 0.9537218413320274
name: Cosine Accuracy@5
- type: cosine_accuracy@10
value: 0.9711067580803134
name: Cosine Accuracy@10
- type: cosine_precision@1
value: 0.8704701273261508
name: Cosine Precision@1
- type: cosine_precision@3
value: 0.31219392752203723
name: Cosine Precision@3
- type: cosine_precision@5
value: 0.1907443682664055
name: Cosine Precision@5
- type: cosine_precision@10
value: 0.09711067580803134
name: Cosine Precision@10
- type: cosine_recall@1
value: 0.8704701273261508
name: Cosine Recall@1
- type: cosine_recall@3
value: 0.9365817825661117
name: Cosine Recall@3
- type: cosine_recall@5
value: 0.9537218413320274
name: Cosine Recall@5
- type: cosine_recall@10
value: 0.9711067580803134
name: Cosine Recall@10
- type: cosine_ndcg@10
value: 0.9229543066733635
name: Cosine Ndcg@10
- type: cosine_mrr@10
value: 0.9073412698412701
name: Cosine Mrr@10
- type: cosine_map@100
value: 0.9085589789318891
name: Cosine Map@100
- task:
type: information-retrieval
name: Information Retrieval
dataset:
name: dim 64
type: dim_64
metrics:
- type: cosine_accuracy@1
value: 0.8665523996082273
name: Cosine Accuracy@1
- type: cosine_accuracy@3
value: 0.9329089128305583
name: Cosine Accuracy@3
- type: cosine_accuracy@5
value: 0.9502938295788442
name: Cosine Accuracy@5
- type: cosine_accuracy@10
value: 0.9708619000979432
name: Cosine Accuracy@10
- type: cosine_precision@1
value: 0.8665523996082273
name: Cosine Precision@1
- type: cosine_precision@3
value: 0.31096963761018603
name: Cosine Precision@3
- type: cosine_precision@5
value: 0.19005876591576887
name: Cosine Precision@5
- type: cosine_precision@10
value: 0.09708619000979433
name: Cosine Precision@10
- type: cosine_recall@1
value: 0.8665523996082273
name: Cosine Recall@1
- type: cosine_recall@3
value: 0.9329089128305583
name: Cosine Recall@3
- type: cosine_recall@5
value: 0.9502938295788442
name: Cosine Recall@5
- type: cosine_recall@10
value: 0.9708619000979432
name: Cosine Recall@10
- type: cosine_ndcg@10
value: 0.9209566810099995
name: Cosine Ndcg@10
- type: cosine_mrr@10
value: 0.9047911516875767
name: Cosine Mrr@10
- type: cosine_map@100
value: 0.9059398753868528
name: Cosine Map@100
SentenceTransformer based on intfloat/multilingual-e5-base
This is a sentence-transformers model finetuned from intfloat/multilingual-e5-base. It maps sentences & paragraphs to a 768-dimensional dense vector space and can be used for semantic textual similarity, semantic search, paraphrase mining, text classification, clustering, and more.
Model Details
Model Description
- Model Type: Sentence Transformer
- Base model: intfloat/multilingual-e5-base
- Maximum Sequence Length: 512 tokens
- Output Dimensionality: 768 tokens
- Similarity Function: Cosine Similarity
Model Sources
Full Model Architecture
SentenceTransformer(
(0): Transformer({'max_seq_length': 512, 'do_lower_case': False}) with Transformer model: XLMRobertaModel
(1): Pooling({'word_embedding_dimension': 768, 'pooling_mode_cls_token': False, 'pooling_mode_mean_tokens': True, 'pooling_mode_max_tokens': False, 'pooling_mode_mean_sqrt_len_tokens': False, 'pooling_mode_weightedmean_tokens': False, 'pooling_mode_lasttoken': False, 'include_prompt': True})
(2): Normalize()
)
Usage
Direct Usage (Sentence Transformers)
First install the Sentence Transformers library:
pip install -U sentence-transformers
Then you can load this model and run inference.
from sentence_transformers import SentenceTransformer
model = SentenceTransformer("BotnoiNLPteam/me5_icd10_test")
sentences = [
'Contact with other powered hand tools and household machinery at residential institution While engaged in leisure activity',
'สัมผัสกับเครื่องมือที่มีเครื่องยนต์และเครื่องจักรกลอื่นในบ้าน ที่พักอาศัยรวมขณะทำกิจกรรมยามว่าง',
'หลอดเลือดอักเสบรูมาตอยด์',
]
embeddings = model.encode(sentences)
print(embeddings.shape)
similarities = model.similarity(embeddings, embeddings)
print(similarities.shape)
Evaluation
Metrics
Information Retrieval
Metric |
Value |
cosine_accuracy@1 |
0.8761 |
cosine_accuracy@3 |
0.9407 |
cosine_accuracy@5 |
0.9569 |
cosine_accuracy@10 |
0.9718 |
cosine_precision@1 |
0.8761 |
cosine_precision@3 |
0.3136 |
cosine_precision@5 |
0.1914 |
cosine_precision@10 |
0.0972 |
cosine_recall@1 |
0.8761 |
cosine_recall@3 |
0.9407 |
cosine_recall@5 |
0.9569 |
cosine_recall@10 |
0.9718 |
cosine_ndcg@10 |
0.9264 |
cosine_mrr@10 |
0.9115 |
cosine_map@100 |
0.9127 |
Information Retrieval
Metric |
Value |
cosine_accuracy@1 |
0.8768 |
cosine_accuracy@3 |
0.94 |
cosine_accuracy@5 |
0.9576 |
cosine_accuracy@10 |
0.9716 |
cosine_precision@1 |
0.8768 |
cosine_precision@3 |
0.3133 |
cosine_precision@5 |
0.1915 |
cosine_precision@10 |
0.0972 |
cosine_recall@1 |
0.8768 |
cosine_recall@3 |
0.94 |
cosine_recall@5 |
0.9576 |
cosine_recall@10 |
0.9716 |
cosine_ndcg@10 |
0.9265 |
cosine_mrr@10 |
0.9118 |
cosine_map@100 |
0.913 |
Information Retrieval
Metric |
Value |
cosine_accuracy@1 |
0.8761 |
cosine_accuracy@3 |
0.9376 |
cosine_accuracy@5 |
0.9557 |
cosine_accuracy@10 |
0.9714 |
cosine_precision@1 |
0.8761 |
cosine_precision@3 |
0.3125 |
cosine_precision@5 |
0.1911 |
cosine_precision@10 |
0.0971 |
cosine_recall@1 |
0.8761 |
cosine_recall@3 |
0.9376 |
cosine_recall@5 |
0.9557 |
cosine_recall@10 |
0.9714 |
cosine_ndcg@10 |
0.9257 |
cosine_mrr@10 |
0.9109 |
cosine_map@100 |
0.9121 |
Information Retrieval
Metric |
Value |
cosine_accuracy@1 |
0.8705 |
cosine_accuracy@3 |
0.9366 |
cosine_accuracy@5 |
0.9537 |
cosine_accuracy@10 |
0.9711 |
cosine_precision@1 |
0.8705 |
cosine_precision@3 |
0.3122 |
cosine_precision@5 |
0.1907 |
cosine_precision@10 |
0.0971 |
cosine_recall@1 |
0.8705 |
cosine_recall@3 |
0.9366 |
cosine_recall@5 |
0.9537 |
cosine_recall@10 |
0.9711 |
cosine_ndcg@10 |
0.923 |
cosine_mrr@10 |
0.9073 |
cosine_map@100 |
0.9086 |
Information Retrieval
Metric |
Value |
cosine_accuracy@1 |
0.8666 |
cosine_accuracy@3 |
0.9329 |
cosine_accuracy@5 |
0.9503 |
cosine_accuracy@10 |
0.9709 |
cosine_precision@1 |
0.8666 |
cosine_precision@3 |
0.311 |
cosine_precision@5 |
0.1901 |
cosine_precision@10 |
0.0971 |
cosine_recall@1 |
0.8666 |
cosine_recall@3 |
0.9329 |
cosine_recall@5 |
0.9503 |
cosine_recall@10 |
0.9709 |
cosine_ndcg@10 |
0.921 |
cosine_mrr@10 |
0.9048 |
cosine_map@100 |
0.9059 |
Training Details
Training Hyperparameters
Non-Default Hyperparameters
eval_strategy
: epoch
per_device_train_batch_size
: 16
per_device_eval_batch_size
: 16
num_train_epochs
: 5
warmup_ratio
: 0.1
bf16
: True
batch_sampler
: no_duplicates
All Hyperparameters
Click to expand
overwrite_output_dir
: False
do_predict
: False
eval_strategy
: epoch
prediction_loss_only
: True
per_device_train_batch_size
: 16
per_device_eval_batch_size
: 16
per_gpu_train_batch_size
: None
per_gpu_eval_batch_size
: None
gradient_accumulation_steps
: 1
eval_accumulation_steps
: None
learning_rate
: 5e-05
weight_decay
: 0.0
adam_beta1
: 0.9
adam_beta2
: 0.999
adam_epsilon
: 1e-08
max_grad_norm
: 1.0
num_train_epochs
: 5
max_steps
: -1
lr_scheduler_type
: linear
lr_scheduler_kwargs
: {}
warmup_ratio
: 0.1
warmup_steps
: 0
log_level
: passive
log_level_replica
: warning
log_on_each_node
: True
logging_nan_inf_filter
: True
save_safetensors
: True
save_on_each_node
: False
save_only_model
: False
restore_callback_states_from_checkpoint
: False
no_cuda
: False
use_cpu
: False
use_mps_device
: False
seed
: 42
data_seed
: None
jit_mode_eval
: False
use_ipex
: False
bf16
: True
fp16
: False
fp16_opt_level
: O1
half_precision_backend
: auto
bf16_full_eval
: False
fp16_full_eval
: False
tf32
: None
local_rank
: 0
ddp_backend
: None
tpu_num_cores
: None
tpu_metrics_debug
: False
debug
: []
dataloader_drop_last
: False
dataloader_num_workers
: 0
dataloader_prefetch_factor
: None
past_index
: -1
disable_tqdm
: False
remove_unused_columns
: True
label_names
: None
load_best_model_at_end
: False
ignore_data_skip
: False
fsdp
: []
fsdp_min_num_params
: 0
fsdp_config
: {'min_num_params': 0, 'xla': False, 'xla_fsdp_v2': False, 'xla_fsdp_grad_ckpt': False}
fsdp_transformer_layer_cls_to_wrap
: None
accelerator_config
: {'split_batches': False, 'dispatch_batches': None, 'even_batches': True, 'use_seedable_sampler': True, 'non_blocking': False, 'gradient_accumulation_kwargs': None}
deepspeed
: None
label_smoothing_factor
: 0.0
optim
: adamw_torch
optim_args
: None
adafactor
: False
group_by_length
: False
length_column_name
: length
ddp_find_unused_parameters
: None
ddp_bucket_cap_mb
: None
ddp_broadcast_buffers
: False
dataloader_pin_memory
: True
dataloader_persistent_workers
: False
skip_memory_metrics
: True
use_legacy_prediction_loop
: False
push_to_hub
: False
resume_from_checkpoint
: None
hub_model_id
: None
hub_strategy
: every_save
hub_private_repo
: False
hub_always_push
: False
gradient_checkpointing
: False
gradient_checkpointing_kwargs
: None
include_inputs_for_metrics
: False
eval_do_concat_batches
: True
fp16_backend
: auto
push_to_hub_model_id
: None
push_to_hub_organization
: None
mp_parameters
:
auto_find_batch_size
: False
full_determinism
: False
torchdynamo
: None
ray_scope
: last
ddp_timeout
: 1800
torch_compile
: False
torch_compile_backend
: None
torch_compile_mode
: None
dispatch_batches
: None
split_batches
: None
include_tokens_per_second
: False
include_num_input_tokens_seen
: False
neftune_noise_alpha
: None
optim_target_modules
: None
batch_eval_metrics
: False
batch_sampler
: no_duplicates
multi_dataset_batch_sampler
: proportional
Training Logs
Epoch |
Step |
Training Loss |
dim_128_cosine_map@100 |
dim_256_cosine_map@100 |
dim_512_cosine_map@100 |
dim_64_cosine_map@100 |
dim_768_cosine_map@100 |
0.4352 |
1000 |
0.5774 |
- |
- |
- |
- |
- |
0.8703 |
2000 |
0.0914 |
- |
- |
- |
- |
- |
1.0 |
2298 |
- |
0.8254 |
0.8378 |
0.8427 |
0.8159 |
0.8439 |
1.3055 |
3000 |
0.0486 |
- |
- |
- |
- |
- |
1.7406 |
4000 |
0.018 |
- |
- |
- |
- |
- |
2.0 |
4596 |
- |
0.8674 |
0.8731 |
0.8756 |
0.8615 |
0.8781 |
2.1758 |
5000 |
0.0123 |
- |
- |
- |
- |
- |
2.6110 |
6000 |
0.0047 |
- |
- |
- |
- |
- |
3.0 |
6894 |
- |
0.8908 |
0.8940 |
0.8952 |
0.8868 |
0.8959 |
3.0461 |
7000 |
0.0031 |
- |
- |
- |
- |
- |
3.4813 |
8000 |
0.0013 |
- |
- |
- |
- |
- |
3.9164 |
9000 |
0.0012 |
- |
- |
- |
- |
- |
4.0 |
9192 |
- |
0.9049 |
0.9065 |
0.9075 |
0.8999 |
0.9079 |
4.3516 |
10000 |
0.0005 |
- |
- |
- |
- |
- |
4.7868 |
11000 |
0.0002 |
- |
- |
- |
- |
- |
5.0 |
11490 |
- |
0.9086 |
0.9121 |
0.9130 |
0.9059 |
0.9127 |
Framework Versions
- Python: 3.8.10
- Sentence Transformers: 3.0.1
- Transformers: 4.42.0.dev0
- PyTorch: 2.3.0+cu121
- Accelerate: 0.30.1
- Datasets: 2.19.1
- Tokenizers: 0.19.1
Citation
BibTeX
Sentence Transformers
@inproceedings{reimers-2019-sentence-bert,
title = "Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks",
author = "Reimers, Nils and Gurevych, Iryna",
booktitle = "Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing",
month = "11",
year = "2019",
publisher = "Association for Computational Linguistics",
url = "https://arxiv.org/abs/1908.10084",
}
MatryoshkaLoss
@misc{kusupati2024matryoshka,
title={Matryoshka Representation Learning},
author={Aditya Kusupati and Gantavya Bhatt and Aniket Rege and Matthew Wallingford and Aditya Sinha and Vivek Ramanujan and William Howard-Snyder and Kaifeng Chen and Sham Kakade and Prateek Jain and Ali Farhadi},
year={2024},
eprint={2205.13147},
archivePrefix={arXiv},
primaryClass={cs.LG}
}
MultipleNegativesRankingLoss
@misc{henderson2017efficient,
title={Efficient Natural Language Response Suggestion for Smart Reply},
author={Matthew Henderson and Rami Al-Rfou and Brian Strope and Yun-hsuan Sung and Laszlo Lukacs and Ruiqi Guo and Sanjiv Kumar and Balint Miklos and Ray Kurzweil},
year={2017},
eprint={1705.00652},
archivePrefix={arXiv},
primaryClass={cs.CL}
}