SentenceTransformer based on denaya/indoSBERT-large
This is a sentence-transformers model finetuned from denaya/indoSBERT-large on the statictable-triplets-all dataset. It maps sentences & paragraphs to a 256-dimensional dense vector space and can be used for semantic textual similarity, semantic search, paraphrase mining, text classification, clustering, and more.
Model Details
Model Description
- Model Type: Sentence Transformer
- Base model: denaya/indoSBERT-large
- Maximum Sequence Length: 256 tokens
- Output Dimensionality: 256 dimensions
- Similarity Function: Cosine Similarity
- Training Dataset:
Model Sources
- Documentation: Sentence Transformers Documentation
- Repository: Sentence Transformers on GitHub
- Hugging Face: Sentence Transformers on Hugging Face
Full Model Architecture
SentenceTransformer(
(0): Transformer({'max_seq_length': 256, 'do_lower_case': False}) with Transformer model: BertModel
(1): Pooling({'word_embedding_dimension': 1024, 'pooling_mode_cls_token': False, 'pooling_mode_mean_tokens': True, 'pooling_mode_max_tokens': False, 'pooling_mode_mean_sqrt_len_tokens': False, 'pooling_mode_weightedmean_tokens': False, 'pooling_mode_lasttoken': False, 'include_prompt': True})
(2): Dense({'in_features': 1024, 'out_features': 256, 'bias': True, 'activation_function': 'torch.nn.modules.activation.Tanh'})
)
Usage
Direct Usage (Sentence Transformers)
First install the Sentence Transformers library:
pip install -U sentence-transformers
Then you can load this model and run inference.
from sentence_transformers import SentenceTransformer
# Download from the 🤗 Hub
model = SentenceTransformer("yahyaabd/indoSBERT-Large-mnrl-2")
# Run inference
sentences = [
'Upah bulanan rata-rata: Hubungan pendidikan tertinggi dan sektor pekerjaan utama, data 2021',
'Rata-rata Upah/Gaji Bersih sebulan Buruh/Karyawan Pegawai Menurut Pendidikan Tertinggi dan Lapangan Pekerjaan Utama, 2021',
'Rata-rata Upah/Gaji Bersih Sebulan Buruh/Karyawan Pegawai Menurut Pendidikan Tertinggi dan Jenis Pekerjaan Utama, 2021',
]
embeddings = model.encode(sentences)
print(embeddings.shape)
# [3, 256]
# Get the similarity scores for the embeddings
similarities = model.similarity(embeddings, embeddings)
print(similarities.shape)
# [3, 3]
Evaluation
Metrics
Information Retrieval
- Dataset:
bps-statictable-ir
- Evaluated with
InformationRetrievalEvaluator
Metric | Value |
---|---|
cosine_accuracy@1 | 0.9218 |
cosine_accuracy@5 | 0.9902 |
cosine_accuracy@10 | 0.9967 |
cosine_precision@1 | 0.9218 |
cosine_precision@5 | 0.2248 |
cosine_precision@10 | 0.1316 |
cosine_recall@1 | 0.7225 |
cosine_recall@5 | 0.793 |
cosine_recall@10 | 0.8182 |
cosine_ndcg@1 | 0.9218 |
cosine_ndcg@5 | 0.8341 |
cosine_ndcg@10 | 0.8332 |
cosine_mrr@1 | 0.9218 |
cosine_mrr@5 | 0.9522 |
cosine_mrr@10 | 0.9532 |
cosine_map@1 | 0.9218 |
cosine_map@5 | 0.792 |
cosine_map@10 | 0.7848 |
Training Details
Training Dataset
statictable-triplets-all
- Dataset: statictable-triplets-all at 24979b4
- Size: 967,831 training samples
- Columns:
query
,pos
, andneg
- Approximate statistics based on the first 1000 samples:
query pos neg type string string string details - min: 3 tokens
- mean: 16.97 tokens
- max: 30 tokens
- min: 3 tokens
- mean: 20.79 tokens
- max: 48 tokens
- min: 4 tokens
- mean: 20.94 tokens
- max: 48 tokens
- Samples:
query pos neg Data input-output antar daerah, 34 provinsi: Transaksi domestik (52 industri, harga produsen, 2016)
Tabel Inter Regional Input-Output Indonesia Transaksi Domestik Atas Dasar Harga Produsen Menurut 34 Provinsi dan 52 Industri, 2016 (Juta Rupiah)
Penduduk Berumur 15 Tahun Ke Atas yang Bekerja Selama Seminggu yang Lalu Menurut Golongan Umur dan Jumlah Jam Kerja Seluruhnya, 2008 - 2024
Data total penghasilan berbagai golongan rumah tangga setelah dipotong pajak, tahun 2000 (dalam miliar rupiah)
Jumlah Pendapatan Setelah Pajak Menurut Golongan Rumah Tangga (miliar rupiah), 2000, 2005, dan 2008
Institusi Korporasi Non Finansial Neraca Institusi Terintegrasi ( triliun rupiah), 2016 2022
Rumah tangga dengan area resapan, data per provinsi, 2014
Persentase Rumah Tangga Menurut Provinsi dan Keberadaan Area Resapan Air, 2013-2014
Nilai Produksi dan Biaya Produksi per Musim Tanam per Hektar Budidaya Tanaman Padi Sawah, Padi Ladang, Jagung, dan Kedelai, 2017
- Loss:
MultipleNegativesRankingLoss
with these parameters:{ "scale": 20.0, "similarity_fct": "cos_sim" }
Evaluation Dataset
statictable-triplets-all
- Dataset: statictable-triplets-all at 24979b4
- Size: 967,831 evaluation samples
- Columns:
query
,pos
, andneg
- Approximate statistics based on the first 1000 samples:
query pos neg type string string string details - min: 3 tokens
- mean: 16.69 tokens
- max: 32 tokens
- min: 3 tokens
- mean: 20.85 tokens
- max: 48 tokens
- min: 3 tokens
- mean: 20.9 tokens
- max: 48 tokens
- Samples:
query pos neg Kredit UMKM bank umum (miliar rupiah), 2012-2016
Posisi Kredit Usaha Mikro, Kecil, dan Menengah (UMKM) 1 pada Bank Umum (miliar rupiah), 2012-2016
Jumlah Penghuni Lapas per Kanwil
Infant Mortality Rate di Indonesia per provinsi, 1971
Angka Kematian Bayi/AKB (Infant Mortality Rate/IMR) Menurut Provinsi, 1971-2020
Jumlah Sekolah, Guru, dan Murid Sekolah Menengah Kejuruan (SMK) di Bawah Kementrian Pendidikan dan Kebudayaan Menurut Provinsi tahun ajaran 2011/2012-2015/2016
Partisipasi sekolah anak dan remaja: Data persentase usia 7-24 tahun per gender dan kelompok umur, 2021
Persentase Penduduk Usia 7-24 Tahun Menurut Jenis Kelamin, Kelompok Umur, dan Partisipasi Sekolah, 2002-2023
Tabel Input-Output Indonesia Transaksi Total Atas Dasar Harga Pembeli (17 Produk), 2016 (Juta Rupiah)
- Loss:
MultipleNegativesRankingLoss
with these parameters:{ "scale": 20.0, "similarity_fct": "cos_sim" }
Training Hyperparameters
Non-Default Hyperparameters
eval_strategy
: stepsper_device_train_batch_size
: 64per_device_eval_batch_size
: 64num_train_epochs
: 1warmup_ratio
: 0.1fp16
: Trueload_best_model_at_end
: Trueeval_on_start
: Truebatch_sampler
: no_duplicates
All Hyperparameters
Click to expand
overwrite_output_dir
: Falsedo_predict
: Falseeval_strategy
: stepsprediction_loss_only
: Trueper_device_train_batch_size
: 64per_device_eval_batch_size
: 64per_gpu_train_batch_size
: Noneper_gpu_eval_batch_size
: Nonegradient_accumulation_steps
: 1eval_accumulation_steps
: Nonetorch_empty_cache_steps
: Nonelearning_rate
: 5e-05weight_decay
: 0.0adam_beta1
: 0.9adam_beta2
: 0.999adam_epsilon
: 1e-08max_grad_norm
: 1.0num_train_epochs
: 1max_steps
: -1lr_scheduler_type
: linearlr_scheduler_kwargs
: {}warmup_ratio
: 0.1warmup_steps
: 0log_level
: passivelog_level_replica
: warninglog_on_each_node
: Truelogging_nan_inf_filter
: Truesave_safetensors
: Truesave_on_each_node
: Falsesave_only_model
: Falserestore_callback_states_from_checkpoint
: Falseno_cuda
: Falseuse_cpu
: Falseuse_mps_device
: Falseseed
: 42data_seed
: Nonejit_mode_eval
: Falseuse_ipex
: Falsebf16
: Falsefp16
: Truefp16_opt_level
: O1half_precision_backend
: autobf16_full_eval
: Falsefp16_full_eval
: Falsetf32
: Nonelocal_rank
: 0ddp_backend
: Nonetpu_num_cores
: Nonetpu_metrics_debug
: Falsedebug
: []dataloader_drop_last
: Falsedataloader_num_workers
: 0dataloader_prefetch_factor
: Nonepast_index
: -1disable_tqdm
: Falseremove_unused_columns
: Truelabel_names
: Noneload_best_model_at_end
: Trueignore_data_skip
: Falsefsdp
: []fsdp_min_num_params
: 0fsdp_config
: {'min_num_params': 0, 'xla': False, 'xla_fsdp_v2': False, 'xla_fsdp_grad_ckpt': False}fsdp_transformer_layer_cls_to_wrap
: Noneaccelerator_config
: {'split_batches': False, 'dispatch_batches': None, 'even_batches': True, 'use_seedable_sampler': True, 'non_blocking': False, 'gradient_accumulation_kwargs': None}deepspeed
: Nonelabel_smoothing_factor
: 0.0optim
: adamw_torchoptim_args
: Noneadafactor
: Falsegroup_by_length
: Falselength_column_name
: lengthddp_find_unused_parameters
: Noneddp_bucket_cap_mb
: Noneddp_broadcast_buffers
: Falsedataloader_pin_memory
: Truedataloader_persistent_workers
: Falseskip_memory_metrics
: Trueuse_legacy_prediction_loop
: Falsepush_to_hub
: Falseresume_from_checkpoint
: Nonehub_model_id
: Nonehub_strategy
: every_savehub_private_repo
: Nonehub_always_push
: Falsegradient_checkpointing
: Falsegradient_checkpointing_kwargs
: Noneinclude_inputs_for_metrics
: Falseinclude_for_metrics
: []eval_do_concat_batches
: Truefp16_backend
: autopush_to_hub_model_id
: Nonepush_to_hub_organization
: Nonemp_parameters
:auto_find_batch_size
: Falsefull_determinism
: Falsetorchdynamo
: Noneray_scope
: lastddp_timeout
: 1800torch_compile
: Falsetorch_compile_backend
: Nonetorch_compile_mode
: Nonedispatch_batches
: Nonesplit_batches
: Noneinclude_tokens_per_second
: Falseinclude_num_input_tokens_seen
: Falseneftune_noise_alpha
: Noneoptim_target_modules
: Nonebatch_eval_metrics
: Falseeval_on_start
: Trueuse_liger_kernel
: Falseeval_use_gather_object
: Falseaverage_tokens_across_devices
: Falseprompts
: Nonebatch_sampler
: no_duplicatesmulti_dataset_batch_sampler
: proportional
Training Logs
Epoch | Step | Training Loss | Validation Loss | bps-statictable-ir_cosine_ndcg@10 |
---|---|---|---|---|
0 | 0 | - | 0.7678 | 0.7378 |
0.1391 | 100 | 0.2164 | 0.0292 | 0.8324 |
0.2782 | 200 | 0.032 | 0.0143 | 0.8383 |
0.4172 | 300 | 0.0221 | 0.0077 | 0.8392 |
0.5563 | 400 | 0.0088 | 0.0055 | 0.8391 |
0.6954 | 500 | 0.0058 | 0.0033 | 0.8301 |
0.8345 | 600 | 0.0039 | 0.0016 | 0.8331 |
0.9736 | 700 | 0.0027 | 0.0019 | 0.8332 |
- The bold row denotes the saved checkpoint.
Framework Versions
- Python: 3.11.11
- Sentence Transformers: 3.4.1
- Transformers: 4.48.3
- PyTorch: 2.6.0+cu124
- Accelerate: 1.3.0
- Datasets: 3.4.1
- Tokenizers: 0.21.1
Citation
BibTeX
Sentence Transformers
@inproceedings{reimers-2019-sentence-bert,
title = "Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks",
author = "Reimers, Nils and Gurevych, Iryna",
booktitle = "Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing",
month = "11",
year = "2019",
publisher = "Association for Computational Linguistics",
url = "https://arxiv.org/abs/1908.10084",
}
MultipleNegativesRankingLoss
@misc{henderson2017efficient,
title={Efficient Natural Language Response Suggestion for Smart Reply},
author={Matthew Henderson and Rami Al-Rfou and Brian Strope and Yun-hsuan Sung and Laszlo Lukacs and Ruiqi Guo and Sanjiv Kumar and Balint Miklos and Ray Kurzweil},
year={2017},
eprint={1705.00652},
archivePrefix={arXiv},
primaryClass={cs.CL}
}
- Downloads last month
- 2
Inference Providers
NEW
This model is not currently available via any of the supported Inference Providers.
Model tree for yahyaabd/indoSBERT-Large-mnrl-2
Base model
denaya/indoSBERT-largeDataset used to train yahyaabd/indoSBERT-Large-mnrl-2
Evaluation results
- Cosine Accuracy@1 on bps statictable irself-reported0.922
- Cosine Accuracy@5 on bps statictable irself-reported0.990
- Cosine Accuracy@10 on bps statictable irself-reported0.997
- Cosine Precision@1 on bps statictable irself-reported0.922
- Cosine Precision@5 on bps statictable irself-reported0.225
- Cosine Precision@10 on bps statictable irself-reported0.132
- Cosine Recall@1 on bps statictable irself-reported0.723
- Cosine Recall@5 on bps statictable irself-reported0.793
- Cosine Recall@10 on bps statictable irself-reported0.818
- Cosine Ndcg@1 on bps statictable irself-reported0.922