SentenceTransformer based on Nerdward/minilm-irembo-ft

This is a sentence-transformers model finetuned from Nerdward/minilm-irembo-ft. It maps sentences & paragraphs to a 384-dimensional dense vector space and can be used for retrieval.

Model Details

Model Description

• Model Type: Sentence Transformer
• Base model: Nerdward/minilm-irembo-ft
• Maximum Sequence Length: 256 tokens
• Output Dimensionality: 384 dimensions
• Similarity Function: Cosine Similarity
• Supported Modality: Text

Model Sources

• Documentation: Sentence Transformers Documentation
• Repository: Sentence Transformers on GitHub
• Hugging Face: Sentence Transformers on Hugging Face

Full Model Architecture

SentenceTransformer(
  (0): Transformer({'transformer_task': 'feature-extraction', 'modality_config': {'text': {'method': 'forward', 'method_output_name': 'last_hidden_state'}}, 'module_output_name': 'token_embeddings', 'architecture': 'BertModel'})
  (1): Pooling({'embedding_dimension': 384, 'pooling_mode': 'mean', 'include_prompt': True})
  (2): Normalize({})
)

Usage

Direct Usage (Sentence Transformers)

First install the Sentence Transformers library:

pip install -U sentence-transformers

Then you can load this model and run inference.

from sentence_transformers import SentenceTransformer

# Download from the 🤗 Hub
model = SentenceTransformer("sentence_transformers_model_id")
# Run inference
queries = [
    'appointed to take documents for ID update',
]
documents = [
    'How to Apply for a National ID Correction. This service enables Rwandans and refugees to apply for the correction of their national identity cards. The citizen will receive an appointment requesting them to visit the selected sector and show supporting documents to justify their reason for the request. The corrected ID card will be collected at the selected sector. The service is provided by the National Identification Agency (NIDA). The processing time for this service is 30 days, and the price depends on the reasons for ID correction. Note: Please follow these essential steps if your reason for ID correction is a photo change before submitting an application on the IremboGov platform. First and foremost, send a photo change request to NIDA on their email (info@nida.gov.rw). Attach a copy of your national ID and most recent passport photo. NIDA will then review your request to change your ID photo. If NIDA approves your request, an appointment date will be set. After successfully completing these steps, proceed to start your application on the IremboGov platform. Prerequisites: Applicants can not apply to this service while logged in to their Irembo account. Applicants should have a Rwandan National ID number. Applicants should have a legitimate reason (date of birth, gender, name, and photo) to request this service. Applicants should have a valid phone number or email address. Follow these simple steps to apply for a national ID correction: Visit Irembo platform: www.irembo.gov.rw and under Identification, click on Application of National ID Correction. Select “Application for National ID correction” from the dropdown menu. Click on Apply. Enter the required Applicant details (ID number, reason for ID correction, place of collection, and processing office), and click Next to proceed. Note: When the reason for ID correction is Change of names, Photo or Damaged the applicant will be charged 1,500 Rwf respectively. Note: It is advised to choose the same processing office as the collection place to make the collection easier. Verify that the information provided is accurate, enter a phone number and email address, check the verification box, and click Submit. A billing number/ID (88….) will be generated. Click pay. Applicants can choose their preferred mode of payment. For more information about payment modes, click here. NOTE: Upon payment, your application will be sent at the sector office of your selection. The sector officer will set an appointment for you to visit their office with proof of correction. Once the application is approved, your documents will be sent to NIDA for a new Id card production and will be ready for pickup in 30 days',
    'How to Apply for a National ID Replacement. This service enables Rwandans and refugees to replace their national ID in case of loss or damage. The applicant may apply for a temporary ID (Certificate of Replacement for National Identification) while waiting for the new ID to be produced. However, this requires an additional fee of Rwf 3000, and it is valid for 60 days only. This service is provided by the National Identification Agency (NIDA). The processing time of this service is 30 days and its price is Rwf 1,500 Prerequisites: Applicants should have an Irembo account to apply for this service. Click here to find out how to create an account on IremboGov or visit the nearest Irembo agent for assistance. Applicants should have a national ID number. Refugees should have a refugee ID number. Follow these simple steps to apply for a National ID Replacement: 1. Visit www.irembo.gov.rw and under Identification, click on National ID Replacement. 2. Click on Apply. 3. Enter the applicant’s ID number. The applicant’s information will automatically appear on the right side of the page. 4. Choose whether or not you want a temporary ID while waiting for the replacement. If yes , the applicant pays an additional fee of Rwf 3000 and selects the processing office of that document. 5. Select the nearest or preferred RIB processing office and place of collection for the new ID. NOTE: It is advisable to choose the same processing office as the place of collection for easy collection. 6. Click Next to proceed. 7. Verify that the information is true, enter a phone number and/or email address, check the verification box, and click Submit. 8. An application number (B2....) will be generated to follow up on the application status. NOTE: After submitting the application, it is sent to the RIB office of your selection for processing. The RIB officer will set an appointment for the applicant to visit their office for application approval. Once the application is approved, the applicant will receive a billing number to pay for the new ID. Applicants can choose to pay using any mode of payments After successfully paying for the service via IremboGov, the application is sent to NIDA for production. For further information, reach out to your selected sector of collection',
    'How to Apply for Meat Transportation Carrier Registration. This is a service requested by an industry to register meat transportation carriers. Industries need the registration certificate to show that they comply with RICA regulatory requirements. Rwanda Inspectorate, Competition, and Consumer Protection Authority (RICA) provides this service. The processing time is 30 days, and the service is free of charge. Prerequisites Applicants with or without an Irembo account can apply for this service. A company or an individual must have a Tax Identification Number (TIN). Applicant should have a valid phone number or email address. Required attachments (5) Authorization for transport of goods by a competent authority Vehicle inspection certificate /control technique Vehicle registration certificate(Carte Jaune) Driver’s Valid medical certificate Business Registration Certificate from RDB. Conditional attachments (2) Calibration certificate Passport copy Follow these simple steps to apply for Meat Transportation Carrier Registration: 1. Visit the Irembo platform at www.irembo.gov.rw and navigate to the "Trade and Industry" category, Click on “Meat transportation carrier registration.” 2. Click "Apply." to start an application. 3. Fill in the applicant\'s details (select if the applicant is the business owner or legal representative, add the age group and other required personal information). 4. Enter business details as prompted. 5. Provide transport details and click on “add new driver” to fill in his information. 6. Upload the required attachments in the correct format. Note: Double-check that the uploaded files are in PDF format and meet the size requirements. 7. Verify that the information is accurate, enter a phone number and/or email address, check the verification box, and click "Submit." 8. An application number (B2…...) to track your application status. Note: Upon successful submission, the application will be processed by RICA. Applicants will receive an email or SMS notification about any changes in the application status. Once the processing officers approve your application, you will receive your e-certificate via email or can download it on IremboGov platform',
]
query_embeddings = model.encode_query(queries)
document_embeddings = model.encode_document(documents)
print(query_embeddings.shape, document_embeddings.shape)
# [1, 384] [3, 384]

# Get the similarity scores for the embeddings
similarities = model.similarity(query_embeddings, document_embeddings)
print(similarities)
# tensor([[ 0.4775,  0.2833, -0.0948]])

Evaluation

Metrics

Information Retrieval

• Dataset: kb-retrieval-eval
• Evaluated with InformationRetrievalEvaluator

Metric	Value
cosine_accuracy@1	0.895
cosine_accuracy@3	0.9834
cosine_accuracy@5	0.9945
cosine_accuracy@10	1.0
cosine_precision@1	0.895
cosine_precision@5	0.2895
cosine_precision@10	0.147
cosine_recall@1	0.6892
cosine_recall@5	0.9834
cosine_recall@10	0.9972
cosine_ndcg@10	0.9505
cosine_mrr@10	0.938
cosine_map@100	0.9321

Training Details

Training Dataset

Unnamed Dataset

• Size: 6,214 training samples
• Columns: anchor, positive, and negative
• Approximate statistics based on the first 100 samples:

  	anchor	positive	negative
  type	string	string	string
  modality	text	text	text
  details	min: 7 tokens mean: 9.62 tokens max: 11 tokens	min: 228 tokens mean: 254.92 tokens max: 256 tokens	min: 15 tokens mean: 224.09 tokens max: 256 tokens

• Samples:

  anchor	positive	negative
  Can I sleep in museum garden tonight?	FAQ's about Camping at the Museum. This service is offered to individuals or groups of people who would like to camp in Museum gardens since museums have big spaces, and tourists can spend their nights in those gardens. The following are the frequently asked questions about camping at the museum; Is camping at the museum available for everyone? Yes, camping at the museum gardens is available for individuals, either Rwandans or foreigners, interested in the experience. What are the prerequisites for applying for camping at the museum? Applicants, whether with or without an account, can apply. Rwandan citizens should have a National ID number, while foreigners need a passport number. What are the modes of camping available at the museum? There are two modes of camping available: Tent: The museum provides tents; you can select this option during your application. The cost of the tent will be included in the service fee. Caravan Car: This option allows you to use your personal camping vehi...	How to schedule a museum Visit. This service allows a person or group of people to schedule a visit to any of the National Museums of Rwanda. The processing time is one day. The price of this service depends on the following reasons: Entrance, admission, and guiding fees for nationals, international non-residents, and EAC/CEPGL. Prerequisites: Applicants with or without an account can apply for this service Rwandan citizens should have a national ID number. Foreigners should have a passport number. Applicants should have a valid phone number and/or email address. Follow these simple steps to schedule a visit to a Rwandan museum Visit www.irembo.gov.rw, and under Rwanda Museum, click on Schedule Visit. Click Apply Choose the facility to visit and the visit date, and fill in the applicant details by clicking in the blank space to enter the required details; the details will be retrieved and displayed on the right side of the page. Click on Next to proceed. Verify that the information is ...
  Can I sleep in museum garden tonight?	FAQ's about Camping at the Museum. This service is offered to individuals or groups of people who would like to camp in Museum gardens since museums have big spaces, and tourists can spend their nights in those gardens. The following are the frequently asked questions about camping at the museum; Is camping at the museum available for everyone? Yes, camping at the museum gardens is available for individuals, either Rwandans or foreigners, interested in the experience. What are the prerequisites for applying for camping at the museum? Applicants, whether with or without an account, can apply. Rwandan citizens should have a National ID number, while foreigners need a passport number. What are the modes of camping available at the museum? There are two modes of camping available: Tent: The museum provides tents; you can select this option during your application. The cost of the tent will be included in the service fee. Caravan Car: This option allows you to use your personal camping vehi...	How to Reschedule a museum Visit. This Service allows a person or a group of people to reschedule a visit to any of the National Museums of Rwanda. The processing time is 1 day; This service is free of charge. Prerequisites: Application number of the scheduled visit. Applicants should have a valid phone number and/or email address. Follow these simple steps to schedule a visit to a Rwandan museum Visit www.irembo.gov.rw and under Rwanda museum, click on Schedule Visit. Click Apply Enter the Application Number of the visit to be rescheduled and the new Visit Date, and fill in the Applicant Details by clicking in the blank space to enter the required details; the details will be retrieved and displayed on the right side of the page. Click on Next to proceed Verify that the information is true, enter your Phone number and/or Email Address, check the verification box and Click on Submit. An application number will be generated. NOTE: After successfully applying via IremboGov, the applicant...
  Can I sleep in museum garden tonight?	FAQ's about Camping at the Museum. This service is offered to individuals or groups of people who would like to camp in Museum gardens since museums have big spaces, and tourists can spend their nights in those gardens. The following are the frequently asked questions about camping at the museum; Is camping at the museum available for everyone? Yes, camping at the museum gardens is available for individuals, either Rwandans or foreigners, interested in the experience. What are the prerequisites for applying for camping at the museum? Applicants, whether with or without an account, can apply. Rwandan citizens should have a National ID number, while foreigners need a passport number. What are the modes of camping available at the museum? There are two modes of camping available: Tent: The museum provides tents; you can select this option during your application. The cost of the tent will be included in the service fee. Caravan Car: This option allows you to use your personal camping vehi...	Museum Visit. Click here to learn how to schedule a Museum visit

• Loss: MultipleNegativesRankingLoss with these parameters:

  {
      "scale": 20.0,
      "similarity_fct": "cos_sim",
      "gather_across_devices": false,
      "directions": [
          "query_to_doc"
      ],
      "partition_mode": "joint",
      "hardness_mode": null,
      "hardness_strength": 0.0
  }
  

Training Hyperparameters

Non-Default Hyperparameters

• per_device_train_batch_size: 128
• num_train_epochs: 6.0
• learning_rate: 2e-05
• warmup_steps: 0.1
• load_best_model_at_end: True
• batch_sampler: no_duplicates

All Hyperparameters

Click to expand

• per_device_train_batch_size: 128
• num_train_epochs: 6.0
• max_steps: -1
• learning_rate: 2e-05
• lr_scheduler_type: linear
• lr_scheduler_kwargs: None
• warmup_steps: 0.1
• optim: adamw_torch_fused
• optim_args: None
• weight_decay: 0.0
• adam_beta1: 0.9
• adam_beta2: 0.999
• adam_epsilon: 1e-08
• optim_target_modules: None
• gradient_accumulation_steps: 1
• average_tokens_across_devices: True
• max_grad_norm: 1.0
• label_smoothing_factor: 0.0
• bf16: False
• fp16: False
• bf16_full_eval: False
• fp16_full_eval: False
• tf32: None
• gradient_checkpointing: False
• gradient_checkpointing_kwargs: None
• torch_compile: False
• torch_compile_backend: None
• torch_compile_mode: None
• use_liger_kernel: False
• liger_kernel_config: None
• use_cache: False
• neftune_noise_alpha: None
• torch_empty_cache_steps: None
• auto_find_batch_size: False
• log_on_each_node: True
• logging_nan_inf_filter: True
• include_num_input_tokens_seen: no
• log_level: passive
• log_level_replica: warning
• disable_tqdm: False
• project: huggingface
• trackio_space_id: None
• trackio_bucket_id: None
• trackio_static_space_id: None
• per_device_eval_batch_size: 8
• prediction_loss_only: True
• eval_on_start: False
• eval_do_concat_batches: True
• eval_use_gather_object: False
• eval_accumulation_steps: None
• include_for_metrics: []
• batch_eval_metrics: False
• save_only_model: False
• save_on_each_node: False
• enable_jit_checkpoint: False
• push_to_hub: False
• hub_private_repo: None
• hub_model_id: None
• hub_strategy: every_save
• hub_always_push: False
• hub_revision: None
• load_best_model_at_end: True
• ignore_data_skip: False
• restore_callback_states_from_checkpoint: False
• full_determinism: False
• seed: 42
• data_seed: None
• use_cpu: False
• accelerator_config: {'split_batches': False, 'dispatch_batches': None, 'even_batches': True, 'use_seedable_sampler': True, 'non_blocking': False, 'gradient_accumulation_kwargs': None}
• parallelism_config: None
• dataloader_drop_last: False
• dataloader_num_workers: 0
• dataloader_pin_memory: True
• dataloader_persistent_workers: False
• dataloader_prefetch_factor: None
• remove_unused_columns: True
• label_names: None
• train_sampling_strategy: random
• length_column_name: length
• ddp_find_unused_parameters: None
• ddp_bucket_cap_mb: None
• ddp_broadcast_buffers: False
• ddp_static_graph: None
• ddp_backend: None
• ddp_timeout: 1800
• fsdp: None
• fsdp_config: None
• deepspeed: None
• debug: []
• skip_memory_metrics: True
• do_predict: False
• resume_from_checkpoint: None
• warmup_ratio: None
• local_rank: -1
• prompts: None
• batch_sampler: no_duplicates
• multi_dataset_batch_sampler: proportional
• router_mapping: {}
• learning_rate_mapping: {}

Training Logs

Epoch	Step	Training Loss	kb-retrieval-eval_cosine_ndcg@10
-1	-1	-	0.9418
0.4082	20	0.8003	-
0.8163	40	0.8074	-
1.0204	50	-	0.9505
1.2245	60	0.6564	-
1.6327	80	0.6818	-
2.0408	100	0.6351	0.9470
2.4490	120	0.5950	-
2.8571	140	0.5772	-
3.0612	150	-	0.9397
3.2653	160	0.5353	-
3.6735	180	0.5523	-
4.0816	200	0.5408	0.9379
-1	-1	-	0.9505

• The bold row denotes the saved checkpoint.

Training Time

• Training: 7.8 minutes
• Evaluation: 4.7 seconds
• Total: 7.9 minutes

Framework Versions

• Python: 3.12.13
• Sentence Transformers: 5.6.0
• Transformers: 5.13.1
• PyTorch: 2.11.0+cu128
• Accelerate: 1.14.0
• Datasets: 4.0.0
• Tokenizers: 0.22.2

Citation

BibTeX

Sentence Transformers

@inproceedings{reimers-2019-sentence-bert,
    title = "Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks",
    author = "Reimers, Nils and Gurevych, Iryna",
    booktitle = "Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing",
    month = "11",
    year = "2019",
    publisher = "Association for Computational Linguistics",
    url = "https://arxiv.org/abs/1908.10084",
}

MultipleNegativesRankingLoss

@misc{oord2019representationlearningcontrastivepredictive,
      title={Representation Learning with Contrastive Predictive Coding},
      author={Aaron van den Oord and Yazhe Li and Oriol Vinyals},
      year={2019},
      eprint={1807.03748},
      archivePrefix={arXiv},
      primaryClass={cs.LG},
      url={https://arxiv.org/abs/1807.03748},
}