ChenyuEcho/corruption_emaillevel_LoRA_newtrainmethod

SentenceTransformer based on Qwen/Qwen3-Embedding-0.6B

This is a sentence-transformers model finetuned from Qwen/Qwen3-Embedding-0.6B. It maps sentences & paragraphs to a 1024-dimensional dense vector space and can be used for semantic textual similarity, semantic search, paraphrase mining, text classification, clustering, and more.

Model Details

Model Description

• Model Type: Sentence Transformer
• Base model: Qwen/Qwen3-Embedding-0.6B
• Maximum Sequence Length: 32768 tokens
• Output Dimensionality: 1024 dimensions
• Similarity Function: Cosine Similarity

Model Sources

• Documentation: Sentence Transformers Documentation
• Repository: Sentence Transformers on GitHub
• Hugging Face: Sentence Transformers on Hugging Face

Full Model Architecture

SentenceTransformer(
  (0): Transformer({'max_seq_length': 32768, 'do_lower_case': False, 'architecture': 'Qwen3Model'})
  (1): Pooling({'word_embedding_dimension': 1024, 'pooling_mode_cls_token': False, 'pooling_mode_mean_tokens': False, 'pooling_mode_max_tokens': False, 'pooling_mode_mean_sqrt_len_tokens': False, 'pooling_mode_weightedmean_tokens': False, 'pooling_mode_lasttoken': True, 'include_prompt': True})
  (2): Normalize()
)

Usage

Direct Usage (Sentence Transformers)

First install the Sentence Transformers library:

pip install -U sentence-transformers

Then you can load this model and run inference.

from sentence_transformers import SentenceTransformer

# Download from the 🤗 Hub
model = SentenceTransformer("sentence_transformers_model_id")
# Run inference
queries = [
    "Confirm Massey-Ferguson harvester availability for Block 3A harvest scheduled for June 18.",
]
documents = [
    'Subject: Agave Harvest Scheduling and Resource Coordination\nDate: 2026-01-05T10:03:00\nFrom: Javier Moreno\nParticipants: Sofia Hernandez\n\nBody:\nHi Sofia,\n\nI wanted to touch base regarding the upcoming agave harvest scheduling for the El Molino and San Pedro fields. Based on current field conditions and the lab’s recent Brix readings (average 26.5), I propose we start with Block 3A on June 18, aiming for 120 tons over three days. Please ensure the Massey-Ferguson harvester is available, and that the standard sanitation protocol for incoming loads is enforced. As always, maintaining optimal ripeness and minimizing core bruising are essential for product quality. Could you confirm equipment availability and crew scheduling?\n\nThanks for your attention to these details. Let me know if you have any concerns or require adjustments.\n\nBest,\nJavier\n\n--\nJavier Moreno\nQuality Control Manager\nDestilería Agave Spirits',
    'Subject: Agave Supplier Delivery Schedule – Harvest Operations Planning\nDate: 2025-08-18T18:12:00\nFrom: Patricia Reeves\nParticipants: Thomas Bradford; Sarah Mitchell\n\nBody:\nDear Team,\n\nI wanted to provide an update regarding our agave sourcing and the delivery schedule for the upcoming harvest season. We have coordinated with our primary suppliers to ensure that initial deliveries will commence the week of July 10th, with subsequent shipments following a bi-weekly cadence. We are closely monitoring crop yields and weather conditions to proactively address any potential delays. Please review the attached delivery timeline and confirm receipt so we can coordinate logistics accordingly. Your timely collaboration will be crucial for maintaining smooth harvest operations and meeting production targets.\n\nBest regards,\nPatricia Reeves\n\n--\nPatricia Reeves\nExecutive Assistant to the CEO\nAgave Spirits International',
    'Subject: Celebrating Our Team Excellence Award Recipient—¡Felicidades, Elena!\nDate: 2025-12-11T19:24:00\nFrom: Carlos Delgado\nParticipants: Agave Spirits Mexico Team\n\nBody:\nDear Team,\n\nI am delighted to announce that this quarter’s Team Excellence Award goes to our very own Elena Fuentes. Elena has shown unparalleled dedication in supporting our operations, always ensuring that even the smallest details are handled with cariño. Her ability to coordinate complex schedules and build confianza with our partners reflects the essence of how we do business here: en México, las relaciones importan, and Elena embodies this every day. Her work reminds us that true excellence comes from caring for each other, not just tasks.\n\nPlease join me in congratulating Elena. We will be honoring her achievement with just a small dinner—nada ostentoso, just a way to come together as equipo and celebrate the relationships that drive our success.\n\nUn abrazo fuerte a todos,\nCarlos\n\n--\nCarlos Delgado\nCountry Manager, Mexico Operations\nAgave Spirits International\nTequila, Jalisco',
]
query_embeddings = model.encode_query(queries)
document_embeddings = model.encode_document(documents)
print(query_embeddings.shape, document_embeddings.shape)
# [1, 1024] [3, 1024]

# Get the similarity scores for the embeddings
similarities = model.similarity(query_embeddings, document_embeddings)
print(similarities)
# tensor([[0.6602, 0.2324, 0.0050]], dtype=torch.bfloat16)

Evaluation

Metrics

Information Retrieval

• Dataset: val_full_corpus
• Evaluated with InformationRetrievalEvaluator

Metric	Value
cosine_accuracy@1	0.8878
cosine_accuracy@3	0.9561
cosine_accuracy@5	0.9805
cosine_accuracy@10	0.9902
cosine_precision@1	0.8878
cosine_precision@3	0.3187
cosine_precision@5	0.1961
cosine_precision@10	0.099
cosine_recall@1	0.8878
cosine_recall@3	0.9561
cosine_recall@5	0.9805
cosine_recall@10	0.9902
cosine_ndcg@10	0.9419
cosine_mrr@10	0.9259
cosine_map@100	0.9265

Training Details

Training Dataset

Unnamed Dataset

• Size: 816 training samples
• Columns: sentence_0 and sentence_1
• Approximate statistics based on the first 816 samples:

  	sentence_0	sentence_1
  type	string	string
  details	min: 6 tokens mean: 32.3 tokens max: 98 tokens	min: 118 tokens mean: 208.52 tokens max: 419 tokens

• Samples:

  sentence_0	sentence_1
  Find records of quality test results for batch QT-2024-0891 showing alcohol content 39.8%, pH 3.6, and approval by Carlos.	Subject: Batch QT-2024-0891 Quality Test Results Date: 2025-10-15T12:26:00 From: Ana Lucia Vega Participants: Javier Moreno Body: Hi Javier, I wanted to share the results from the routine quality testing on batch QT-2024-0891. The alcohol content was measured at 39.8%, which is within our expected range. pH levels were 3.6, also within normal parameters. Taste panel notes described the flavor profile as clean and balanced, no anomalies reported. Carlos reviewed and approved these results before I sent this email. Please let me know if you need any more details or if you want the full lab report attached. Thanks! Regards, Ana Lucia Vega
  What is the decision on replacing the battery for forklift unit 4: replace now or wait?	Subject: Forklift Fleet Routine Maintenance Completed – Service Report Date: 2026-01-19T18:34:00 From: Diego Ramirez Participants: Pedro Villanueva Body: Hi Pedro, Just wanted to let you know we wrapped up routine maintenance on the forklift fleet this afternoon. We replaced the hydraulic filters on units 3 and 5, and topped up fluids on all machines. Belts on forklift 2 were showing a lot of wear, so we swapped those out—I'll need to order a few more spares from Guadalajara soon. Everything else checked out, but the battery on unit 4 is starting to lose charge faster than it should—might not last till next service, so let me know if you want to replace it now or wait. Next scheduled maintenance is set for July 21st. Let me know if you have any other issues you want the guys to check on. Regards, Diego
  Is there documentation of permit delays impacting production, including any contingency plan to shut down if the permit is not obtained?	Subject: Downtime Analysis – Tequila Distillery Production Update Date: 2025-10-13T10:55:00 From: Roberto Garza Participants: Thomas Bradford Body: Hi Tom, I wanted to share our latest downtime analysis for the month. We experienced a total of 23 hours of unplanned downtime, primarily due to maintenance on the primary fermentation tanks and a hold in bottling while awaiting inspection. Output for the period was 212,000 liters, down approximately 8% from last month. I’m concerned the ongoing permit delays could impact our next production cycle; if we don't get the permit, we shut down. I trust Rick is moving things forward on the environmental side. Let me know if there’s anything else you need from my team. Best, Roberto -- Roberto Garza Plant Manager Destilería Agave Spirits - Tequila

• Loss: MultipleNegativesRankingLoss with these parameters:

  {
      "scale": 20.0,
      "similarity_fct": "cos_sim",
      "gather_across_devices": false
  }
  

Training Hyperparameters

Non-Default Hyperparameters

• per_device_train_batch_size: 16
• per_device_eval_batch_size: 16
• multi_dataset_batch_sampler: round_robin

All Hyperparameters

Click to expand

• do_predict: False
• eval_strategy: no
• prediction_loss_only: True
• per_device_train_batch_size: 16
• per_device_eval_batch_size: 16
• gradient_accumulation_steps: 1
• eval_accumulation_steps: None
• torch_empty_cache_steps: None
• learning_rate: 5e-05
• weight_decay: 0.0
• adam_beta1: 0.9
• adam_beta2: 0.999
• adam_epsilon: 1e-08
• max_grad_norm: 1
• num_train_epochs: 3
• max_steps: -1
• lr_scheduler_type: linear
• lr_scheduler_kwargs: None
• warmup_ratio: None
• warmup_steps: 0
• log_level: passive
• log_level_replica: warning
• log_on_each_node: True
• logging_nan_inf_filter: True
• enable_jit_checkpoint: False
• save_on_each_node: False
• save_only_model: False
• restore_callback_states_from_checkpoint: False
• use_cpu: False
• seed: 42
• data_seed: None
• bf16: False
• fp16: False
• bf16_full_eval: False
• fp16_full_eval: False
• tf32: None
• local_rank: -1
• ddp_backend: None
• debug: []
• dataloader_drop_last: False
• dataloader_num_workers: 0
• dataloader_prefetch_factor: None
• disable_tqdm: False
• remove_unused_columns: True
• label_names: None
• load_best_model_at_end: False
• ignore_data_skip: False
• fsdp: []
• fsdp_config: {'min_num_params': 0, 'xla': False, 'xla_fsdp_v2': False, 'xla_fsdp_grad_ckpt': False}
• accelerator_config: {'split_batches': False, 'dispatch_batches': None, 'even_batches': True, 'use_seedable_sampler': True, 'non_blocking': False, 'gradient_accumulation_kwargs': None}
• parallelism_config: None
• deepspeed: None
• label_smoothing_factor: 0.0
• optim: adamw_torch_fused
• optim_args: None
• group_by_length: False
• length_column_name: length
• project: huggingface
• trackio_space_id: trackio
• ddp_find_unused_parameters: None
• ddp_bucket_cap_mb: None
• ddp_broadcast_buffers: False
• dataloader_pin_memory: True
• dataloader_persistent_workers: False
• skip_memory_metrics: True
• push_to_hub: False
• resume_from_checkpoint: None
• hub_model_id: None
• hub_strategy: every_save
• hub_private_repo: None
• hub_always_push: False
• hub_revision: None
• gradient_checkpointing: False
• gradient_checkpointing_kwargs: None
• include_for_metrics: []
• eval_do_concat_batches: True
• auto_find_batch_size: False
• full_determinism: False
• ddp_timeout: 1800
• torch_compile: False
• torch_compile_backend: None
• torch_compile_mode: None
• include_num_input_tokens_seen: no
• neftune_noise_alpha: None
• optim_target_modules: None
• batch_eval_metrics: False
• eval_on_start: False
• use_liger_kernel: False
• liger_kernel_config: None
• eval_use_gather_object: False
• average_tokens_across_devices: True
• use_cache: False
• prompts: None
• batch_sampler: batch_sampler
• multi_dataset_batch_sampler: round_robin
• router_mapping: {}
• learning_rate_mapping: {}

Training Logs

Epoch	Step	val_full_corpus_cosine_ndcg@10
1.0	51	0.9334
2.0	102	0.9344
3.0	153	0.9419

Framework Versions

• Python: 3.12.12
• Sentence Transformers: 5.2.3
• Transformers: 5.0.0
• PyTorch: 2.10.0+cu128
• Accelerate: 1.12.0
• Datasets: 4.0.0
• Tokenizers: 0.22.2

Citation

BibTeX

Sentence Transformers

@inproceedings{reimers-2019-sentence-bert,
    title = "Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks",
    author = "Reimers, Nils and Gurevych, Iryna",
    booktitle = "Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing",
    month = "11",
    year = "2019",
    publisher = "Association for Computational Linguistics",
    url = "https://arxiv.org/abs/1908.10084",
}

MultipleNegativesRankingLoss

@misc{henderson2017efficient,
    title={Efficient Natural Language Response Suggestion for Smart Reply},
    author={Matthew Henderson and Rami Al-Rfou and Brian Strope and Yun-hsuan Sung and Laszlo Lukacs and Ruiqi Guo and Sanjiv Kumar and Balint Miklos and Ray Kurzweil},
    year={2017},
    eprint={1705.00652},
    archivePrefix={arXiv},
    primaryClass={cs.CL}
}