ChenyuEcho/corruption_emaillevel_DPO

SentenceTransformer based on Qwen/Qwen3-Embedding-0.6B

This is a sentence-transformers model finetuned from Qwen/Qwen3-Embedding-0.6B. It maps sentences & paragraphs to a 1024-dimensional dense vector space and can be used for semantic textual similarity, semantic search, paraphrase mining, text classification, clustering, and more.

Model Details

Model Description

• Model Type: Sentence Transformer
• Base model: Qwen/Qwen3-Embedding-0.6B
• Maximum Sequence Length: 32768 tokens
• Output Dimensionality: 1024 dimensions
• Similarity Function: Cosine Similarity

Model Sources

• Documentation: Sentence Transformers Documentation
• Repository: Sentence Transformers on GitHub
• Hugging Face: Sentence Transformers on Hugging Face

Full Model Architecture

SentenceTransformer(
  (0): Transformer({'max_seq_length': 32768, 'do_lower_case': False, 'architecture': 'Qwen3Model'})
  (1): Pooling({'word_embedding_dimension': 1024, 'pooling_mode_cls_token': False, 'pooling_mode_mean_tokens': False, 'pooling_mode_max_tokens': False, 'pooling_mode_mean_sqrt_len_tokens': False, 'pooling_mode_weightedmean_tokens': False, 'pooling_mode_lasttoken': True, 'include_prompt': True})
  (2): Normalize()
)

Usage

Direct Usage (Sentence Transformers)

First install the Sentence Transformers library:

pip install -U sentence-transformers

Then you can load this model and run inference.

from sentence_transformers import SentenceTransformer

# Download from the 🤗 Hub
model = SentenceTransformer("sentence_transformers_model_id")
# Run inference
queries = [
    "Which employees discussed circumventing compliance protocols to secure government approvals?",
]
documents = [
    'Subject: PRIVILEGED AND CONFIDENTIAL - ATTORNEY-CLIENT COMMUNICATION - Mexico Operations Concern\nDate: 2025-09-17T08:05:00\nFrom: Jennifer Walsh\nParticipants: Sarah Mitchell\n\nBody:\nHi Sarah,\n\nFor the record, I am forwarding a summary of a serious anonymous complaint received via our ethics hotline regarding our Mexico operations. The report alleges that the EHS Manager (Ricardo Mendez) hosted multiple dinners with a local government official, provided gifts, and facilitated payments through fake invoices. There are further red flags: the Country Manager (Carlos Delgado) is alleged to have been aware, and there are mentions of organized crime involvement. These allegations, if substantiated, raise significant FCPA exposure.\n\nI strongly recommend we immediately engage outside counsel with FCPA expertise to lead an independent investigation. We must document everything, preserve all potentially relevant documents and communications, and strictly limit knowledge of this matter to essential personnel only. Please confirm next steps and advise on initial outreach to external counsel. Are there any additional protocols you want implemented at this stage?\n\nBest regards,\nJennifer\n\n--\nJennifer Walsh\nVice President, Global Compliance & Ethics\nAgave Spirits International\nCONFIDENTIAL - Attorney Work Product',
    "Subject: Re: Request for Special Pricing Exception – Key Distributor Opportunity (Q4 Impact)\nDate: 2025-09-15T16:23:00\nFrom: Thomas Bradford\nParticipants: Kevin O'Brien\n\nBody:\nHi Kevin,\n\nThank you for flagging this opportunity and providing context regarding the potential impact on our Q4 numbers. I agree that winning this distributor could be a game-changer, but we need to ensure any pricing exception aligns with our compliance framework and avoids unintended precedents. Sarah, could you please review the specifics from a legal/compliance standpoint and advise on any restrictions or additional documentation required? Assuming there are no major obstacles, I'm supportive of proceeding with a tailored discount to secure the account.\n\nLet’s coordinate a quick call Monday to finalize.\n\nBest,\nTom",
    'Subject: Re: Urgent: Water Permit Renewal Delay Threatening Production Levels\nDate: 2025-08-25T15:02:00\nFrom: Carlos Delgado\nParticipants: Roberto Garza\n\nBody:\nHola Roberto,\n\nThank you for raising your concerns. I completely understand the pressure this delay puts on the plant, but I want to reassure you that Rick has the situation well in hand. We’ve been investing a lot in our relationship with the local authorities—In Mexico, relationships matter, ya sabes. The mayor has been very receptive to our efforts, and we’ve had several productive meetings (and just a small dinner or two) with municipal leadership. This is how business is done here; these things take relationship building, and I’m confident we’ll have good news soon. Por favor, don’t worry—déjalo en nuestras manos. I’ll keep you posted as soon as I hear anything definitive.\n\nUn abrazo,\nCarlos',
]
query_embeddings = model.encode_query(queries)
document_embeddings = model.encode_document(documents)
print(query_embeddings.shape, document_embeddings.shape)
# [1, 1024] [3, 1024]

# Get the similarity scores for the embeddings
similarities = model.similarity(query_embeddings, document_embeddings)
print(similarities)
# tensor([[ 0.3281, -0.1060,  0.0874]], dtype=torch.bfloat16)

Evaluation

Metrics

Information Retrieval

• Dataset: val_full_corpus
• Evaluated with InformationRetrievalEvaluator

Metric	Value
cosine_accuracy@1	0.2708
cosine_accuracy@3	0.5046
cosine_accuracy@5	0.5815
cosine_accuracy@10	0.6769
cosine_precision@1	0.2708
cosine_precision@3	0.1682
cosine_precision@5	0.1163
cosine_precision@10	0.0677
cosine_recall@1	0.2708
cosine_recall@3	0.5046
cosine_recall@5	0.5815
cosine_recall@10	0.6769
cosine_ndcg@10	0.4684
cosine_mrr@10	0.4022
cosine_map@100	0.4139

Training Details

Training Dataset

Unnamed Dataset

• Size: 1,548 training samples
• Columns: sentence_0, sentence_1, sentence_2, sentence_3, and sentence_4
• Approximate statistics based on the first 1000 samples:

  	sentence_0	sentence_1	sentence_2	sentence_3	sentence_4
  type	string	string	string	string	string
  details	min: 7 tokens mean: 13.67 tokens max: 27 tokens	min: 129 tokens mean: 226.59 tokens max: 511 tokens	min: 129 tokens mean: 218.17 tokens max: 511 tokens	min: 129 tokens mean: 216.09 tokens max: 511 tokens	min: 129 tokens mean: 218.05 tokens max: 511 tokens

• Samples:

  sentence_0	sentence_1	sentence_2	sentence_3	sentence_4
  Internal communications regarding upcoming tequila industry regulations	Subject: Upcoming Regulatory Comment Period: Tequila Industry Associations Date: 2025-12-08T07:41:00 From: Sarah Mitchell Participants: Thomas Bradford; Patricia Reeves Body: Dear Colleagues, I wanted to bring to your attention the forthcoming public comment period for the draft regulations being proposed for tequila industry associations. This is a critical opportunity for industry stakeholders to present their perspectives and ensure that the final framework reflects operational realities and best practices. I encourage all relevant departments to review the proposal closely and prepare any formal comments or suggestions to submit before the deadline. Please coordinate internally to consolidate feedback and let me know if you require any support or clarification regarding the submission process. Thank you and best regards, Sarah Mitchell General Counsel	Subject: Request for Approval: Maintenance Budget Variance – Q2 Date: 2026-01-26T10:51:00 From: David Chen Participants: Maria Santos Body: Hi Maria, Following a detailed review of our Q2 maintenance expenditures, the data shows that actual costs exceeded the initial budget by 18%. The variance stems primarily from unanticipated repairs to the distillation equipment and supply chain delays that increased third-party maintenance fees. From a financial perspective, I require a full breakdown of each line item, accompanying invoices, and a variance analysis compared to the original budget. Vague descriptors such as 'misc. repairs' are not sufficient for approval—please provide supporting documentation and justifications for each overage. Once I have this detail, I will evaluate the request accordingly. Thanks, David	Subject: Request for Travel Budget Exception: Supplier Audit in Guadalajara Date: 2025-10-13T18:03:00 From: Miguel Torres Participants: Maria Santos Body: Hi Maria, I am writing to request a travel budget exception for our upcoming supplier audit in Guadalajara. The current accounts do not cover the additional airfare and lodging costs, which are $1,830 MXN and $3,200 MXN respectively, as vendor rates have increased by 15% since last quarter. The data shows this audit is required to maintain ISO compliance and mitigate supply chain risks. Please advise what documentation you require to process this exception. I have attached initial quotes and the audit schedule for your review. Thank you for your prompt attention to this matter. Best regards, Miguel -- Miguel Torres Procurement Manager ASI Mexico	Subject: Campaña en redes sociales: control del mensaje y estrategia proactiva Date: 2025-11-10T08:50:00 From: Daniel Wright Participants: Tom Bradford; Patricia Reeves Body: Estimados, Quisiera poner sobre la mesa la importancia de definir una narrativa clara y cohesiva antes de lanzar cualquier contenido de nuestra próxima campaña en redes sociales. Me preocupa cómo pueden interpretarse ciertos mensajes externos, especialmente en el actual clima mediático. Debemos asegurarnos de que cada publicación refuerce la reputación ética y la transparencia de Agave Spirits International. Sugiero que revisemos juntos el calendario de publicaciones y los mensajes clave antes de su difusión para anticipar posibles titulares en la prensa y evitar interpretaciones adversas. Quedo atento a sus comentarios para coordinar una reunión esta semana. Saludos cordiales, Daniel Wright -- Daniel Wright Director, Corporate Communications Agave Spirits International
  How did managers justify onboarding shell or lightly documented consulting firms?	Subject: Re: New Vendor Setup - Consultoria Verde de Jalisco (Urgent) Date: 2025-12-22T09:26:00 From: Carlos Delgado Participants: Michelle Santos; Ricardo Mendez Body: Hola Michelle, Rick is right—this is a top priority for our Mexico operations and we need Consultoria Verde de Jalisco operational immediately. I’ve personally reviewed the engagement: it’s just straightforward EHS consulting, nothing more. Please process their setup under the standard track and we’ll deliver any remaining documentation early next week. En México, las relaciones importan y así se mueve el negocio—if we wait, we risk operational disruption. I appreciate your attention to compliance, but sometimes local realities require us to act with agility. Let me know if you need anything else from my side, but for now, I ask you to move forward as requested. Gracias por tu apoyo y comprensión. Saludos, Carlos	Subject: PRIVILEGED AND CONFIDENTIAL - ATTORNEY-CLIENT COMMUNICATION - Mexico Operations Concern Date: 2025-09-17T08:05:00 From: Jennifer Walsh Participants: Sarah Mitchell Body: Hi Sarah, For the record, I am forwarding a summary of a serious anonymous complaint received via our ethics hotline regarding our Mexico operations. The report alleges that the EHS Manager (Ricardo Mendez) hosted multiple dinners with a local government official, provided gifts, and facilitated payments through fake invoices. There are further red flags: the Country Manager (Carlos Delgado) is alleged to have been aware, and there are mentions of organized crime involvement. These allegations, if substantiated, raise significant FCPA exposure. I strongly recommend we immediately engage outside counsel with FCPA expertise to lead an independent investigation. We must document everything, preserve all potentially relevant documents and communications, and strictly limit knowledge of this matter to essential ...	Subject: Campaña en redes sociales: control del mensaje y estrategia proactiva Date: 2025-11-10T08:50:00 From: Daniel Wright Participants: Tom Bradford; Patricia Reeves Body: Estimados, Quisiera poner sobre la mesa la importancia de definir una narrativa clara y cohesiva antes de lanzar cualquier contenido de nuestra próxima campaña en redes sociales. Me preocupa cómo pueden interpretarse ciertos mensajes externos, especialmente en el actual clima mediático. Debemos asegurarnos de que cada publicación refuerce la reputación ética y la transparencia de Agave Spirits International. Sugiero que revisemos juntos el calendario de publicaciones y los mensajes clave antes de su difusión para anticipar posibles titulares en la prensa y evitar interpretaciones adversas. Quedo atento a sus comentarios para coordinar una reunión esta semana. Saludos cordiales, Daniel Wright -- Daniel Wright Director, Corporate Communications Agave Spirits International	Subject: Upcoming Regulatory Comment Period: Tequila Industry Associations Date: 2025-12-08T07:41:00 From: Sarah Mitchell Participants: Thomas Bradford; Patricia Reeves Body: Dear Colleagues, I wanted to bring to your attention the forthcoming public comment period for the draft regulations being proposed for tequila industry associations. This is a critical opportunity for industry stakeholders to present their perspectives and ensure that the final framework reflects operational realities and best practices. I encourage all relevant departments to review the proposal closely and prepare any formal comments or suggestions to submit before the deadline. Please coordinate internally to consolidate feedback and let me know if you require any support or clarification regarding the submission process. Thank you and best regards, Sarah Mitchell General Counsel
  What internal controls are in place for reviewing beneficiary account changes before processing payments?	Subject: Re: Urgent: Discrepancy in Beneficiary Account Information for Broker Payment Date: 2025-10-22T20:38:00 From: Maria Santos Participants: James Cooper Body: Hi James, Thank you for catching this and bringing it to my attention. I've already reached out to the broker requesting the latest official documentation confirming their beneficiary account details. As soon as I receive the required paperwork, I will forward it to you for review. If you have a preferred template or checklist you'd like the broker to use, please send it along, and I'll make sure it's followed to the letter. My priority is to ensure compliance and avoid any delays in the payment process. Appreciate your vigilance on this, Maria	Subject: Trade Association Political Activity Disclosure and Government Engagement Compliance Date: 2025-10-13T08:19:00 From: Amanda Foster Participants: Sarah Mitchell Body: Hi Sarah, I wanted to touch base regarding our upcoming trade association political activity disclosures and ensure we remain fully compliant with both US (e.g., LDA, FARA) and Mexican (Transparencia y Acceso a la Información Pública) regulations. As you know, all ASI government interactions are run through our pre-clearance process, including detailed documentation of business purposes and regulatory filings. Routine reporting—such as our semi-annual LDA filings in the US and quarterly submissions to Mexican authorities—reflects only legitimate, transparent engagement, all pre-approved in writing and logged for audit purposes. This diligence is especially critical given recent events involving unofficial channels. Let me know if you need any further documentation from Government Relations for the upcoming comp...	Subject: RE: Request for Departmental Budget Inputs – FY2025 Planning Date: 2025-08-12T15:38:00 From: Carlos Delgado Participants: Maria Santos Body: Dear Maria, Thank you for your detailed instructions on the budget process. Attached you will find our department’s completed budget template for FY2025, with all supporting documentation as requested. We have taken special care to ensure that each line item is accompanied by clear justifications. As siempre, I want to highlight that in Mexico, relationships matter—a veces eso means a small dinner with community leaders or a modest contribution to local fiestas (you'll see these reflected under 'Community & Stakeholder Engagement'). This is how business is done here; these are key to maintaining our strong ties and smooth operational flow. Please note, we have pending invoices for environmental consulting that will be forwarded next week. If there are any doubts or further documentation needed, let me know and our team will resolve the...	Subject: Revisión de posicionamiento de marca y objetivos de Q4 Date: 2025-10-13T15:16:00 From: Kevin O'Brien Participants: Thomas Tom Bradford; Sarah Mitchell; Patricia Reeves Body: Estimado equipo ejecutivo, Quisiera destacar que, mientras nos acercamos al cierre de Q4, hemos observado cambios importantes en el posicionamiento de nuestra marca dentro del mercado norteamericano. Nuestros números preliminares de ventas indican una mejora notable en la cuota de mercado, especialmente en canales premium y distribución independiente. Sin embargo, la competencia está reaccionando agresivamente, lo que implica un ajuste ágil de nuestras estrategias de precio y visibilidad en el punto de venta. Si bien entiendo la importancia de los procesos de compliance, me preocupa que algunas revisiones internas estén ralentizando cierres claves justo en temporada alta. Sugiero que exploremos vías para agilizar la aprobación de nuevos contratos sin perder el control ni la transparencia requerida. Agra...

• Loss: main.DPOLoss

Training Hyperparameters

Non-Default Hyperparameters

• per_device_train_batch_size: 6
• per_device_eval_batch_size: 6
• multi_dataset_batch_sampler: round_robin

All Hyperparameters

Click to expand

• do_predict: False
• eval_strategy: no
• prediction_loss_only: True
• per_device_train_batch_size: 6
• per_device_eval_batch_size: 6
• gradient_accumulation_steps: 1
• eval_accumulation_steps: None
• torch_empty_cache_steps: None
• learning_rate: 5e-05
• weight_decay: 0.0
• adam_beta1: 0.9
• adam_beta2: 0.999
• adam_epsilon: 1e-08
• max_grad_norm: 1
• num_train_epochs: 3
• max_steps: -1
• lr_scheduler_type: linear
• lr_scheduler_kwargs: None
• warmup_ratio: None
• warmup_steps: 0
• log_level: passive
• log_level_replica: warning
• log_on_each_node: True
• logging_nan_inf_filter: True
• enable_jit_checkpoint: False
• save_on_each_node: False
• save_only_model: False
• restore_callback_states_from_checkpoint: False
• use_cpu: False
• seed: 42
• data_seed: None
• bf16: False
• fp16: False
• bf16_full_eval: False
• fp16_full_eval: False
• tf32: None
• local_rank: -1
• ddp_backend: None
• debug: []
• dataloader_drop_last: False
• dataloader_num_workers: 0
• dataloader_prefetch_factor: None
• disable_tqdm: False
• remove_unused_columns: True
• label_names: None
• load_best_model_at_end: False
• ignore_data_skip: False
• fsdp: []
• fsdp_config: {'min_num_params': 0, 'xla': False, 'xla_fsdp_v2': False, 'xla_fsdp_grad_ckpt': False}
• accelerator_config: {'split_batches': False, 'dispatch_batches': None, 'even_batches': True, 'use_seedable_sampler': True, 'non_blocking': False, 'gradient_accumulation_kwargs': None}
• parallelism_config: None
• deepspeed: None
• label_smoothing_factor: 0.0
• optim: adamw_torch_fused
• optim_args: None
• group_by_length: False
• length_column_name: length
• project: huggingface
• trackio_space_id: trackio
• ddp_find_unused_parameters: None
• ddp_bucket_cap_mb: None
• ddp_broadcast_buffers: False
• dataloader_pin_memory: True
• dataloader_persistent_workers: False
• skip_memory_metrics: True
• push_to_hub: False
• resume_from_checkpoint: None
• hub_model_id: None
• hub_strategy: every_save
• hub_private_repo: None
• hub_always_push: False
• hub_revision: None
• gradient_checkpointing: False
• gradient_checkpointing_kwargs: None
• include_for_metrics: []
• eval_do_concat_batches: True
• auto_find_batch_size: False
• full_determinism: False
• ddp_timeout: 1800
• torch_compile: False
• torch_compile_backend: None
• torch_compile_mode: None
• include_num_input_tokens_seen: no
• neftune_noise_alpha: None
• optim_target_modules: None
• batch_eval_metrics: False
• eval_on_start: False
• use_liger_kernel: False
• liger_kernel_config: None
• eval_use_gather_object: False
• average_tokens_across_devices: True
• use_cache: False
• prompts: None
• batch_sampler: batch_sampler
• multi_dataset_batch_sampler: round_robin
• router_mapping: {}
• learning_rate_mapping: {}

Training Logs

Epoch	Step	val_full_corpus_cosine_ndcg@10
1.0	258	0.4684

Framework Versions

• Python: 3.12.12
• Sentence Transformers: 5.2.3
• Transformers: 5.0.0
• PyTorch: 2.10.0+cu128
• Accelerate: 1.13.0
• Datasets: 4.0.0
• Tokenizers: 0.22.2

Citation

BibTeX

Sentence Transformers

@inproceedings{reimers-2019-sentence-bert,
    title = "Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks",
    author = "Reimers, Nils and Gurevych, Iryna",
    booktitle = "Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing",
    month = "11",
    year = "2019",
    publisher = "Association for Computational Linguistics",
    url = "https://arxiv.org/abs/1908.10084",
}