Panhapich/diarizen-khmer-sim-finetuned

Khmer DiariZen Fine-Tuned on FastMSS Simulated Meetings

This is a Khmer speaker-diarization checkpoint fine-tuned from BUT-FIT/diarizen-wavlm-large-s80-md-v2 on the Panhapich/fastmss_meeting_simulation dataset.

The base model is DiariZen with an 80%-pruned WavLM Large encoder and a Conformer EEND head. The fine-tune keeps WavLM frozen and trains the diarization head for 8 epochs on 117.18 hours of simulated Khmer meetings.

Research pilot: the training and test data are synthetic, clean, close-talk read speech with only 20 speakers. This checkpoint has not been evaluated on real Khmer meetings. Do not treat its scores as real-world performance.

Results

Evaluation is on the 82-meeting / 1.70 h Khmer simulated test split, using DER at 0.25 s collar and ignore_overlaps=false.

System	DER	JER	B3-F1
Zero-shot base DiariZen v2	9.58%	19.95%	0.89
This fine-tuned checkpoint	4.62%	15.53%	0.96

• Absolute DER improvement: 4.96 points
• Gate target: ≤ 8.58% DER
• Gate result: PASS
• Best checkpoint: epoch 8
• Validation NLL: 0.1506
• Validation DER: 3.10%

The checkpoint was selected by validation NLL, not by test-set DER.

Usage

Install DiariZen following its repository instructions:

• https://github.com/BUTSpeechFIT/DiariZen

Then load this checkpoint:

from diarizen.pipelines.inference import DiariZenPipeline

pipeline = DiariZenPipeline.from_pretrained(
    "Panhapich/diarizen-khmer-sim-finetuned"
)

diarization = pipeline("meeting.wav", sess_name="meeting")

for turn, _, speaker in diarization.itertracks(yield_label=True):
    print(f"{turn.start:.2f}-{turn.end:.2f}s  {speaker}")

To write an RTTM file:

pipeline = DiariZenPipeline.from_pretrained(
    "Panhapich/diarizen-khmer-sim-finetuned",
    rttm_out_dir=".",
)
pipeline("meeting.wav", sess_name="meeting")

from_pretrained downloads the WeSpeaker ResNet34 embedding model from pyannote/wespeaker-voxceleb-resnet34-LM automatically.

The checkpoint expects 16 kHz mono audio. Resample multi-channel or higher-rate recordings before inference.

Training details

Item	Value
Base checkpoint	BUT-FIT/diarizen-wavlm-large-s80-md-v2
Fine-tune type	Frozen WavLM, train EEND head
Epochs	8
Best epoch	8
Chunk size	16 s
Train batch size	1
Gradient accumulation	4
Validation batch size	2
Optimizer	AdamW
Head LR	2e-5
Other LR	1e-3
Seed	42
GPU	4 GB RTX 3050 Laptop
Train data	6,207 meetings / 117.176 h / 16 speakers
Dev data	611 meetings / 11.497 h / 17 speakers
Test data	82 meetings / 1.699 h / 16 speakers

See training_config.toml for the full training configuration. Paths in that file are placeholders; replace them with your local dataset and model paths. See dataset_split.json for the exact split and leakage report.

Data

Training data: Panhapich/fastmss_meeting_simulation

• 6,900 simulated Khmer meetings, 130.37 h, 20 speakers
• 2–4 speakers per meeting
• 16 kHz mono, 16-bit PCM WAV
• No reverberation, noise, or FIR filtering
• Source corpus: Digital-Divide-Data/khmer-speech-dataset (CC BY-SA 4.0)
• Simulation: FastMSS, https://github.com/popcornell/FastMSS

The split is a designated-speaker holdout, but 39 source cuts overlap train and test and 451 source cuts overlap train and dev. It is not a fully source-cut-disjoint split.

Files

config.toml             DiariZen inference and clustering configuration
pytorch_model.bin       Fine-tuned epoch-8 checkpoint
plda/                   PLDA and x-vector transform for VBx clustering
dataset_split.json      Train/dev/test split and leakage report
training_config.toml    Training configuration with placeholder paths
README.md               This model card
LICENSE                 CC BY-NC 4.0 notice

Limitations

• Synthetic, clean, close-talk read speech
• Only 20 source speakers
• No real Khmer meeting evaluation
• No noise, reverberation, far-field, or spontaneous speech evaluation
• Source cuts overlap across splits
• Supports up to four concurrent speakers, matching the base model
• Non-commercial use only, inherited from the base model

License

This checkpoint is released under CC BY-NC 4.0, following the base DiariZen model license. The dataset is distributed separately under CC BY-SA 4.0.

See LICENSE.

This model is not affiliated with or endorsed by the DiariZen authors or the source corpus publisher.

Citation

If you use this checkpoint, please cite DiariZen and FastMSS.

@unpublished{polok2026mindthegap,
  author = {Polok, Alexander and Medennikov, Ivan and Watanabe, Shinji and
            {\v{C}}ernock{\'y}, Jan and Burget, Luk{\'a}{\v{s}} and
            Cornell, Samuele},
  title = {Mind the Gap: Impact of Synthetic Conversational Data on
           Multi-Talker {ASR} and Speaker Diarization},
  note = {Submitted to INTERSPEECH},
  year = {2026}
}

DiariZen: https://github.com/BUTSpeechFIT/DiariZen