phonsobon/whisper-base-khmer-finetune

Whisper Base — Khmer Fine-tune

This is a fine-tuned version of openai/whisper-base for automatic speech recognition (ASR) on the Khmer language.

• Base model: openai/whisper-base
• Fine-tuning dataset: phonsobon/khmer-speech-dataset
• Checkpoint: step 2000
• Task: transcribe
• Language: Khmer (km)

⚠️ Status: experimental / under evaluation. This checkpoint is uploaded for testing. Word Error Rate (WER) and other evaluation numbers have not been finalized yet.

How to use

from transformers import WhisperProcessor, WhisperForConditionalGeneration
import torch
import torchaudio

repo_id = "phonsobon/whisper-base-khmer-finetune"

processor = WhisperProcessor.from_pretrained(repo_id)
model = WhisperForConditionalGeneration.from_pretrained(repo_id)

# Force Khmer transcription (not translation)
forced_decoder_ids = processor.get_decoder_prompt_ids(language="khmer", task="transcribe")

# Load an audio file (mono, resampled to 16kHz)
waveform, sr = torchaudio.load("your_audio.wav")
if sr != 16000:
    waveform = torchaudio.functional.resample(waveform, sr, 16000)

inputs = processor(waveform.squeeze().numpy(), sampling_rate=16000, return_tensors="pt")

with torch.no_grad():
    predicted_ids = model.generate(inputs["input_features"], forced_decoder_ids=forced_decoder_ids)

transcription = processor.batch_decode(predicted_ids, skip_special_tokens=True)
print(transcription[0])

Using the pipeline API

from transformers import pipeline

asr = pipeline(
    "automatic-speech-recognition",
    model="phonsobon/whisper-base-khmer-finetune",
    generate_kwargs={"language": "khmer", "task": "transcribe"},
)

result = asr("your_audio.wav")
print(result["text"])

Training details

• Framework: 🤗 Transformers Seq2SeqTrainer
• Checkpoint step: 4000
• Training/optimizer state (optimizer.pt, scheduler.pt, rng_state.pth, scaler.pt, trainer_state.json, training_args.bin) is intentionally excluded from this repo — only the files needed for inference are included.

Limitations

• This checkpoint has not yet been benchmarked against a held-out test set.
• Performance on noisy audio, code-switched Khmer-English speech, or dialectal variation is untested.
• Feedback and evaluation contributions are welcome via the community tab.

Intended use

Built as part of Khmer language technology tooling for the Chaktomuk Managements (CTM) system under Cambodia's Ministry of Post and Telecommunications, Digital Transformation Working Group.