PsarAI-ASR-mHuBERT

A small (94.5M parameters, 378 MB) speech recognizer for Khmer, English, French and speech that mixes them in one sentence. It is mHuBERT-147 fine-tuned with a CTC head on one shared character vocabulary (158 symbols: Khmer script, Latin letters, French accents). No language setting: one model, any of the three languages, switching mid-sentence.

It is meant as a pretrained starting point: a 10× smaller alternative to MMS-1B for Khmer, to fine-tune further or to build smaller speech models on.

Usage

import torch, soundfile as sf
from transformers import AutoProcessor, AutoModelForCTC

processor = AutoProcessor.from_pretrained("nphearum/PsarAI-ASR-mHuBERT")
model = AutoModelForCTC.from_pretrained("nphearum/PsarAI-ASR-mHuBERT").eval()

audio, sr = sf.read("speech.wav", dtype="float32")          # 16 kHz mono
inputs = processor(audio, sampling_rate=16000, return_tensors="pt")
with torch.inference_mode():
    ids = model(inputs.input_values).logits.argmax(-1)[0]
print(processor.decode(ids))

Input is 16 kHz mono. Do not pass an attention_mask when batching (mHuBERT uses group-norm features: pad with zeros instead).

Results

Content CER (letters and marks only; spaces, punctuation and case ignored), greedy decoding, held-out data, next to the MMS-1B model this was compared against (MMS-1B-all with a fine-tuned code-switching adapter, same vocabulary):

Set	PsarAI-ASR-mHuBERT (95M)	MMS-1B + adapter (965M)
Mixed speech, held out (Khmer+English, Khmer+French, 300 clips)	3.02%	2.19%
FLEURS Khmer test (100 clips)	13.40%	12.44%
FLEURS English test (100 clips)	11.72%	5.08%
FLEURS French test (100 clips)	12.70%	5.01%

Khmer is close to the 10× larger model. English and French are recognized by sound but spelled less reliably ("mad of mater" for "made of matter"); the training data has much less English (14 h) than Khmer.

Training

• Data (97,957 clips, 205 h): Khmer read speech from the Digital Divide Data Khmer speech dataset (100 h), FLEURS train (km, en, fr; 17 h), Common Voice French (44 h), and synthetic single-voice Khmer, English, French and mixed Khmer–English / Khmer–French speech (VoxCPM2, ~44 h). Clips over 20 s were left out.
• Recipe: CNN feature encoder frozen throughout. 1,000 steps training only the CTC head on the frozen encoder, then ~4 epochs (15,096 steps) of the whole transformer with three learning rates: encoder 5e-5, feature projection 3e-4, head 1.5e-3 (warm-up 1,000 steps, linear decay). Batches of up to 200 s of padded audio, bf16, layerdrop 0.05, time masking 0.05. The best step was picked on the held-out mixed set. Training the encoder at a high learning rate from the start, or unfreezing it at 3e-4, sent CTC back to all-blank output; the head-first, slow-encoder recipe avoids that.

Limitations

• Output is lowercase; numbers are written as words; some punctuation (។ . ?) is produced, mostly after synthetic training data.
• Khmer is written without spaces between words, so word-level metrics and timestamps are per phrase.
• English and French spelling is weaker than larger models (see results).
• About a fifth of the training audio is one synthetic voice; accuracy on other voices is measured by FLEURS above.

License

CC-BY-NC-SA-4.0 (non-commercial, share-alike), following the base model utter-project/mHuBERT-147. Check the licenses of the training datasets for your use.