PsarAI-ASR-1B (PyTorch + ONNX)

Speech recognition for Khmer (ខ្មែរ), English, and mixed Khmer–English speech (wav2vec2, ~965M parameters, CTC).

This repository contains the original PyTorch weights of nphearum/PsarAI-ASR-1B plus ONNX exports (fp32, fp16, int8) that run with onnxruntime only, without PyTorch.

Files

Path	Format	Size	Use it for
model.safetensors (+ configs)	PyTorch, fp16 weights	1.9 GB	🤗 Transformers, GPU or CPU
onnx/fp32/	ONNX, fp32	3.9 GB	reference accuracy, CPU/GPU
onnx/fp16/	ONNX, fp16 (fp32 inputs/outputs)	1.9 GB	GPU (CUDA / DirectML)
onnx/int8/	ONNX, int8 dynamic quantization	1.0 GB	CPU (smallest, recommended for CPU)
onnx/infer_onnx.py	script		command-line transcription with ONNX
onnx/export_onnx.py	script		how the ONNX files were made
results/	benchmark		CER / speed comparison of all variants

Each onnx/<variant>/ folder is self-contained (model.onnx + model.onnx.data + vocab/tokenizer files).

Install

# PyTorch / Transformers
pip install torch transformers librosa soundfile

# ONNX only (no torch needed)
pip install onnxruntime librosa huggingface_hub     # CPU
pip install onnxruntime-gpu librosa huggingface_hub # NVIDIA GPU

Quick test — 🤗 Transformers

import librosa
import torch
from transformers import AutoProcessor, Wav2Vec2ForCTC

REPO = "phonsobon/PsarAI-ASR-1B"
device = "cuda" if torch.cuda.is_available() else "cpu"

processor = AutoProcessor.from_pretrained(REPO)
model = Wav2Vec2ForCTC.from_pretrained(REPO, dtype=torch.float32).to(device).eval()

audio, _ = librosa.load("speech.wav", sr=16_000, mono=True)  # any format librosa reads; 16 kHz mono
inputs = processor(audio, sampling_rate=16_000, return_tensors="pt").to(device)
with torch.inference_mode():
    logits = model(**inputs).logits
print(processor.decode(logits.argmax(-1)[0]))

Weights are stored in float16; load them as float32 as above, especially on CPU. Clips of up to ~30 s fit comfortably on a 24 GB GPU.

Long audio (pipeline with chunking)

from transformers import pipeline

asr = pipeline(
    "automatic-speech-recognition",
    model="phonsobon/PsarAI-ASR-1B",
    device=0,             # -1 for CPU
    chunk_length_s=20,    # split long recordings into 20 s windows
    stride_length_s=4,    # overlap between windows
)
print(asr("long_recording.wav")["text"])

Quick test — ONNX Runtime (no PyTorch)

Command line

# Download only the int8 model + script (~1 GB instead of ~9 GB)
hf download phonsobon/PsarAI-ASR-1B --include "onnx/int8/*" "onnx/infer_onnx.py" --local-dir PsarAI-ASR-1B

python PsarAI-ASR-1B/onnx/infer_onnx.py speech.wav                                # int8 on CPU
python PsarAI-ASR-1B/onnx/infer_onnx.py speech.wav --variant fp16 --device cuda   # needs onnx/fp16/*
python PsarAI-ASR-1B/onnx/infer_onnx.py speech.wav --variant fp16 --device dml    # Windows DirectML

(With older huggingface_hub, use huggingface-cli download instead of hf download.)

Python

import json
from pathlib import Path

import librosa
import numpy as np
import onnxruntime as ort
from huggingface_hub import snapshot_download

VARIANT = "int8"  # "int8" for CPU, "fp16" for GPU, "fp32" for reference
local = snapshot_download("phonsobon/PsarAI-ASR-1B", allow_patterns=[f"onnx/{VARIANT}/*"])
model_dir = Path(local) / "onnx" / VARIANT

vocab = json.loads((model_dir / "vocab.json").read_text(encoding="utf-8"))
id_to_token = {i: t for t, i in vocab.items()}
skip_ids = {vocab[t] for t in ("<pad>", "<s>", "</s>", "<unk>") if t in vocab}

session = ort.InferenceSession(
    str(model_dir / "model.onnx"),
    providers=["CPUExecutionProvider"],  # ["CUDAExecutionProvider", "CPUExecutionProvider"] for GPU
)

def transcribe(path: str) -> str:
    audio, _ = librosa.load(path, sr=16_000, mono=True)
    x = ((audio - audio.mean()) / np.sqrt(audio.var() + 1e-7)).astype(np.float32)[None, :]
    mask = np.ones_like(x, dtype=np.int64)
    (logits,) = session.run(["logits"], {"input_values": x, "attention_mask": mask})

    # Greedy CTC decoding: merge repeats, drop blanks/special tokens, "|" = space.
    out, prev = [], None
    for i in logits[0].argmax(-1).tolist():
        if i != prev and i not in skip_ids:
            out.append(id_to_token[i])
        prev = i
    return " ".join("".join(out).replace("|", " ").split())

print(transcribe("speech.wav"))

Test accuracy on a public Khmer test set

Computes CER on a few clips of seanghay/khmer_grkpp_speech (real Khmer news speech, never used in training).

pip install "datasets[audio]" jiwer

import re

import jiwer
import torch
from datasets import Audio, load_dataset
from transformers import AutoProcessor, Wav2Vec2ForCTC

REPO = "phonsobon/PsarAI-ASR-1B"
N_CLIPS = 20
device = "cuda" if torch.cuda.is_available() else "cpu"

processor = AutoProcessor.from_pretrained(REPO)
model = Wav2Vec2ForCTC.from_pretrained(REPO, dtype=torch.float32).to(device).eval()

ds = load_dataset("seanghay/khmer_grkpp_speech", split="train", streaming=True)
ds = ds.cast_column("audio", Audio(sampling_rate=16_000))

def norm(text: str) -> str:
    # letters only: drop spaces, punctuation and zero-width characters
    return re.sub(r"[\s​‌‍។៖!?.,\"'()\-]", "", text.lower())

refs, hyps = [], []
for sample in ds:
    audio = sample["audio"]["array"]
    if not 1 <= len(audio) / 16_000 <= 30:  # same 1–30 s range as the reported results
        continue
    inputs = processor(audio, sampling_rate=16_000, return_tensors="pt").to(device)
    with torch.inference_mode():
        pred = processor.decode(model(**inputs).logits.argmax(-1)[0])
    refs.append(norm(sample["transcription"]))
    hyps.append(norm(pred))
    print(f"REF: {sample['transcription']}\nHYP: {pred}\n")
    if len(refs) == N_CLIPS:
        break

print(f"CER on {len(refs)} clips: {jiwer.cer(refs, hyps):.2%}")

This simple normalization does not convert digits to words, so expect a CER somewhat above the 13.73% reported below (which uses the full normalization described in Results).

Results

Character error rate (CER, letters only: spaces and punctuation ignored) and word error rate (WER), lower is better. Before scoring, references and transcripts are normalized the same way: canonical Khmer character order, one spelling per word (ឲ្យ/អោយ → ឱ្យ), numbers as spoken words (២០១៩ → ពីរពាន់ដប់ប្រាំបួន).

Test set	Clips	What it is	Greedy	With Khmer LM decoder*
grkpp (news)	348	seanghay/khmer_grkpp_speech, new speakers and topics	13.73% CER	9.99% CER
grkpp, no digits	125	same, clips whose transcript has no digits	11.68% CER	8.95% CER
khmer10	10	DDD speaker never used in training	4.84% CER	3.01% CER
DDD held-out	150	Digital-Divide-Data clips not used in training	5.87% CER	2.12% CER
Jarvis Khmer	100	held-out synthetic voice, Khmer	1.27% CER	0.17% CER
Jarvis English	100	held-out synthetic voice, English	1.39% WER	0.49% WER
Jarvis mixed	100	held-out synthetic voice, Khmer–English	12.43% WER	2.99% WER

* Greedy is what this model outputs on its own. The second column adds a separate decoder: CTC beam search with a Khmer character 6-gram and a word trigram, which is not included in this repository.

PyTorch vs ONNX variants

Benchmark on 136 clips (42 min: English, Khmer, and mixed Khmer–English voice-over). RTF = processing time / audio duration (lower is faster). Full table: results/summary.md.

Variant	Device	CER	RTF	Size
PyTorch	CUDA	6.28%	0.009	1.93 GB
ONNX fp32	CPU	6.28%	0.370	3.86 GB
ONNX fp16	CPU	6.28%	0.732	1.93 GB
ONNX int8	CPU	6.34%	0.545	1.02 GB

• All ONNX variants match PyTorch accuracy (int8 loses < 0.1 pt CER).
• On CPU, use int8 (smallest) or fp32 (fastest on CPU in this test). fp16 is meant for GPU; on CPU it is slow.
• On GPU, use PyTorch or ONNX fp16 with CUDAExecutionProvider.

Training

Base	facebook/mms-1b-all, Khmer adapter (khm)
Trained	adapter layers + output head (~2.3M of ~965M parameters); base model frozen
Output vocabulary	the Khmer adapter's 152 characters + | word separator (153)
Data	~15 h of Digital-Divide-Data/khmer-speech-dataset (real read speech, many speakers); nphearum/dataset-asr-tts synthetic "Jarvis" voice: 3,000 Khmer, 2,000 English and 800 mixed clips
Steps	4,000; AdamW, learning rate 1e-3 with warm-up and linear decay; bf16 autocast
Hardware	1 × RTX 3090

See the original model card at nphearum/PsarAI-ASR-1B for full details.

ONNX export: opset 17, dynamic batch and length axes, weights in an external model.onnx.data file (the fp32 graph is above protobuf's 2 GB limit). int8 uses dynamic quantization of the transformer MatMuls only; the convolutional feature encoder stays fp32. See onnx/export_onnx.py.

Limitations

• English: trained and tested only on one synthetic voice; expect clearly weaker results on real English speakers and accents. Output is lowercase.
• Numbers: the vocabulary has the Khmer digits ០ ១ ២ ៣ ៥ ៧ but not ៤ ៦ ៨ ៩, so the model sometimes writes numbers in mixed scripts (e.g. ២០១9).
• Spaces: Khmer is written without spaces between words; the model places spaces where its training text had them (between phrases), not between every word.
• Missing characters: a few rare Khmer characters are not in the vocabulary (e.g. ៛ ៕ ឰ ឝ ឞ).
• Domain words and names it has not heard are often misspelled; an LM decoder with domain text or hotwords helps.

License and credits

Released under CC-BY-NC-4.0 (non-commercial), following the base model facebook/mms-1b-all.

• Model: nphearum/PsarAI-ASR-1B
• Training data: Digital-Divide-Data/khmer-speech-dataset (CC-BY-SA-4.0), nphearum/dataset-asr-tts (CC-BY-NC-4.0; its synthetic voice imitates a character voice whose rights belong to their owners).

Check that your use is permitted before using this model.

@article{pratap2023mms,
  title   = {Scaling Speech Technology to 1,000+ Languages},
  author  = {Pratap, Vineel and Tjandra, Andros and Shi, Bowen and others},
  journal = {arXiv preprint arXiv:2305.13516},
  year    = {2023}
}