moonshine-tiny-accent-ONNX

Moonshine-tiny (27M params) fine-tuned for non-native-accented English — speakers whose first language is Korean, Hindi, Mandarin, Vietnamese, Arabic, or Spanish — exported to ONNX for transformers.js so it runs entirely in the browser (WebGPU / WASM). Built for the live-transcription demo on lucascho.fyi: audio never leaves the visitor's device.

Results (normalized WER, held-out speakers never seen in training)

eval set	base moonshine-tiny	this model	rel. change
L2-ARCTIC, unseen accented speakers	25.9%	17.1%	−34%
— Korean L1	12.7%	6.1%	−52%
— Hindi L1	12.7%	8.0%	−37%
— Arabic L1	23.1%	14.6%	−37%
— Mandarin L1	26.3%	18.9%	−28%
— Vietnamese L1	44.4%	29.1%	−34%
— Spanish L1	37.6%	27.1%	−28%
VCTK, unseen native speakers	6.1%	4.3%	−30%
LibriSpeech test-clean	4.8%	7.0%	+46% (regression)

Honest caveat: the LibriSpeech regression reflects domain shift toward short read sentences (the training data) and away from long-form audiobook audio. For the intended use — short live-caption segments — the accented and native short-form gains dominate.

Training

• Data: L2-ARCTIC v5.0 (24 non-native speakers, 6 L1s, 27h; ×2 in the mix) + VCTK 0.92 (110 native UK/US/etc speakers, ~40h). One speaker per L1 / per accent held out for eval.
• Method: LoRA (r=16, α=32) on all attention projections of encoder+decoder, 663K trainable params (2.4%), fp32, 2 epochs, LR 1e-4, then merged into the base (merge_and_unload) — so this repo is a plain Moonshine checkpoint.
• Hardware: 1× V100, Northeastern University Explorer cluster (SLURM).
• Fine-tuning initially collapsed into word-skipping under transformers 4.57 due to a double label shift in the loss path (shift_tokens_right in the model + ForCausalLMLoss shifting again); fixed by passing decoder_input_ids explicitly with aligned labels.

Usage (transformers.js)

import { pipeline } from "@huggingface/transformers";

const transcriber = await pipeline(
  "automatic-speech-recognition",
  "lucascho/moonshine-tiny-accent-ONNX",
  { device: "webgpu", dtype: "q4" } // fallback: { device: "wasm", dtype: "q8" }
);
const { text } = await transcriber(float32PcmAt16kHz);

License & attribution

CC-BY-NC-4.0 (inherited from L2-ARCTIC training data — non-commercial). Base model: Moonshine (© Useful Sensors, MIT). Corpora: L2-ARCTIC (Texas A&M PSI lab), CSTR VCTK (University of Edinburgh, CC-BY-4.0).