vanshnawander/sapc2-moonshine-tiny-rl

Moonshine Tiny (27M) for dysarthric speech: SFT + RL (SAPC2)

Fine-tuned for the Speech Accessibility Project Challenge 2 (SAPC2): English speech from speakers with ALS, cerebral palsy, Down syndrome, Parkinson's disease and stroke. Starting point: moonshine-ai/moonshine-tiny, fine-tuned on SAPC2 Train (SOAP, duration-balanced sampling), then one epoch of RLOO-style RL with a WER/CER reward and a cross-entropy anchor. This is the final RL checkpoint (step 4446), used for the Track 2 submission.

Evaluation

The checkpoint was selected on a fixed 1,000-recording subset of the official Dev set; the scores below are on the full Dev set (47,929 recordings). Scores use the competition normalizer against the better of the two references; they are within about 0.1 points of the official SCTK scorer on the same predictions.

Full Dev (47,929 recordings)

Whole-recording decoding (200 s windows). Quick scorer: competition normalizer, better of the two references. Official: the challenge's evaluate.py with sclite against both references (where it was run; it scores about 0.05 points lower than the quick scorer).

decoding	WER	CER	official SCTK WER	official SCTK CER	empty transcripts	capped (ran to the token limit)
greedy	18.62%	12.77%	18.57%	12.73%	0	31
beam 2	17.34%	11.93%	--	--	0	33
beam 4	16.72%	11.55%	16.68%	11.52%	0	58

Exact transcripts 61.47%; substitutions 13.32%, deletions 2.99%, insertions 4.00% of reference words (uncapped WER 20.32%).

CPU streaming package cost

The ONNX package run through the challenge's streaming interface on a 300-recording Dev subset (0.64 h of audio), two worker processes, one ONNX Runtime thread each, on a laptop CPU, with no other load. Compute is the wall-clock seconds spent per hour of audio; WER/CER are on that subset only.

final decode	compute per audio hour	WER	CER
greedy	313 s	18.75%	12.42%
beam 4	368 s	17.36%	11.76%

WER by duration

bucket	recordings	WER	share of all errors
1-2s	670	24.98%	0.42%
2-3s	4354	15.04%	2.87%
3-5s	14700	12.39%	10.10%
5-8s	14739	15.19%	17.30%
8-12s	7023	20.34%	16.11%
12-20s	3867	24.29%	17.59%
20-35s	1657	22.90%	15.94%
>35s	919	20.37%	19.66%

WER by etiology

etiology	recordings	WER	share of all errors
ALS	10232	15.66%	19.19%
Cerebral Palsy	11472	29.26%	29.46%
Down Syndrome	7719	25.57%	15.87%
Parkinson's Disease	14543	13.67%	30.29%
Stroke	3963	17.93%	5.20%

Runaway transcripts (hypothesis over 1.5x the reference length): 355, 8.98% of all errors; truncated transcripts: 210.

CPU streaming package (Track 2)

The submission packages this checkpoint as ONNX FP32 (encoder, decoder and decoder-with-cache graphs exported with optimum; 240 MB zipped) behind the challenge's 5-method streaming interface: greedy partial transcripts every 0.5 s for the first 4 s and every 10 s afterwards, then one final decode of the whole recording with the configured beam width. ONNX and PyTorch greedy transcripts match on 39 of 40 parity recordings; through the streaming interface the package scores 18.75% greedy / 17.36% beam 4 on a 300-recording Dev pilot.

Streaming latency (CPU package)

Measured with the challenge's own local decoder and latency script on a 100-recording Dev subset (12.5 min of audio), audio sent at real-time pace in 100 ms chunks, one ONNX Runtime thread, on a laptop CPU. TTFT here is the first partial relative to the start of the audio (the official metric subtracts the MFA speech onset, which this subset does not carry); partial transcripts are greedy at every setting, so TTFT does not depend on the final beam width. TTLT is the final transcript after the last audio chunk.

final beams	TTFT median	TTLT median	TTLT p90	TTLT p95	TTLT max	full-Dev WER / CER
1	420 ms	148 ms	311 ms	414 ms	2037 ms	18.62% / 12.77%
2	422 ms	166 ms	348 ms	405 ms	2777 ms	17.34% / 11.93%
4	421 ms	201 ms	455 ms	542 ms	5036 ms	16.72% / 11.55%

Training recipe

{
  "objective": "rl",
  "init": "runs/sft_continue_v1/best",
  "peak_lr": 2e-06,
  "epochs": 1,
  "balanced": true,
  "balance_max_repeats": 2,
  "rl": {
    "samples_per_recording": 2,
    "kl_weight": 0.02,
    "ce_anchor": 0.1
  },
  "policy": {
    "max_items": 96,
    "padded_seconds": 576,
    "attention_cost": 60000,
    "decoder_tokens": 12288,
    "accumulation": 2
  },
  "seed": 20261023
}

Optimizer: blocked SOAP (Adam in a Shampoo eigenbasis), FP32 weights with BF16 autocast. Code: the moonshine package in the project repository (training, distillation, RL and evaluation), with resumable checkpoints.

Files

Hugging Face format (config.json, model.safetensors, tokenizer/processor files). Load with transformers (MoonshineForConditionalGeneration or CohereAsrForConditionalGeneration).