wav2vec2-base-960h Forced Aligner (CoreML)

On-device CTC forced aligner for word-level lyric timestamps in the Chord AI iOS app. This is the alignment half of a WhisperX-style pipeline: a transcript is aligned to audio by Viterbi-decoding the CTC emissions of a wav2vec2 acoustic model. English only. (A multilingual MMS-FA variant is published separately.)

Converted from torchaudio.pipelines.WAV2VEC2_ASR_BASE_960H (Facebook/Meta wav2vec2-base fine-tuned on LibriSpeech 960h; Apache-2.0 lineage).

I/O contract


input	`waveform` float32 `[1, 960000]` — a fixed 60 s window @ 16 kHz mono
output	`emissions` float32 `[1, 2999, 29]` — log-probs; 20 ms / frame (stride 320)
dict	`base960h_dict.json` = `{label: index}`, blank `-`=0, word-sep `

Long audio is run in overlapping 60 s windows (hop 50 s) with center-crop, and a single CTC Viterbi over the concatenated emissions (no chunk-boundary drift).

Note: the 60 s window is deliberate — a 20 s window let group-norm skew on instrumental-heavy segments (the model hallucinated letters over intros). Align on a separated vocal stem for heavily-produced tracks.

Precision: FLOAT16. Deployment target: iOS 17.

Downloads last month: 3