ytm-lyrics-aligner

ONNX exports of mrfakename/w2v-bert-2.0-music-lyrics-ctc, a character-level CTC lyrics model for full music mixes, for forced alignment of known lyrics in ytm, a YouTube Music terminal player. ytm runs these files with ONNX Runtime.

Credits and licence

• Model: mrfakename/w2v-bert-2.0-music-lyrics-ctc (MIT), fine-tuned from mrfakename/w2v-bert-2.0-music, itself based on facebook/w2v-bert-2.0 (MIT) by Meta AI.
• These files are a format conversion only: same weights, no further training. They keep the MIT licence.
• Training data: the original model card describes the training procedure (weakly supervised CTC on whole-song lyrics) but does not document the training data. Nothing is known here beyond that card.

Files

File	Precision	Notes
aligner-fp32.onnx + aligner-fp32.onnx.data	fp32	Reference export; weights in the external data file.
aligner-fp16.onnx	fp16	Converted with onnxconverter_common.float16; casts at the graph edges, so inputs and outputs stay fp32. LogSoftmax runs in fp32.
aligner-int8.onnx	int8 weights	ONNX Runtime dynamic quantization: QInt8 weights for MatMul and the pointwise Conv; the depthwise (grouped) Conv stays fp32. For CPU.
vocab.json		{char: id}, 921 tokens. <blank> = 0 is the CTC blank, `
frontend-golden.json		A synthetic signal and its features from the source model's AutoFeatureExtractor, to test ports of the front end.
SHA256SUMS		sha256 of every file above.

Exported with opset 17 (IR version 8), torch 2.11.0+cu128, transformers 5.20.2, onnx 1.23.2, onnxruntime 1.26.0. Source revision: 1be91455793f86de803ac0707303a0998e423ce1.

Graph contract (all precisions)

	Name	Type	Shape
input	features	float32	[1, T_in, 160]: the SeamlessM4T feature extractor's output (80 log-mel bins, per-window normalised, stride-2 stacked)
input	attention_mask	int64	[1, T_in]: all ones for unpadded input
output	log_probs	float32	[1, T_in, 921]: log-softmax over the CTC head, one frame per 20 ms (50 Hz)

• The original's vocal-activity and F0 heads are not exported.
• Like the original, the model was trained on windows of up to 30 s. For longer audio, run 30 s windows with 2 s of context on each side, extract features per window, keep each window's centre 26 s and concatenate.
• Lyrics are matched after NFKC + lowercase, punctuation removed, apostrophes kept inside words, and Hangul decomposed into jamo. Put | between words and at both ends of the target.