SPRINGLab

/

Indic-Mio

like 28

Follow

SPRINGLab 132

This one cost me the most time and I think it will catch others, so it may be worth a
line on the card.

There are three MioCodec variants, and their content-token spaces are not
interchangeable, even though all three are FSQ with levels [8,8,8,5,5] = 12800
entries. Because the vocabulary size matches, every index is accepted by every codec
and nothing raises — you just get fluent, well-articulated speech saying completely
different words.

Comparing the content-tokenizer weights (conv_downsample, local_encoder,
local_quantizer) across variants:

And empirically, encoding one real Hindi clip with both and comparing indices:

exact index agreement, legacy vs 24kHz : 0.0000%   (chance = 0.0078%)

log-mel corr vs original:
   24kHz codec   <- its OWN tokens                     +0.938
   44.1k-legacy  <- its OWN tokens                     +0.940
   44.1k-legacy  <- 24kHz codec's tokens               +0.418     <- silent failure

What that looks like end to end (openai/whisper-small transcribing):

PROMPT   : नमस्ते, आप कैसे हैं? आज मौसम बहुत अच्छा है।
legacy   : अज़्ट उद आयार मुशिलो के लब शिबगो जो लिख चिए आया     <- wrong codec
24kHz    : नमस्ते आप कैसे है, आज मोशम बहुत अच्छा है            <- correct
v2       : नमस्ते आप कैसे है, आज मोसम बहुत अच्छा है            <- correct

PROMPT   : The mountains remember every footstep, even the ones you regret.
legacy   : "That's it for today. Thank you for watching."
24kHz    : "The mountains remember every footstep, even the ones you regret."

So 24kHz and 44.1kHz-v2 both work and are token-compatible; the legacy 44.1kHz
is a different token space entirely.

Two suggestions:

1. State on the card which variant to use. The card links MioCodec-25Hz-24kHz but
   claims 44 kHz output and writes at 44100; MioTTS-Inference defaults to
   MioCodec-25Hz-44.1kHz-v2. Naming -v2 explicitly, alongside the note that its
   tokenizer matches the 24 kHz model's, would remove the ambiguity.
2. Note the loader split: MioCodecModel for 24kHz / 44.1kHz-v2 (integrated
   iSTFT head), MioCodec for the legacy external-vocoder build. Loading the wrong one
   gives No vocoder weights found with prefix 'vocoder.', which is a correct refusal
   but reads as a broken download.