pocket-tts Japanese — phase 1 validation run
This is not a finished model. It is the artifact of a pipeline validation run, kept so the numbers behind it can be checked. Read the whole of this file before using anything here.
What it is
A 24-layer pocket-tts model,
continued from the released English 24L weights with the text embedding
reinitialised, finetuned for 15,000 steps on 85.5 hours of Japanese speech from
27 speakers of ayousanz/moe-speech-plus.
Its purpose was to answer three questions before spending real money on a production run:
| question | answer |
|---|---|
| Does the data pipeline produce trainable data? | yes — 50,164 utterances aligned, none skipped, 95.6% carry a usable cut point |
| Does the Japanese tokenizer / morphological segmentation / forced alignment chain work? | yes — 0 unknown tokens, 0.45% of words without a timestamp |
| Does the model acquire Japanese phonology at all? | yes — confirmed by ear on the samples in samples/ |
It cost about $4 on one rented GPU. That was the point: to refute a broken pipeline cheaply rather than discover it after a $145 run.
What it is not
- Not production quality. Production is 250k steps on ~2,640 hours. This is 15k steps on 85.5.
- Not distilled. Still 24 layers, not the 6-layer student the released models ship.
- Narrow. 27 speakers of anime and visual-novel character acting. Neutral read-aloud prosody is outside what it saw.
- Trained on ASR transcripts. Clips were kept where two independent ASR systems disagreed on up to 20% of characters (median 15.4%), because no manual transcription exists. Some training pairs therefore have text that does not match the audio.
The result worth knowing
Validation loss reached its minimum at step 7,500 (0.1018) and then rose monotonically to 0.2123 at step 15,000, while training loss kept falling. That is 19.5 epochs over 49,200 utterances from 27 speakers.
But the samples get better to step 15,000 by ear, not worse. Both are true
and they measure different things: train.py takes the sample voice prompt from
a training batch, so the samples show quality on a seen voice, while the
validation set is a single held-out speaker. The model kept improving on the
voices it saw while getting worse on the one it did not.
For a model whose purpose is voice cloning, the held-out number is the one that matters — and one speaker is far too thin to conclude from. A larger validation set with several speakers is the concrete requirement this run produced for the next one.
Files
model.safetensors |
final weights, step 15,000 |
checkpoint_000{10000,12500,15000}.pt |
training states. Step 7,500 — the validation minimum — was already deleted by num_ckpt_keep: 3 |
optim_00015000.pt |
optimizer state |
samples/ |
90 wavs, three sentences every 500 steps |
progress.jsonl, args.yaml, logs/ |
the loss curve and the exact resolved config |
probe.json |
the corpus measurement the cutoffs were read off: mutual-CER and speechMOS distributions over 74,612 clips |
Data and licence
Weights are CC-BY-4.0, inheriting from the kyutai release they continue from.
The training audio comes from ayousanz/moe-speech-plus, which is licensed for
information-analysis use only and may not be redistributed. No audio from it
is in this repository, and the samples here are synthesised by the model rather
than taken from the corpus. Publishing weights trained on it is permitted;
republishing the corpus is not.
- Downloads last month
- -