tiny-voca-ja
ćĽćŹčŞčŚç´: tiny-voca-ja ăŻăčąčŞ TTS ă˘ă㍠tiny-vocaďźn4ze3mďźăćĽćŹčŞă§ ăźăăăĺŚçżăç´ăăć´žçă˘ăăŤă§ăăăăŠăĄăźăż 4.5Mă声㯠F2ďźĺĽłć§ďźă¨ M2ďźçˇć§ďźăŽ 2 声ăĺşĺ㯠24 kHz ă˘ăăŠăŤăĺŚçżăăźăżăŻ Qwen3-TTS ă§çćăăĺćéłĺٰ 16,321 ăŻăŞăăďź16.6 ćéďźă§ăĺŽĺ¨ăŽčŠąč ăŽé˛éłăŻĺŤăžăăžăăăCPU ă§ĺŽćéăăéă ĺăăžăăćŞĺŚçż16ćăŽćŁĺźčŠäžĄă§ăŻ CER ĺšłĺ 6.6%ďźć°ĺăťĺä˝ăŽčĄ¨č¨ăčŞăżăŤăăăăćŁčŚĺĺžăŻ 1.6%ďźă§ăăăćĽäťăťć°ĺăŽčŞăżčޤăăŽä¸ťĺ ăŻă˘ăăŤă§ăŻăŞăăčŞăżĺŚçďźăăăłăă¨ăłăďźĺ´ăŤăăăžăăă2026ĺš´11ć20ćĽăŤăčŞăżĺŚçă pyopenjtalk-plus ăŤăăăăŚĺ注éăăăăźăżă§čż˝ĺ ĺŚçżăă v2ďź114,000ăšăăăďźăĺ Źéăăžăăăĺă16ćčŠäžĄă§ăćŁčŚĺĺžCER㯠1.6% ăă 1.0% ăŤćšĺăăŚăăžăă
tiny-voca-ja is a Japanese text-to-speech model with two voices, F2 (female) and M2 (male). It is a from-scratch retrain of tiny-voca â the same architecture and recipe by Muhammed Nazeem â on Japanese data. It is not a fine-tune of the English weights: no pretrained weights are part of the model. The complete model, including the waveform generator, has 4.5M parameters and was trained on about 16.6 hours of synthetic speech. It runs faster than real time on CPU. A fine-tuned v2 (step 114,000, 2026-11-20) is the current checkpoint in this repository; see Fine-tune v2 below. The v1 checkpoint remains available in the revision history.
- Code (Japanese recipe): https://github.com/gachimoto-code/tiny-voca-ja
- Original project: https://github.com/n4ze3m/tiny-voca
- Dataset: https://huggingface.co/datasets/gachimoto/tiny-voca-ja
| Parameters | 4,538,827 |
| Voices | F2, M2 (Japanese) |
| Output | 24 kHz mono |
| Input | Japanese text, converted to phonemes with eSpeak NG (ja voice); kanji readings resolved with pykakasi |
| Pretrained weights in the model | None |
| License | Apache-2.0 |
The parameter count is 1,536 higher than the English model (4,537,291) because the phoneme inventory grew from 76 to 84 symbols (8 Japanese additions).
Files
| File | Size | Contents |
|---|---|---|
tiny-voca-ja.pt |
18.2 MB | PyTorch weights, float32 (inference checkpoint, format 4; v2, step 114,000) |
onnx/model.onnx |
21.4 MB | ONNX export, float32, opset 17 |
config.json |
Model settings, phoneme vocabulary, voices, front-end identity | |
samples/ |
The audio samples on this page, synthesized by this model | |
metrics.jsonl |
11.0 MB | Full training log of the v1 run |
metrics-finetune-v2.jsonl |
1.1 MB | Training log of the v2 fine-tune run |
SHA256:
tiny-voca-ja.pt:bace068fa0c0a52974df6abe33b650f7b1eeed97ff714f3bab559c14d9b96476onnx/model.onnx:105ac483371607ff25b9ac694d40b1f6d8d979d838e5848ff4bd366ee14bb0c9
The v1 ONNX export was verified against the PyTorch model on a Japanese sentence for both voices: identical durations, and a waveform difference of at most 0.16% of the peak amplitude (the noise signal is drawn by each runtime, as in the English model). For v2, the web-app ONNX export of the same checkpoint (same weights, a different graph wrapper) was verified against PyTorch synthesis with correlation 1.000000 and a maximum waveform difference of 0.0005.
Samples
Synthesized on a CPU-only Linux machine (no GPU). CER is the character error rate of a Whisper small (multilingual) transcription against the input text, computed in a kana-normalized space so that kanji spelling variants are not counted as errors. See Evaluation for the caveats â this is four sentences, not a benchmark.
| # | Voice | Text | CER |
|---|---|---|---|
| 1 | F2 | ăăăŤăĄăŻăăăăŻćĽćŹčŞăŽéłĺٰĺćăŽăăšăă§ăă | 0.0% |
| 2 | F2 | ăŻăăăžăăŚăç§ăŻćĽćŹčŞă§čŠąăăĺ°ăăŞéłĺٰĺćă˘ăăŤă§ăă | 0.0% |
| 3 | M2 | äťćĽăŻĺä¸ćĺäšćĽăć¨ććĽă§ăăćąäşŹăŽĺ¤Šć°ăŻăć´ăăŽăĄćăă§ăăăă | 0.0% |
| 4 | F2 | 人塼çĽč˝ăăćĽćŹčŞăŽćçŤ ăčŞăżä¸ăăžăă声ăŽĺ¤§ăăăăéăăăčŞçąăŤĺ¤ăăăăžăă | 0.0% |
Sample 3 was scored 29.2% in the original spot check; that figure later turned out to be mostly a scoring artifact, and the same audio scores 0.0% once numeral notation is normalized on both sides â see the correction in Evaluation.
Usage
v2 checkpoint note. The current checkpoint (v2, step 114,000) was
fine-tuned for the pyopenjtalk-plus front end (front-end identity oj-1),
so the stock CLI below â which targets the v1 eSpeak front end â will
refuse it as a front-end mismatch. Use the OJ adapter in the repository
(frontend-swap/oj_frontend.py), or the web app
tiny-voca-ja-web,
whose phonemizer is matched to v2. The v1 checkpoint in the revision
history works with the CLI as written.
The Japanese front end uses eSpeak NG with the ja voice, so the checkpoint
only speaks Japanese text. You need the espeak-ng system package and the
code from the Japanese recipe repository.
sudo apt install espeak-ng # or your platform's equivalent
pip install git+https://github.com/gachimoto-code/tiny-voca-ja
hf download gachimoto/tiny-voca-ja --local-dir tiny-voca-ja
tiny-voca-speak \
--checkpoint tiny-voca-ja/tiny-voca-ja.pt \
--voice F2 \
--text "ăăăŤăĄăŻăăăăŻćĽćŹčŞăŽéłĺٰĺćăŽăăšăă§ăă" \
--output hello.wav
(python -m tiny_voca.infer with the same arguments works too.)
| Option | Default | Meaning |
|---|---|---|
--text-file |
Read the text from a file instead of --text. |
|
--voice |
F2 |
F2 or M2 |
--speed |
1.0 | Speaking rate |
--pitch-shift |
0 | Added to log F0. 0.058 is about one semitone. |
--pitch-range |
1.0 | Scales pitch movement around the utterance mean. |
Text of any length works. The command reads it one sentence at a time and
joins the audio. The checkpoint records the front end it was trained with;
inference refuses to run with a mismatched front end unless
--allow-frontend-mismatch is given.
Synthesis measured on a small shared CPU-only VM: 2.8â6.3 seconds per sentence including model load, for 4.4â8.1 seconds of audio â faster than real time, but this VM is far from a benchmark machine.
ONNX
The graph takes phoneme ids and stress values and returns the waveform.
Text is converted to ids outside the graph with the Japanese front end
(eSpeak NG ja + pykakasi); the ids are the positions in vocab in
config.json. Inputs and outputs are the same as the English model:
inputs tokens, stress, voice (0 = F2, 1 = M2), speed, pitch_shift,
pitch_range; outputs audio (24 kHz float32) and durations. The model
reads one utterance per call, up to 512 tokens and 32 seconds. The audio can
peak a little above 1.0, so scale it down before saving 16-bit samples.
Model
Text is converted to phonemes with a stress value for each. A transformer encoder (4 blocks) reads the phonemes, a duration predictor gives each one a length in frames, and two frame-level transformers (2 and 5 blocks) predict pitch, voicing and energy and then the parameters of the waveform generator. All blocks use 192 channels, 4 heads and rotary position embeddings.
The waveform is produced by a source-filter synthesizer instead of a neural vocoder. A band-limited pulse train at the predicted pitch and a noise signal are each shaped by a time-varying filter, whose cepstrum the network predicts once per frame (256 samples, 10.7 ms). This part has 25,862 parameters.
The Japanese model uses the same components as the English one; only the phoneme embedding grows with the 84-symbol inventory.
Training
| Data | gachimoto/tiny-voca-ja: 16,321 clips, 16.6 hours, two voices; 11,561 clips (11.5 hours) passed annotation and were used for training (10,884 train / 332 validation / 345 test) |
| Steps | 104,000 |
| Hardware | One RTX 4090 (rented, RunPod), about 5.3 hours |
| Final loss | 10.717 (base loss 6.494) at step 104,000 |
| Losses | Multi-resolution spectral losses; adversarial loss with multi-period and multi-resolution discriminators after step 8,000 (gan-ratio 0.7) |
The training audio is synthetic. Both voices were created from text descriptions with Qwen3-TTS VoiceDesign, and the recordings were generated with Qwen3-TTS 1.7B Base. No recording of a real speaker is involved.
Phoneme durations come from forced alignment with
prj-beatrice/japanese-hubert-base-phoneme-ctc-v2, with labels from
pyopenjtalk-plus. This differs from the English recipe on purpose: the
eSpeak-based aligner used there (facebook/wav2vec2-lv-60-espeak-cv-ft)
accepted 0 of 150 Japanese smoke-test clips â it has not learned Japanese
acoustics â so a Japanese phoneme CTC model was used instead. 11,561 of the
16,321 clips (70.8%) passed the alignment checks.
Fine-tune v2 (2026-11-20)
The v1 checkpoint was fine-tuned for +10,000 steps (114,000 total) on the
same synthetic corpus, re-annotated end to end: pyopenjtalk-plus labels
with forced alignment by the beatrice phoneme recognizer, mapped to the
84-symbol inventory. 15,558 of the 16,321 clips passed the checks
(14,652 train / 453 validation / 453 test). The front-end identity of the
checkpoint therefore changed from eSpeak (v3) to oj-1
(pyopenjtalk-plus, g2p_prosody mode).
One honest caveat: because of a learning-rate schedule quirk (the decay term used the absolute step), the effective learning rate was about 2e-6 â roughly 1/40 of the intended 1e-4. The improvement below was obtained anyway, at about $2 of rented GPU time.
Samples from the v2 evaluation set (the 16-sentence formal evaluation, matched front end):
- ăććĽăŽĺĺžăé§ ĺăŽćŹĺąă§ĺäşşă¨ĺž ăĄĺăăăăăŚăăžăăă
- ăăăŽéťčťăŻăĺĺ¤čĄăăŽçšćĽă§ăăăăăă¨ăćŽéĺčťă§ăăăă â v1 misread çšćĽ here; v2 reads it correctly.
- ăäşç´ăŻ2027ĺš´2ć25ćĽăŽĺĺ10ć25ĺçşăŽäžżă§ăăă
Evaluation
Character error rate of synthesized speech, compared in a kana-normalized space so that kanji spelling variants are not counted as errors. Two checks have been run, both synthesized locally on CPU: a 4-sentence sanity check transcribed with Whisper small (multilingual), and a formal evaluation of 16 unseen sentences transcribed with Whisper large-v3:
| Sentences | Clips | CER |
|---|---|---|
| New sentences, not in the dataset | 4 | 7.3% mean (0.0 / 0.0 / 29.2 / 0.0) with the original kana-only scoring; 0.0% mean after numeral normalization |
| Formal evaluation, 16 unseen sentences (Whisper large-v3) | 16 | 6.6% mean raw, 1.6% after numeral/unit normalization |
| Formal evaluation, 16 unseen sentences, v2 fine-tune with the matched pyopenjtalk-plus front end (Whisper large-v3) | 16 | 6.0% mean raw, 1.0% after numeral/unit normalization; UTMOS 3.73 (v1 with the same front end: 1.6% normalized, UTMOS 3.67) |
The 4-sentence row is a sanity check, not a benchmark. The formal evaluation has since been run (2026-11-19): 16 unseen sentences covering dates in both numeral styles, times, money, counters, place and personal names, a phone number, decimals, homophones and one long sentence, transcribed with Whisper large-v3. The stock front end scored 6.6% mean CER raw and 1.6% after normalizing numeral and unit notation on both sides; 13 of the 16 sentences scored 0.0% normalized. No listening test has been run.
Correction (2026-11-19). Sample 3's 29.2% was mostly a measurement artifact: the ASR writes dates as digits ("11ć19ćĽ") while the reference is read as kanji numerals, and the kana scorer drops digits, so matching sounds were counted as errors on one side only. Normalized, that sentence scores 0.0% â the audio was correct. The genuine reading errors were in Arabic-numeral date sentences (ăäťćĽăŻă read as the greeting ăăăăŤăĄăŻă, a wrong weekday); an experiment in the GitHub repository (experiments/frontend-swap) shows they come from the front end, not the model: with pyopenjtalk-plus readings the same checkpoint scores 0.0% on that sentence, and 1.6% mean normalized CER over the 16-sentence set.
Limitations
- Japanese only, two fixed voices, no voice cloning.
- Numbers and dates depend on the front end. The stock front end misreads some Arabic-numeral date sentences (ăäťćĽăŻă read as the greeting ăăăăŤăĄăŻă, weekday mix-ups). An earlier version of this card also described the kanji-numeral date sample as a reading failure (CER 29.2%); that was a scoring artifact, corrected in Evaluation above. Writing numbers out in kana before synthesis avoids most of the genuine cases.
- Pitch accent is not modeled explicitly; the model only sees what the eSpeak front end provides, so accent can sound flat or foreign.
- The voice is clear but can sound slightly synthetic.
- The same text is always read the same way. There is no sampling of prosody.
- Foreign names and unusual kanji readings depend on the front end and are often misread.
- No streaming.
- The training data is itself synthetic, so the model cannot be more natural than the model that generated it.
Cost
About $20 of rented GPU time in total (estimated from pod hours Ă the $0.89/h rate; not reconciled against the RunPod billing screen). Roughly: $9.18 for dataset generation, $4.69 for the final training run, the rest for voice design, environment setup and the failed annotation attempts. The English original cost $9.56; the Japanese version cost about twice that, mainly because of the aligner replacement and the higher per-hour rate of the GPU that was actually available.
License
The weights and code are released under Apache-2.0.
tiny-voca-ja is a derivative work of tiny-voca by Muhammed Nazeem, used under Apache-2.0. The architecture, training recipe and code base are his; the Japanese front end, corpus, annotation pipeline and training run are this project's. If you use this model, please credit both.
Citation
@software{fujimoto2026tinyvocaja,
author = {Fujimoto, Satoshi},
title = {tiny-voca-ja: a 4.5M-parameter two-voice Japanese text-to-speech model},
year = {2026},
url = {https://huggingface.co/gachimoto/tiny-voca-ja}
}
Original model:
@software{nazeem2026tinyvoca,
author = {Nazeem, Muhammed},
title = {tiny-voca: a 4.5M-parameter two-voice English text-to-speech model},
year = {2026},
url = {https://huggingface.co/n4ze3m/tiny-voca}
}
Author: gachimoto (Satoshi Fujimoto).
Acknowledgements
The design builds on FastSpeech 2 (explicit duration and prosody prediction), DDSP and the neural homomorphic vocoder (source-filter synthesis with predicted filters), and HiFi-GAN and UnivNet (discriminators). No code or weights from these projects are used. The Japanese alignment uses prj-beatrice/japanese-hubert-base-phoneme-ctc (Apache-2.0) and pyopenjtalk-plus.
- Downloads last month
- 18
Model tree for gachimoto/tiny-voca-ja
Base model
n4ze3m/tiny-voca