saryps-labs/pocket-tts-hindi

Pocket TTS Hindi

v0.1 — early research release

Pocket TTS Hindi is a compact, streaming Hindi text-to-speech model with prompt-based voice cloning. It uses the six-layer Pocket TTS runtime architecture and contains 109.5M parameters. The release weights use BF16 storage and occupy 219 MB.

This model was produced in two stages: a 24-layer Hindi teacher was trained from scratch, then distilled into a six-layer student while classifier-free guidance was baked into a single inference pass.

Highlights

• Hindi speech synthesis and voice cloning
• 109,502,146 parameters
• 219 MB BF16 weight file
• streaming generation
• faster than real-time on the tested x86 CPU
• optional dynamic-int8 CPU execution
• 24 kHz mono output

This is a public research checkpoint, not a production safety-certified service.

Quick start

The current release requires a Pocket TTS checkout containing support for append_terminal_punctuation: false in custom configs. From that environment:

uv run pocket-tts generate \
  --config hf://saryps-labs/pocket-tts-hindi/config.yaml \
  --voice ./your_hindi_voice.wav \
  --text "आज का दिन बहुत अच्छा है" \
  --output-path hindi_output.wav

Dynamic int8 CPU inference:

uv run pocket-tts generate \
  --config hf://saryps-labs/pocket-tts-hindi/config.yaml \
  --voice ./your_hindi_voice.wav \
  --text "आज का दिन बहुत अच्छा है" \
  --output-path hindi_output_int8.wav \
  --quantize

Use a clear voice prompt from a speaker who has consented to cloning. The current model was trained without meaningful punctuation coverage, so plain Hindi text without punctuation is recommended.

Model architecture

Component	Configuration
FlowLM transformer	6 layers, width 1024, 16 heads
Flow head	depth 6, width 512
Audio codec	Mimi, 12.5 latent frames/s
Output sample rate	24 kHz
Tokenizer	Hindi SentencePiece, 4,000 tokens
Total parameters	109,502,146
Storage dtype	BF16
Weight size	219,029,196 bytes

The student was distilled for 200,000 optimizer steps from a 24-layer teacher trained for 400,000 steps. CFG coefficient 2.0 was distilled into the student, which runs with one conditioned backbone pass at inference.

Training data

The training manifest contains 469,795 utterances and 932.11 hours:

Dataset	Utterances	Hours
AI4Bharat IndicVoices Hindi	381,496	803.31
AI4Bharat Kathbath Hindi	88,299	128.81

Separate pools contain 13.49 validation hours and 11.37 speaker-disjoint evaluation hours. Audio/text pairs were filtered, normalized, and aligned with a Hindi wav2vec2 CTC aligner. Dataset licenses and attribution are documented in NOTICE.md.

Evaluation

A deterministic 100-item cross-sentence protocol used different prompt and target utterances from the same held-out speaker:

• 50 IndicVoices and 50 Kathbath cases
• 21 held-out speakers
• temperature 0.3
• one flow decode step
• EOS threshold -1.0
• seed 20260901

Intelligibility and reliability

Metric	Generated	Genuine held-out audio
Whisper large-v3 WER	23.28%	26.74%
Whisper large-v3 CER	11.54%	13.89%
WavLM speaker similarity	0.9305	0.9292
UTMOS	3.1234	2.5469

• silent outputs: 0/100
• maximum-length/no-EOS failures: 1/100

Automatic metrics are evaluator-dependent. Better ASR scores than genuine references can reflect cleaner or more canonical synthesized speech and do not prove perfect human-perceived quality.

CPU benchmark

AMD EPYC 7V13 Azure VM, one PyTorch thread, three Hindi sentences, three measured repetitions after warm-up:

Metric	FP32 runtime	Dynamic int8
Median speed	3.39× real-time	3.74× real-time
Median time to first audio	110 ms	72 ms
Peak process RSS	1.32 GiB	1.20 GiB

Int8 preserved intelligibility on the same 100-item protocol (10.66% CER and 22.32% WER), with the same one no-EOS case and zero silent generations. Hardware results should not be generalized to every edge device.

Sample generations

All samples are synthetic outputs from held-out voice prompts. Prompt audio is not redistributed in this release candidate.

Voice	Sentence	Audio
Female-labelled 1	आज सुबह मौसम बहुत सुहावना था और बच्चे मैदान में खेल रहे थे	Listen
Female-labelled 2	भारत में अनेक भाषाएँ बोली जाती हैं और हर भाषा की अपनी सुंदर परंपरा है	Listen
Female-labelled 3	आज सुबह मौसम बहुत सुहावना था और बच्चे मैदान में खेल रहे थे	Listen
Male-labelled 1	भारत में अनेक भाषाएँ बोली जाती हैं और हर भाषा की अपनी सुंदर परंपरा है	Listen
Male-labelled 2	आज सुबह मौसम बहुत सुहावना था और बच्चे मैदान में खेल रहे थे	Listen
Male-labelled 3	भारत में अनेक भाषाएँ बोली जाती हैं और हर भाषा की अपनी सुंदर परंपरा है	Listen

Limitations

• Hindi only; code-switching and English words were not systematically tested.
• Punctuation was effectively absent from training. The release config disables automatic terminal punctuation, and punctuation-controlled prosody is not supported reliably.
• One of 100 held-out evaluation cases failed to emit EOS before the maximum generation length.
• Some prompts can transfer recording noise into generated speech. Female samples occasionally exhibited reduced crispness during informal listening.
• Numbers, abbreviations, rare names, dialects, accents, and long-form text need broader human evaluation.
• WavLM, Whisper, and UTMOS scores are model-dependent proxies.
• Consumer-laptop and low-power edge benchmarks are pending.

Intended use and safety

Intended for Hindi TTS research, local/offline applications, accessibility experiments, and evaluation of compact speech synthesis.

Only clone voices with informed permission. Clearly disclose synthetic speech. Do not use this model for impersonation, fraud, harassment, deceptive media, or rights violations. Users are responsible for complying with applicable law, dataset terms, and voice/publicity rights.

Licenses and attribution

This release uses license: other to avoid collapsing the distinct licenses and attribution requirements of its components into an inaccurate single label. Pocket TTS code is MIT; training datasets are reported as CC BY 4.0. See NOTICE.md for component-level attribution and review notes.

Collaboration and deployment

We are open to research and commercial collaborations involving deployment, device-specific optimization, and compact custom TTS training for particular voices, domains, or languages. If you are interested in using this model in a product, deploying it on-device, or developing a custom model, contact Saryps Labs through this model's Community tab.

Acknowledgements

Built with Kyutai Pocket TTS. Training speech was provided by AI4Bharat IndicVoices and Kathbath. We thank Kyutai and AI4Bharat for making their work available to the research community.