kale-tts-1-base

kale-tts-1-base

kale-tts-1-base is Kokoro-82M with its text side and its timing-and-melody planner fine-tuned on our 10 voices designed with Maya-1; its sound decoder is stock Kokoro-82M. It's the teacher of kale-tts-1-tiny and kale-tts-1-nano: both learned their timing and melody from it and have their own small sound generators, so they run on a low-end tablet. kale-tts-1-base itself is too big and slow for that; use it on a desktop or GPU, or to train and distill your own small voices.

Distilled into

model parameters CPU file (int8) WebGPU file (float32) Galaxy Tab S10 Lite unclear words vs Maya-1
kale-tts-1-tiny 45.7M 79.9 MB 183.2 MB 24x real time +1.4 points
kale-tts-1-nano 7.5M 14.8 MB 30.6 MB 44x real time +1.6 points

tiny keeps this model's timing-and-melody planner (frozen) and adds the Ettin word reader; nano is a smaller student distilled from it. Both have their own small sound generator and run on a low-end tablet.

measure value
parameters 81.8M (28.5M fine-tuned on our voices: text side + planner; the rest is stock Kokoro-82M's sound decoder)
size 327 MB (kale-tts-1-base.pth, PyTorch)
voices 10, as voice packs in voices/
trained on kale-tts-1-data: 1,300 takes per voice, 26.9 hours
clarity and melody (our release bench, 10 voices, 18 Nov) unclear words +3.9 points vs Maya-1's own recordings (Maya-1: 5.82 %; tiny +1.4, nano +1.6); melody gap 12.2 (tiny 7.48, nano 7.46). Not built for the tablet

Voices

name voice pronunciation
Kenzie esports hype woman US English
Simone deep and confident woman US English
Amy crisp news-style woman British English
Frankie hard-boiled detective's partner US English
Darlene rough low-voiced old trucker woman US English
Owen bright young host US English
Toby mischievous prankster US English
Jack crisp news-style man British English
Earl African American jazz granddad US English
Magnus deep dramatic trailer narrator US English

Every voice was designed from a one-line text description; none is a real person.

Use it

pip install kokoro soundfile
pip install -U huggingface_hub
hf download KaleComputer/kale-tts-1-base --local-dir kale-tts-1-base
cd kale-tts-1-base

Save as speak.py in that folder, then run python speak.py:

import json, torch, soundfile as sf
from kokoro import KModel, KPipeline

model = KModel(config="config.json", model="kale-tts-1-base.pth").eval()
voices = {v["name"]: v for v in json.load(open("voices.json"))}

# American voices use lang_code "a", the two British ones (Amy, Jack) "b"
v = voices["Kenzie"]
pipe = KPipeline(lang_code=v["lang_code"], model=model)
pack = torch.load(v["file"], weights_only=True)

audio = torch.cat([a for _, _, a in pipe("Hello there! A volcano is a place where very hot, melted rock comes up out of the ground.", voice=pack)])
sf.write("hello.wav", audio.numpy(), 24000)

KPipeline turns text into sounds with Kokoro's own text front end (misaki). Our checked dictionary and text rules are in the kale-tts-1-tiny and kale-tts-1-nano repos (frontend/).

How it was made

How we picked the base model, at selection

  1. Maya-1 designs each of the 10 voices from a voice prompt and reads thousands of sentences with it; a filter keeps the clean takes (kale-tts-1-data).
  2. We trained our own aligner on 1,300 takes per voice (26.9 hours, 30 epochs on an NVIDIA L40S), then fine-tuned Kokoro-82M on them in two stages on an NVIDIA H100 (three epochs, then one more with the adversarial stage).
  3. The voice packs come from the fine-tuned style encoder, one per voice.
  4. kale-tts-1-tiny keeps this planner (frozen) and adds the Ettin word reader; kale-tts-1-nano is a smaller student distilled from it. Both learn their sound straight from Maya-1's recordings.

Limits

  • English only. The two British voices read best with lang_code="b".
  • Too big and slow for a low-end tablet: that's what kale-tts-1-tiny and kale-tts-1-nano are for.
  • Not for impersonating real people or passing the output off as human speech.

Licence and credits

Apache-2.0.

AI-generated speech: everything this model says is synthetic speech made by an AI model, in voices that belong to no real person. Tell listeners it is AI-generated, never present it as a human recording, and never use it to imitate a real person.

Credits: based on Maya-1: all training audio is original speech from Maya-1 (Maya Research, Apache-2.0), decoded with SNAC (MIT). Fine-tuned from Kokoro-82M (hexgrad, Apache-2.0), whose sound decoder this model keeps unchanged, with StyleTTS2 (MIT) and kikiri-tts (Apache-2.0) code. Training texts: public-domain books made available by Project Gutenberg. Dataset: kale-tts-1-data (CC BY 4.0).

Downloads last month
50
Inference Providers NEW
This model isn't deployed by any Inference Provider. 🙋 Ask for provider support

Model tree for KaleComputer/kale-tts-1-base

Finetuned
(88)
this model
Finetunes
2 models

Dataset used to train KaleComputer/kale-tts-1-base

Collection including KaleComputer/kale-tts-1-base