sanoTTS (28 tiny voices)

Ampixa's sanoTTS: tiny distilled TTS voices, 0.29M to 2.27M parameters, in 14 languages, exported for loom.cpp. Two lines: piperlite (distilled from Piper voices, 22.05 kHz) and nano (distilled from Kokoro's af_heart through a Vocos-style decoder, 24 kHz).

This is a loom.cpp export: a single self-describing GGUF that carries its own graph topologies, tokenizer (if any) and driver script, produced by loom-exporter.

Original model

Exported from ampixa/sanoTTS. Each file is one upstream voice, its parameters in loom.cpp's GGUF format (see Files).

License

gpl-3.0, inherited from the base model above.

Language(s)

en, ar, cs, de, es, fr, id, it, pt, ro, ru, tr, vi, zh

Usage

Run it with loom-py -- loom-py-rt on PyPI:

pip install -U "loom-py-rt[hub,phonemes]"

Needs loom-py-rt 1.0.0rc18 or later. This file uses the engine's ISTFT op, new in that release; an older runtime stops with unknown op 'ISTFT'.

NOTE: This is a work in progress. For now, in order to avoid license conflicts and keep dependencies at a minimum, we opted for orthography2ipa as our "swiss-knife" phonemizer. For deep-orthography languages like English, to get stressing rules and context-based phonemization, the phonemizer must register a reference lexicon (e.g., ipa-dict) to get highly accurate phonemization. Full quality can be achieved by phonemizing the text yourself using your engine of choice and feeding it to the model via the argument phonemes as in the example below.

import loom

# This repo holds many voices, each its own model; name the file you want (see "Files" below).
model = loom.Model.from_pretrained("loom-ai-org/sanotts-loom", "sanotts-heartnano.gguf")

# The built-in text door: orthography2ipa, plus a pronunciation lexicon for English that ships in this
# repo (hexgrad/misaki's, Apache-2.0). The voice declares the IPA conventions it was trained in and the
# door rewrites the G2P's output to them before encoding.
loom.phonemizers.set_lexicon("https://huggingface.co/loom-ai-org/sanotts-loom/resolve/main/lexicons/en_US-misaki.tsv", language="en-US")
audio = model.text2speech.infer("hello world")
audio.save("hello.wav")

# Full quality: phonemes from your own G2P, in the voice's own conventions (espeak's for the piperlite
# voices, misaki's for heart and heart-nano), are encoded as given.
# audio = model.text2speech.infer(phonemes="...")

The layer underneath

The call above is the high-level door: one per task, named for the modality pair it maps between, with the windowing, sampling and assembly this model needs already applied. Under it, model.infer(...) passes your arguments straight to the driver this GGUF embeds -- which is where you go for a knob the door does not name.

model.driver_source prints that driver, including a header comment documenting every argument it accepts for this model, and is the authority on it. See loom-py for the API and loom.cpp for what the engine does between the two.

Known limitations

GPL-3.0, like upstream's weights and code: these files carry upstream's parameters. loom.cpp, loom-exporter and loom-py contain no GPL code; the export imports upstream's training scripts from a pinned clone rather than vendoring them.

Small, but not real time on the smallest boards. On a Raspberry Pi Zero W (one ARMv6 core at 1 GHz, no SIMD), synthesis takes 1.1 times the audio's duration with heart-nano (34 MB of memory), 2.3 times with the 0.51M piperlite voices, 2.5 times with heart and 8.4 times with amy-1.46M. On a 2-core x86 laptop CPU, amy-1.46M takes a fourteenth of it.

The text door is measured in English only. Through text2speech.infer(text) with the bundled lexicon, heart-nano scored 11.9% word error rate on 30 LibriSpeech sentences (Whisper) against 8.1% with upstream's own front end, and amy-1.46M 12.3% against 8.5%. Without the lexicon, English is far worse. The other languages use orthography2ipa's rules with only the language-independent convention fixes; for full quality there, pass phonemes= from an espeak-style G2P. Chinese has no text door: orthography2ipa's output for it lands almost entirely outside the zh voice's phoneme table (a 13-character sentence encoded to 3 ids), so pass that voice espeak-style pinyin phonemes yourself.

heart and heart-nano take at most 207 phoneme symbols per call (upstream's own limit); split longer text at its sentences. Their decoder is noise-fed: seed changes the voice slightly, and the default reproduces upstream's default rendering. Upstream's optional "sibilant injection" (applied by its dashboard and Arduino runtime, not by its pip package) is not reproduced.

Files

Several voices; each file is its own model, exported from its own upstream package, with its own graph and driver.

sanotts-heartnano.gguf -- heart-nano: the nano line's smallest voice (Kokoro af_heart's student, int8 upstream, dequantised here). The default above: on a Raspberry Pi Zero W it synthesises at 1.1x real time in 34 MB.
sanotts-heart.gguf -- heart: the nano line at 2.27M (Kokoro af_heart's student). 2.5x real time on a Pi Zero W, 42 MB.
sanotts-amy-en-1p46m.gguf -- amy (Piper en_US-amy-medium's student), 1.46M: upstream's best-scored English voice (SCOREQ 4.13). 8.4x real time on a Pi Zero W.
sanotts-amy-en-1p1m.gguf -- amy, 1.08M.
sanotts-amy-en-1p8m.gguf -- amy, 1.83M.
sanotts-hfc-en-1p8m.gguf -- hfc (Piper en_US-hfc_female-medium), 1.83M.
sanotts-kristin-en-1p4m.gguf -- kristin (Piper en_US-kristin-medium), 1.40M, with upstream's waveform post-filter.
sanotts-ar-kareem-1p57m.gguf -- Arabic, kareem (Piper ar_JO-kareem-medium), 1.57M.
sanotts-cs-jirka-0p51m.gguf -- Czech, jirka, 0.51M.
sanotts-cs-jirka-1p57m.gguf -- Czech, jirka, 1.57M.
sanotts-de-thorsten-0p51m.gguf -- German, thorsten, 0.51M: 2.3x real time on a Pi Zero W, 54 MB.
sanotts-de-thorsten-1p57m.gguf -- German, thorsten, 1.57M.
sanotts-es-davefx-0p51m.gguf -- Spanish, davefx, 0.51M.
sanotts-es-davefx-1p56m.gguf -- Spanish, davefx, 1.56M.
sanotts-fr-siwis-1p57m.gguf -- French, siwis, 1.57M.
sanotts-id-newstts-1p46m.gguf -- Indonesian, news_tts, 1.56M.
sanotts-it-serena-0p51m.gguf -- Italian, serena, 0.51M.
sanotts-it-serena-1p57m.gguf -- Italian, serena, 1.57M.
sanotts-pt-cadu-0p51m.gguf -- Portuguese (Brazil), cadu, 0.51M.
sanotts-pt-cadu-1p57m.gguf -- Portuguese (Brazil), cadu, 1.57M.
sanotts-ro-mihai-0p51m.gguf -- Romanian, mihai, 0.51M.
sanotts-ro-mihai-1p57m.gguf -- Romanian, mihai, 1.57M.
sanotts-ru-irina-0p51m.gguf -- Russian, irina, 0.51M.
sanotts-ru-irina-1p57m.gguf -- Russian, irina, 1.57M.
sanotts-tr-dfki-0p51m.gguf -- Turkish, dfki, 0.51M.
sanotts-tr-dfki-1p56m.gguf -- Turkish, dfki, 1.56M.
sanotts-vi-vais1000-1p46m.gguf -- Vietnamese, vais1000, 1.57M.
sanotts-zh-xiaoya-1p55m.gguf -- Chinese, xiao_ya, 1.55M.
lexicons/en_US-misaki.tsv -- hexgrad/misaki's US English pronunciation dictionaries (us_gold.json over us_silver.json, each word's default reading), Apache-2.0 (lexicons/LICENSE.misaki), as word<TAB>ipa -- the shape loom.phonemizers.set_lexicon reads.