WhissleAI/steer-tts-edge

SteerTTS v1 edge

The 12 MB on-device build of SteerTTS v1, a small English text-to-speech model that you steer with numbers. Everything an app has to ship (weights, voices, text front end and the Python runtime) is 11.84 MB:

• Acoustic model: the SteerTTS v1 acoustic model (9.25M parameters; attention + bidirectional Mamba-3, AdaLN conditioning), quantization-aware trained to int4, with int8 kept for the conditioning path.
• Vocoder: a Vocos-style vocoder (6.2M parameters) distilled from charactr/vocos-mel-24khz, also int4 QAT.
• Text front end: pure numpy. A residual CMUdict lexicon, an int8 copy of g2p_en's neural G2P for unknown words, and number / currency / unit normalisation. No NLTK, g2p_en, num2words, Vocos or torchaudio at runtime.

It keeps the steering of the full model:

• 200 built-in voices (voice_0005 … voice_4223, a subset of the full model's 2,825; default voice_1688).
• 50 style tags (whisper, sadness, anger, fast, high_pitch, …).
• 12 numeric controls (rate, loudness, median pitch, pitch range, final rise, pauses, tilt, breathiness, …), each a z-score relative to the chosen voice.
• Inline markup: *emphasis*, [pause 400ms], <slow>, <whisper>…</whisper>, …

Not on device. Use the full model, WhissleAI/steer-tts, on a host for these:

• free-text prompts ("whispering, slow and secretive"): the host resolves them into tag + ctrl, and those you can send to the device;
• voice cloning: the WavLM speaker encoder is not included;
• the flow-matching refiner: the edge build is acoustic model + vocoder only.

Licence: weights CC BY-NC 4.0 (non-commercial use only); package code MIT. The weights are distilled from SteerTTS v1, which was trained partly on non-commercial data. See LICENSE.md.

Listen

Raw output of this package (int4 acoustic model + int4 vocoder, default voice voice_1688), rendered on a CPU with the steertts-edge CLI (tools/render_samples.py).

Request	Audio
"Hi, thanks for calling. I can help you reschedule your appointment today." (no controls)
"Are you coming with us tonight?" ctrl={"f0_end": 2, "f0_iqr": 1.5}: rising, livelier
"The weather was cold, but the view from the top was worth every step." ctrl={"rate": -1.5}
Same text, ctrl={"rate": 1.5}
"Please keep this between the two of us." tag="whisper"
"I really thought it would work this time." tag="sadness", guidance=1.5
Markup: Well, *that* was fast. [pause 400ms] <slow>Let me think about it.</slow> <whisper>Don't tell anyone.</whisper>

Size

The on-device payload, as counted by tools/sync_package.py (weights, voices, front end, runtime code, config, notices; docs, tests and tooling excluded):

Part	Bytes
acoustic.safetensors	6,648,886	Acoustic model, int4 QAT (+ int8 / fp16 for sensitive tensors)
vocoder.safetensors	3,862,640	Vocoder, int4 QAT (Vocos-style, dim 384, 8 ConvNeXt layers)
front/	1,150,249	Text front end: residual CMUdict lexicon, int8 GRU G2P, normaliser tables
steertts_edge/	121,859	Runtime code (Python)
voices.safetensors	30,544	200 built-in voices (64-d conditioning slots)
voices.json	19,720	Voice names and descriptors
NOTICE	1,652	Third-party notices
config.json	1,170	Config (tags, control scales)
Total	11,836,720	11.84 MB, budget 12,000,000 bytes

For comparison, the full SteerTTS v1 install is about 765 MB of weights (fp32, including the external encoders), or 193 MB in its int8 variant.

Quick start

git clone https://huggingface.co/WhissleAI/steer-tts-edge && cd steer-tts-edge
pip install .                    # torch, numpy, safetensors; add [hub] for from_pretrained("WhissleAI/steer-tts-edge")
steertts-edge "Hello there, nice to meet you." -o hello.wav
steertts-edge "I can't believe it worked." --voice voice_0047 --tag happiness --ctrl f0_iqr=1.5 --rate 1 -o happy.wav
steertts-edge "Well, *that* was fast. [pause 400ms] <slow>Let me think.</slow>" --markup -o markup.wav
steertts-edge --list voices      # or: tags, controls

The CLI uses --model DIR_OR_REPO, else $STEERTTS_EDGE_MODEL, else the current directory if it holds the package, else downloads WhissleAI/steer-tts-edge. A CPU-only torch wheel is enough: pip install torch --index-url https://download.pytorch.org/whl/cpu.

from steertts_edge import EdgeTTS

tts = EdgeTTS("path/to/steer-tts-edge", threads=4)            # a local directory, CPU
# tts = EdgeTTS.from_pretrained("WhissleAI/steer-tts-edge")   # downloads ~12 MB (needs huggingface_hub)
wav = tts("The meeting moved to Thursday.")                    # float32 numpy, 24 kHz, voice_1688
wav = tts("Are you serious?", voice="voice_0047", ctrl={"f0_end": 2.0, "f0_iqr": 1.5})   # rising, lively
wav = tts("It is over.", tag="sadness", ctrl={"loud": -1}, guidance=1.5)
wav = tts("Please keep this quiet.", tag="whisper", ctrl={"rate": -1})
wav = tts("Well, *that* was fast. [pause 400ms] <whisper>Don't tell anyone.</whisper>", markup=True)
tts.save(wav, "out.wav")                                       # 16-bit PCM WAV

for chunk in tts.stream("First sentence. Second sentence. <slow>Third.</slow>"):   # sentence-level streaming
    play(chunk.audio)                                          # chunk.segment has the text, tag and controls

Renders are deterministic: the same request gives the same samples, on the same device.

The request

Argument	Meaning
text	English text. Numbers, currency, units, ordinals, ranges and unknown acronyms are spelled out. Text with nothing to say (empty, punctuation only, non-Latin script) returns an empty array.
voice	One of the 200 built-in voices (tts.voices); default voice_1688. The leaderboard rows use voice_1361.
tag	One of 50 style tags (tts.tags, plus none).
ctrl	{control: z} over the 12 controls, each in [−5, 5].
guidance	Steering scale (default 1). Above 1 it extrapolates away from the uninstructed render.
pace	Global tempo divisor (> 1 is faster).
word_shift	{word_index: {"pitch": st, "energy": dB, "dur": ×}} per-word edits (not with markup / stream()).
markup	Parse inline markup (see below). stream() always does.
spk_vec	Advanced: a 512-d WavLM-base-plus-sv speaker vector computed on a host by the full model, in place of voice. Not evaluated on the edge build.

prompt= and ref_wav= raise an error that points to the full model. flow= and seed= are accepted and ignored, so code written for steertts.SteerTTS(flow=False) runs unchanged.

Controls. Each is a z-score against the chosen voice's own neutral speech, in units of the pooled within-speaker SD: rate (articulation rate), loud (speech level), f0_med (median pitch), f0_iqr (pitch range), f0_end (final pitch slope), pause (pause share), tilt (spectral tilt), voiced (voiced share), flat (spectral flatness), centroid (spectral centroid), cpps (cepstral peak prominence), h1h2 (H1–H2). A control you leave out is chosen by the model from voice, tag and text; {"rate": 0} asks for the voice's neutral rate.

Style tags. adoration, amazement, amusement, anger, anxiety, apologetic, bored, confidence, confusion, contempt, contentment, cuteness, desire, determined, disappointment, disgust, distress, ecstasy, embarrassment, emphasis, enunciated, excited, fast, fear, frustration, guilt, happiness, hesitant, high_pitch, interest, irritation, laughing, loud, low_pitch, mysterious, neutral, nostalgia, pain, panic, pride, realization, relief, sadness, sarcasm, serenity, skeptical, slow, sympathy, tired, whisper.

Markup (markup=True or stream()):

Markup	Effect
*word* or <emph>…</emph>	Emphasis: +2.5 st, +4 dB, ×1.2 duration
<slow> <fast> <high> <low> <loud> <soft>	Per-word duration / pitch / energy changes; the sentence is still one pass
[pause], [pause 300ms], [pause 1.5s]	Exact silence (default 400 ms, max 5 s)
<whisper>…</whisper>, <sadness>…</sadness>, any style tag	The span is rendered as its own segment with that tag

Unknown markup tags are dropped and their text is spoken normally. <style prompt="…"> needs the host-side prompt adapter and raises an error on the device.

Quality

Build	Size	Seed-TTS en WER	Seed-TTS en UTMOS	CV3-Eval en WER	CV3-Eval en UTMOS
SteerTTS v1 fp32, flow refiner off	~765 MB	0.96 %	4.20	2.62 %	4.17
steer-tts-edge (this repo)	11.84 MB	0.96 %	4.11	2.67 %	4.08

Open TTS Leaderboard protocol, run locally (not an official submission): Qwen3-ASR-1.7B WER, UTMOS22-strong, voice voice_1361, Seed-TTS-eval English (1,088 sentences) and CV3-Eval English (500 sentences). The fp32 row is the full model's acoustic model + Vocos without the flow refiner, the configuration the edge build reproduces.

• Intelligibility is unchanged (+0.00 / +0.05 WER points).
• UTMOS drops by 0.09 (Seed) / 0.10 (CV3). Almost all of it comes from the 6M-parameter int4 vocoder, which replaces the 13.5M-parameter fp32 Vocos. The int4 acoustic model with the fp32 Vocos scores 4.20 / 4.17 UTMOS (fp32: 4.20 / 4.17) and 0.95 % / 2.66 % WER. An earlier vocoder of the same size (the first release candidate) scored 4.09 / 4.06.

How well it steers

Readback of the controls from the rendered audio (the measurement code that labelled the training data), 8 voices × 6 sentences, requests of −2 and +2, on CPU. Δz is the mean achieved change in z; hit is the share of renders that move more than 0.5 SD the requested way. The fp32 columns are the full model without the refiner, measured the same way on the same voices and sentences.

Control	fp32 Δz at −2 / +2	edge Δz at −2 / +2	edge / fp32 (worse side)	edge hit at −2 / +2
loud	-1.44 / +1.83	-1.43 / +1.81	0.99	1.00 / 1.00
f0_med	-1.15 / +1.51	-1.17 / +1.48	0.98	1.00 / 1.00
rate	-1.50 / +1.38	-1.51 / +1.36	0.99	1.00 / 1.00
h1h2	-1.77 / +1.57	-1.71 / +1.51	0.96	1.00 / 1.00
tilt	-1.51 / +0.97	-1.50 / +0.97	0.99	1.00 / 1.00
f0_iqr	-0.65 / +0.87	-0.62 / +0.85	0.95	0.65 / 0.79

Tag (readback)	fp32 hit	edge hit	Δ hit	fp32 Δz	edge Δz
whisper (voiced −)	1.00	1.00	+0.00	-9.47	-9.06
loud (loud +)	1.00	1.00	+0.00	+1.42	+1.35
fast (rate +)	0.50	0.44	-0.06	+0.50	+0.46
slow (rate −)	0.98	1.00	+0.02	-1.03	-1.07
high_pitch (f0_med +)	1.00	1.00	+0.00	+2.26	+2.32
anger (loud +)	0.12	0.23	+0.10	+0.24	+0.26
sadness (loud −)	0.79	0.73	-0.06	-0.78	-0.75

• Every control keeps at least 95 % of the fp32 effect (worst: f0_iqr at −2, 0.96×).
• Tags move the same way as in fp32, slightly less for two of them. fast (Δz +0.46 vs +0.50) and sadness (−0.75 vs −0.78) each lose 3 of 48 hits at the 0.5 SD threshold (0.50 → 0.44 and 0.79 → 0.73).
• The pitch-range control (f0_iqr) and the fast/anger tags are weak in fp32 too.

Speed

CPU threads	RTF (20 sentences)	RTFx	Time to first audio (short sentence)	First streamed chunk	Load	Peak RSS
1	0.079	12.7×	138 ms	137 ms	1.5 s	363 MB
4	0.053	18.7×	118 ms	119 ms	1.5 s	376 MB

Measured with tools/bench_speed.py on this package: CPU only (AMD Ryzen Threadripper 2970WX, a 2018 desktop part), torch 2.14.1+cpu, a shared machine with a load average of about 14 from other jobs, so treat these as lower bounds. RTF is synthesis time ÷ audio duration over 20 sentences (65.6 s of audio); time to first audio is a full call on a 4-word sentence ("Hello, how are you?"), warm, and the first streamed chunk is that sentence as the first segment of a two-sentence stream(). Peak RSS is the whole Python process, most of it torch itself. Four threads are only about 1.5× faster than one at this model size.

Architecture and quantization

The acoustic model is SteerTTS v1 (FastPitch-style: phonemes → encoder → per-phone duration, pitch and energy → length regulator → decoder → 100-band log-mel). Every block is attention followed by a bidirectional Mamba-3 branch, and every LayerNorm is adaptive, conditioned on [voice slot, tag, controls ⊙ mask, mask]. See the full model card for details.

• Acoustic model. Quantization-aware training from the released v1 checkpoint (distillation + a control-delta loss), then packed: 6.49M weights int4 (groups of 32 along the input dimension, one fp16 scale per group), 2.68M int8 (embeddings, the conditioning MLP, AdaLN projections, the variance heads, Mamba-3 step-size projections; one fp16 scale per row), 85k fp16. The built-in voices are stored as their 64-d conditioning slots.
• Vocoder. A Vocos-style ConvNeXt + iSTFT vocoder (dim 384, intermediate 896, 8 layers) distilled from charactr/vocos-mel-24khz, then quantization-aware trained: 5.51M weights int4, 0.68M int8, the rest fp16.
• Compute. Weights are dequantized to fp32 at load and compute stays fp32, so the gain is size, not speed. steertts_edge/packed.py has the format.
• Front end. The same symbols and rules as the full model's front end (g2p_en semantics, text normalisation), re-implemented on compact assets: CMUdict entries that the G2P model reproduces are stored without phones, the G2P GRU is int8, and heteronyms are resolved by a small context classifier instead of the NLTK tagger.

Limitations

• Audio quality is a step below the full model. UTMOS −0.09 / −0.10 (Seed / CV3) vs fp32 without the refiner, and further below the full model with its flow refiner (UTMOS 4.27 / 4.24). The cause is the small int4 vocoder.
• English only. ARPAbet front end, one speaker per request, utterance-level controls.
• Unknown words go through a small G2P model. Unusual names and foreign words can be mispronounced. Non-English letters are reduced to ASCII (café → cafe), and text in other scripts is dropped.
• Text normalisation has the full model's blind spots. The edge front end reproduces the v1 front end (identical phonemes on 99.6 % of Seed-TTS and 98.2 % of CV3-Eval sentences; phoneme error rate vs v1 below 0.1 %), including its mistakes: Dr. is read "drive", a phone number like 555-0198 as a range ("555 to 0198"), and clock times come out garbled (5:30 becomes one made-up word). Spell such text out when it matters.
• No cloning or free-text prompts on the device, and no flow refiner (see the top of this card). Passing a host-computed spk_vec works mechanically but was not evaluated.
• Emotion tags are style switches, not emotions (as in the full model). Voice-quality tags such as whisper act strongly; most emotion tags are acoustically close to neutral.
• The controls are acoustic measurements, not percepts. voiced, flat, cpps and pause barely move in the full model either.
• Test-set-informed choices. The vocoder was chosen between two candidates of the same architecture by their UTMOS on the leaderboard sets (Seed-TTS and CV3-Eval), so its numbers here are slightly optimistic. The leaderboard rows use voice_1361, the full model's pre-registered v1.0.0 default.
• No human evaluation. Every number here is from automatic metrics.

Intended use and misuse

Intended uses:

• Offline, on-device speech output where size matters: apps, embedded and edge devices, accessibility tools.
• Research on controllable, interpretable and compressed speech synthesis.

Do not use this model to impersonate real people, produce deceptive or fraudulent audio, bypass voice authentication or create non-consensual content. Disclose synthetic speech to listeners. There is no watermark.

Built-in voices are anonymised. They are training speakers under the full model's opaque public names. No speaker or clip identifiers from the source corpora are included.

Training data

The acoustic model and voices are distilled from SteerTTS v1, and the vocoder from charactr/vocos-mel-24khz. The full model's training data and terms apply: see Training data on its card.

Licence: CC BY-NC 4.0. Non-commercial use only.

Files

File	Contents
acoustic.safetensors	Acoustic model, packed int4 / int8 / fp16 (layout in the safetensors metadata)
vocoder.safetensors	Vocoder, packed int4 / int8 / fp16
voices.safetensors	Per voice: 64-d conditioning slot and neutral control means
voices.json	Voice names and measured descriptors (gender probe, pitch register, pace, median F0)
config.json	Tags, control scales, architecture, file map
front/	Text front end assets: residual lexicon, int8 G2P, heteronym and normaliser tables
steertts_edge/	The runtime package (pip install .)
tests/	pytest suite
samples/*.wav	The clips in Listen (not part of the device payload)
tools/	sync_package.py (copy a built package in), bench_speed.py, render_tables.py, render_samples.py, release_numbers.json
TEST_LOG.md	Release verification log
LICENSE.md, NOTICE	Code MIT, weights CC BY-NC 4.0; third-party notices

Release verification

Verified in a fresh Python 3.11 venv with the CPU torch wheel, the package installed with pip and tested from outside the source tree; TEST_LOG.md has the full log.

• Tests. The pytest suite passes: all 200 voices, all 50 tags, each control at −2 / 0 / +2, markup, streaming, text edge cases, error messages, determinism, the CLI, and no NLTK / g2p_en / Vocos / torchaudio / transformers import at runtime.
• Parity. Renders are bit-identical to the evaluated build, including across torch 2.11 and 2.14.
• Steering gates vs fp32. Every control keeps ≥ 0.9× its fp32 effect (worst 0.96×). Tag hit rates stay within 0.05 of fp32 except fast and sadness (−0.06 each, one render past the gate; see How well it steers).

Citation

@misc{steertts2026,
  title  = {SteerTTS: A Tiny, Measurably Steerable Speech Synthesizer},
  author = {{Whissle AI}},
  year   = {2026},
  howpublished = {\url{https://huggingface.co/WhissleAI/steer-tts}}
}