af_heart_arm_tts

A distilled single-voice, single-language text-to-speech model (af_heart, English), built for Arm inference on NVIDIA GB10 (DGX Spark).

Trained, exported and served entirely by GPL-free code. No piper, no GPL dependency in any runtime path.

Code, benchmarks and the Arm CPU↔GPU cooperative pipeline: https://github.com/dlyog/arm-cooperative-tts

---

Why it is small

Kokoro-82M is general purpose — 11 voices, multiple languages. Measured from its own module tree, most of that capacity is unnecessary for one voice:

Kokoro-82M component	params	needed for one voice?
predictor (prosody)	16.20 M	no
decoder.decode (style conditioning)	27.94 M	no — multi-voice machinery
decoder.encode	5.66 M	no
bert + text_encoder	11.90 M	ours is smaller
decoder.generator (vocoder)	19.69 M	yes — ours is 3.76 M

---

Measurements

DGX Spark GB10, idle GPU, 20 held-out sentences, each engine in its own process. Produced by dgx-spark/benchmark.py in the repo; raw output is committed as dgx-spark/results.json.

engine	RTF	latency	peak RSS	size
this model — Arm CPU	0.03481	92.4 ms	396 MB	68.5 MB
this model — Arm CPU → GPU	0.01596	43.3 ms	1974 MB	68.5 MB
Kokoro-82M — GPU	0.01421	40.2 ms	3239 MB	326 MB
Kokoro-82M — Arm CPU	0.34894	984.6 ms	2775 MB	326 MB

On the Arm CPU: 10.0× faster than Kokoro, 7.0× less memory, 4.8× smaller. The cooperative path also runs in 1.6× less memory than Kokoro on the GPU.

Full detail, including word and character error rates, is in EVAL.md. Audio you can listen to is in SAMPLES.md.

Cooperative split

stage	time	share
Arm CPU — encoder, duration predictor, flow	15.85 ms	59.2%
handoff — unified memory	0.1215 ms	0.45%
Blackwell GPU — vocoder	10.79 ms	40.3%

Stated plainly: the hybrid path is 0.89× Kokoro-GPU — slightly slower. Running a whole model on CUDA beats splitting it. The Arm result that stands alone is the CPU comparison.

---

Files

file	size	needed for	what it is
af_heart.onnx	68.5 MB	CPU inference	the full model
af_heart.onnx.json	5 KB	everything	sample rate, phoneme map, inference scales
encoder_prefix.onnx	54.1 MB	hybrid	the Arm CPU half — encoder, duration predictor, flow
af_heart_decoder.pt	14.4 MB	hybrid	the GPU half — the 3.76M vocoder

Everything here totals ~138 MB — smaller than the 327 MB Kokoro-82M checkpoint it is compared against.

encoder_prefix.onnx is a subgraph of af_heart.onnx, exported separately so it can run on the Arm CPU while the vocoder runs on the GPU.

The 834 MB training checkpoint is deliberately not published. Inference reads exactly one thing out of it — the vocoder — which ships on its own as af_heart_decoder.pt. Distributing the checkpoint would have made this repo ~1 GB, three times the size of the baseline this model is meant to undercut, to ship discriminator and optimizer state that inference never touches. To train your own, use the pipeline in the code repository on your own audio.

---

Usage

git clone https://github.com/dlyog/arm-cooperative-tts
cd arm-cooperative-tts
bash run.sh setup --full     # venv, deps, and these weights
bash run.sh demo

Direct use:

from tts import TTSModel
m = TTSModel("models/af_heart.onnx")     # auto-tunes threads for asymmetric Arm
m.synthesize_to_wav("Hello there.", "out.wav")

Requires espeak-ng installed. It is invoked as a separate process, never linked, which is what keeps its GPL terms from propagating.

Inference settings

Ships with noise_scale: 0.0, noise_w: 0.6 rather than piper's 0.667 / 0.8. Chosen by sweeping both and scoring each with a Whisper ASR round-trip.

---

Training

• Data: 4,104 clips, 3.0000 hours, 24 kHz mono — synthesized by Kokoro-82M (af_heart), i.e. knowledge distillation
• Architecture: VITS, 3.76 M vocoder (ResBlock1, 256 channels), MRD enabled
• Trainer: GPL-free, adapted from MIT sources — see the repo's training/
• Hardware: NVIDIA DGX Spark (GB10), 374 epochs / 48,000 steps

---

Limitations

Quality is below Kokoro's. Because the training audio came from Kokoro, Kokoro is a hard ceiling — a student can approach its teacher, not exceed it.

WER 0.0362 / CER 0.0308 on the held-out set, against Kokoro's 0.0299 / 0.0321 measured the same way — character error rate is lower than the teacher's, word error rate within 1.2×. The two sentences we miss are single-word confusions (sugar → "shower", report → "record"). See EVAL.md.

WER measures only whether words are recognizable. It is blind to naturalness, prosody and tonal artifacts. Listen to samples/.

English only, by design. Not validated beyond ~6.25 s per chunk — the longest clip in the training data — so longer text should be chunked by sentence, which the repo does automatically.

---

Citation

@software{chawdhury2026afheartarmtts,
  author       = {Chawdhury, Tarun Kumar},
  title        = {af\_heart\_arm\_tts: A Distilled Single-Voice TTS Model
                  for Arm Platforms},
  year         = {2026},
  url          = {https://huggingface.co/dlyog/af_heart_arm_tts},
  organization = {DLYog Lab Research Services LLC}
}

Tarun Kumar Chawdhury, DLYog Lab Research Services LLC. af_heart_arm_tts: A Distilled Single-Voice TTS Model for Arm Platforms. 2026. https://huggingface.co/dlyog/af_heart_arm_tts

---

License and attribution

Copyright © 2026 Tarun Kumar Chawdhury, DLYog Lab Research Services LLC. Apache 2.0.

• Kokoro-82M (Apache-2.0) — teacher model for the training audio, and the comparison baseline. This work is not affiliated with or endorsed by it.
• Model definitions adapted from jaywalnut310/vits (MIT) and jik876/hifi-gan (MIT).
• espeak-ng (GPL-3.0) is invoked as a separate process only, never linked.