drbaph/Raon-OpenTTS-comfyui

Raon-OpenTTS for ComfyUI (fp32 / bf16 / INT8 ConvRot safetensors)

ComfyUI-ready repack of KRAFTON Raon-OpenTTS — open-weight, open-data zero-shot voice cloning (F5-TTS-style CFM/DiT, 16 kHz, English) — converted to clean inference-only safetensors, with an INT8 ConvRot build that runs natively through comfy-kitchen quantized kernels.

ComfyUI node pack: https://github.com/Saganaki22/ComfyUI-Raon-OpenTTS

Original models: KRAFTON/Raon-OpenTTS-1B · KRAFTON/Raon-OpenTTS-0.3B · GitHub · arXiv:2605.20830 · Dataset

Vocoder: speechbrain/tts-hifigan-libritts-16kHz (mirrored here as tts-hifigan-libritts-16kHz/generator.ckpt for one-stop download)

What's inside

Raon-OpenTTS-1B/   (1042M params, dim 1408, depth 28, heads 24)
  config.yaml
  vocab.txt
  Raon-OpenTTS-1B-fp32.safetensors           4.17 GB   lossless EMA extraction (reference)
  Raon-OpenTTS-1B-bf16.safetensors           2.08 GB   half-precision runtime build
  Raon-OpenTTS-1B-int8-convrot.safetensors   1.41 GB   INT8 ConvRot (comfy-kitchen)
  Raon-OpenTTS-1B-int8-convrot-verify.json   per-layer quantization error table
Raon-OpenTTS-0.3B/ (339M params, dim 1024, depth 22, heads 16)
  config.yaml
  vocab.txt
  Raon-OpenTTS-0.3B-fp32.safetensors         1.35 GB
  Raon-OpenTTS-0.3B-bf16.safetensors         0.68 GB
  Raon-OpenTTS-0.3B-int8-convrot.safetensors 0.50 GB
  Raon-OpenTTS-0.3B-int8-convrot-verify.json
tts-hifigan-libritts-16kHz/generator.ckpt    HiFi-GAN vocoder (16 kHz)

All builds are the EMA weights from the official training checkpoints (model_520000.pt / model_225000.pt), which is what the official inference path uses. The fp32 extraction is verified bit-exact (every tensor torch.equal against the source EMA state dict; strict load_state_dict passes with 0 missing / 0 unexpected keys).

INT8 ConvRot

Comfy-native int8_tensorwise + ConvRot format (<layer>.weight int8, <layer>.weight_scale fp32 per-row, <layer>.comfy_quant JSON marker). Only the repeated transformer-block GEMMs are quantized — attention Q/K/V/out and the two FFN projections; embeddings, AdaLN modulation, time MLP, input projection, ConvNeXt text blocks, norms, convolutions and the final projection stay full precision.

	1B	0.3B
quantized layers	168 (28 blocks x 6)	132 (22 blocks x 6)
group sizes	112x GS64 + 56x GS256	132x GS256
quantized params	686.3M (65.9%)	184.5M (54.6%)
weight error (mean / max)	0.84% / 1.26%	0.80% / 0.89%
mel vs bf16 build (rel L2 / cosine)	~0.95% / 0.99996	~0.87% / 0.99997

GS64 matters here: the 1B transformer width 1408 is not divisible by 256, so 1408-wide layers use ConvRot group size 64 (1408 % 64 == 0) while the 1536/5632-wide layers use 256.

Measured on RTX 5090 (load peak VRAM / inference peak VRAM / speed):

1B	load	inference	RTF
fp32	4.26 GB	4.47 GB	0.37
bf16	2.22 GB	2.39 GB	0.22
int8-convrot	1.50 GB	1.67 GB	0.37

INT8 trades some speed for the smallest memory footprint at these GEMM sizes on this GPU; bf16 is the fastest build. Generated audio transcribes word-perfectly with Whisper across all builds (see the node pack repo for the validation harnesses).

Usage

Install the node pack: https://github.com/Saganaki22/ComfyUI-Raon-OpenTTS — the Raon OpenTTS Load Model node downloads from this repo automatically when the files are missing, into ComfyUI/models/raon_opentts/.

Manual download:

from huggingface_hub import snapshot_download
snapshot_download("drbaph/Raon-OpenTTS-comfyui",
                  local_dir="ComfyUI/models/raon_opentts")

Vocab note (5559 vs 5555)

The shipped vocab.txt has 5559 tokens while both checkpoints embed 5555 rows. The node pack builds the model from the checkpoint shape (ground truth) and drops the four overhanging tokens — the four highest-codepoint entries of the sorted vocab (U+FDFA, U+FDFB, U+FFFD, U+1F3B5) — from the runtime map. All realistic text is unaffected (verified by Whisper transcription of generated audio).

Citation

@article{kim2026raonopentts,
  title     = {Raon-OpenTTS: Open Models and Data for Robust Text-to-Speech},
  author    = {Kim, Semin and Chung, Seungjun and Moon, Taehong and Lee, Sangheon and Ahn, Minyoung and Lee, Keon and Kim, Nam Soo and Cho, Jaewoong and Schmidt, Ludwig and Lee, Kangwook and Park, Dongmin},
  journal   = {arXiv preprint arXiv:2605.20830},
  year      = {2026},
  url       = {https://arxiv.org/abs/2605.20830}
}

License

This repository is licensed under the Creative Commons Attribution-NonCommercial 4.0 International License, matching the upstream KRAFTON model weights. Please credit KRAFTON AI (see citation above) when using these weights.