shadowtec/Audio8-TTS-0.1B-ONNX-W8A16

Audio8 TTS 0.1B ONNX W8A16 (experimental)

This is an independently generated, experimental weight-only INT8 export of Audio8 TTS Preview 0.1B. It uses the same quantized linear operator layout as the official 0.6B INT4 ONNX model, changing the packed weight precision from 4 to 8 bits:

com.microsoft::MatMulNBits
bits = 8
block_size = 128
accuracy_level = 4
asymmetric block-wise quantization
FP16 activation inside each quantized linear operation

The 238 linear weights were quantized directly from the original BF16 model.safetensors (217 Slow AR and 21 Fast AR). They were not recovered from or requantized from the official W8A8 ONNX weights. The public W8A8 graph was used only as a structural template to retain the exact public input, output, cache, and codec contracts.

Why this exists

The official 0.1B W8A8 graphs use dynamically quantized U8 activations and S8 MatMulInteger weights. That U8S8 path produced corrupted audio on one tested Windows/AMD AVX2 system. This W8A16 graph contains no MatMulInteger and no DynamicQuantizeLinear; it uses MatMulNBits(bits=8) instead.

This is not an official Audio8 release. Keep the official model as the quality and compatibility reference.

Runtime compatibility

The model preserves every official 0.1B Slow/Fast ONNX input and output name, shape, and dtype. No inference algorithm or cache-handling source change is required. Use ONNX Runtime 1.22 or newer with CPUExecutionProvider.

runtime_manifest.json selects W8A16 by default. It also maps the legacy int8 selector to the W8A16 files, allowing the unmodified official launcher, which may force ARKTTS_PRECISION=int8, to load this repository. In that legacy case the health endpoint can display int8 even though the loaded graph is W8A16; this is only a compatibility alias.

Runtime implementation and reproducible converter:

• https://github.com/Aaaou/Audio8_TTS/tree/experiment/onnx-w8a16
• https://github.com/Aaaou/Audio8_TTS/blob/experiment/onnx-w8a16/docs/onnx-w8a16-experiment.md

Download and run

Clone or download the runtime, then place this entire repository in its onnx_runtime_0_1b_int8/model directory:

git clone --branch experiment/onnx-w8a16 \
  https://github.com/Aaaou/Audio8_TTS.git
cd Audio8_TTS/onnx_runtime_0_1b_int8
python3 -m pip install -r requirements.txt
hf download shadowtec/Audio8-TTS-0.1B-ONNX-W8A16 --local-dir model
bash setup.sh
python3 scripts/register_default_voice.py
bash run_infer.sh \
  --text "这是语言合成量化效果测试" \
  --voice default \
  --max-new-tokens 256 \
  --output outputs/w8a16.wav

The manifest makes W8A16 the default, so --precision is optional. With the updated runtime, it can also be selected explicitly with --precision w8a16.

The included registration encoder is optional for synthesis but enables registering new authorized reference voices through the local service.

Validation

Tested on Windows 11, AMD Ryzen 5 5600, ONNX Runtime 1.23.2:

• Slow and Fast graphs load with CPUExecutionProvider;
• 238 MatMulNBits(bits=8) nodes;
• zero MatMulInteger nodes;
• zero DynamicQuantizeLinear nodes;
• end-to-end synthesis succeeds using unmodified official runtime source;
• first prefill logits correlate at approximately 0.991 with a verified U8U8 control and select the same first semantic token in the fixed diagnostic.

On this system W8A16 reached 9.86 AR frames/s at four threads, versus 10.01 for U8U8 and 7.44 for the original PyTorch FP32 CPU path. See the full benchmark report.

Linux memory check

The official U8S8 Slow AR, Fast AR, and FP16 codec decoder were also measured on Ubuntu 22.04 x86-64 with ONNX Runtime 1.29.0. Loaded-process RSS was 633.95 MiB with four threads and 635.62 MiB with eight threads, consistent with the official model card's “about 0.6 GB” figure. This check validates the official graph's memory claim; it is not a Linux quality or performance result for W8A16.

This is a real BF16-to-INT8 quantization, so token sequences and waveforms are not expected to be byte-identical to BF16 or U8U8. Additional listening tests, long prompts, other CPUs, W8A16 on Linux, and macOS still require validation.

中文说明

这是从原版 0.1B BF16 权重重新量化得到的实验版 W8A16 ONNX 模型。它完全避开 官方 0.1B W8A8 图中的 U8S8 MatMulInteger 路径，线性层改用与官方 0.6B INT4 相同类型的 MatMulNBits，仅把权重位宽从 4 改成 8。

Slow/Fast ONNX 的全部输入输出协议与官方 0.1B 一致。除模型选择或 manifest 外， 推理算法源码不需要修改。仓库 manifest 已提供兼容别名，因此即使旧启动脚本强制 选择 int8，实际也会加载 W8A16 文件。

本模型不是 Audio8 官方发布。当前已在 Windows 11、AMD Ryzen 5 5600、ORT 1.23.2 上验证；其他系统和 CPU 仍需测试。

License

These derived weights were generated from the original BF16 checkpoint and are distributed under the Audio8 Community License v1.0 included in this repository. It permits free non-commercial use and commercial use below its specified annual-revenue threshold; consult LICENSE for the authoritative terms. This repository does not replace or relax the upstream license.