ONNX Runtime mismatch: run_infer.sh fails — 0.1B graph incompatible with current onnx_runtime``
Hi Audio8 team — thanks for the 0.1B ONNX INT8 release.
Tested on Jetson Orin Nano Ubuntu 22.04 Python 3.10.12 onnxruntime 1.22.1 CPUExecutionProvider following the card instructions:
git clone https://github.com/Audio8-AI/Audio8_TTS.git
cd Audio8_TTS/onnx_runtime
hf download Audio8/audio8-TTS-0.1B-ONNX-INT8 --local-dir model
bash setup.sh # fails: setup.sh:13 requires Python 3.11+
Bypassed the 3.11 check, created voices/default from reference_codes.npy + runtime_manifest.json, then:
PYTHONPATH=. python -m arktts_runtime.cli --voice default --text "这是一个中文测试" --output out.wav
Fails with:
File arktts_runtime/runtime.py:94, in _empty_slow_caches
dtype = ORT_DTYPES[self.slow_inputs["cache_key_0"].type]
KeyError: 'cache_key_0'
Root cause: master onnx_runtime/arktts_runtime/runtime.py is for 0.6B INT4 (cache_key_0 per layer [1,2,2048,64]). This 0.1B hybrid Falcon-H1 graph uses a different recurrent contract:
- slow_ar_int8.onnx inputs: codes [1,11,1], position [1], cache_keys [24,1,2,2048,64], cache_values [24,1,2,2048,64], conv_states [24,1,896,4], ssm_states [24,1,24,32,64]
- fast_ar_int8.onnx + codec_decoder_fp16.onnx also differ
- Card references python3 scripts/register_default_voice.py which no longer exists on master (was deleted at 0bb2060)
The *.onnx files do run via direct onnxruntime.InferenceSession (slow 0.26s/step, fast 0.02s, decoder 5s for 20 frames OK), so hardware is fine — just the bundled onnx_runtime is incompatible.
Would be great to either publish the matching 0.1B onnx_runtime code/branch or update the model card to point to the correct runtime.
Hi @jerryyong5555 ,
Thanks A LOT for this report, this does reveal a critical mistake we made during the release.
The root cause of this issue is that the 0.1B-specific code we prepared and tested locally was not released alongside the Hugging Face model, according to the convention. And my assumption that the GitHub onnx_runtime part can be shared is a funny lazy wish(Aha) since the architectures are not identical.
Your Jetson reproduction, stack trace, and exact ONNX input shapes made the mismatch clear. Thank you!
For anyone who may encounter the similar issue: We’ve now added the matching 0.1B runtime here, including a website UI for your convenience.
We also updated the Hugging Face model card and usage instructions, so the further practice following the manual shall be safe.