hugging-mac/audio8-tts-0.1b-coreai

Audio8 TTS 0.1B — native Core AI

Core AI conversion of Edge0/Audio8-TTS-Preview-0.1b, for the hugging-mac SDK. Original Audio8 architecture, tokenizer, and weights are credited to the upstream authors. The original Audio8 Community License v1.0 applies unchanged.

Current native build

All neural-network stages execute in Core AI:

Stage	Graph
Falcon-H1 Slow AR (attention and Mamba recurrence)	slow.aimodel
Fast AR, including embeddings	fast.aimodel
Reference audio encoding and quantization	encoder.aimodel
Waveform decoding	codec.aimodel

Inference requires no PyTorch, Transformers, or torchaudio. Host code uses NumPy for orchestration and sampling, the Rust tokenizer for text, and SoundFile/SciPy for audio I/O and resampling. CPU host processing still exists; this does not mean every operation runs on GPU or ANE.

Slow AR keeps native KV, convolution, and recurrent states across tokens, resetting them between requests. Prompt ingestion currently uses sequential recurrent steps rather than parallel prefill. The fast graph resets native KV state at the start of every audio frame. Output is mono 44.1 kHz. Total prompt plus generation is limited to 2048 positions; the codec supports 1–2048 frames.

auto lets Core AI select accelerators. Some operators fail ANE validation, so ANE-only execution is not claimed. In one local test generating 48 frames (2.23 seconds of audio), native auto synthesis took 8.98 seconds initially and 3.25 seconds on repeat. The former hybrid build measured 2.71 seconds on repeat on the same machine. Native conversion removes inference framework dependencies; it does not guarantee a speedup.

Layout

The root represents SDK model ID audio8/audio8-tts-preview:

0.1b-preview/coreai/coreai-native/
  slow.aimodel/{metadata.json,main.mlirb,main.hash}
  fast.aimodel/{metadata.json,main.mlirb,main.hash}
  encoder.aimodel/{metadata.json,main.mlirb,main.hash}
  codec.aimodel/{metadata.json,main.mlirb,main.hash}
  runtime.json
_shared/tokenizer-0.1b/
  special_tokens_map.json
  tokenizer.json
  tokenizer_config.json

The SDK downloads each declared prefix into its canonical artifact directory beneath <model_home>/audio8/audio8-tts-preview/. native-checksums.json records native payload sizes and SHA-256 hashes. The older coreai-hybrid directory remains for older SDK checkouts; it is not used by the current native integration. Download or convert the native artifact when updating the SDK.

Usage

Use a hugging-mac checkout containing the native Audio8 integration, on a compatible Apple Silicon Mac:

python3.12 -m venv .venv-audio8-coreai
.venv-audio8-coreai/bin/pip install -e './packages/hugging_mac_sdk[coreai]' tokenizers scipy soundfile

import asyncio
from pathlib import Path
from hugging_mac_sdk import ModelSdk, SpeechSynthesisRequest
from hugging_mac_sdk.capabilities import SpeechSynthesis
from hugging_mac_sdk.models.audio8_tts import register_audio8_tts

async def main():
    sdk = ModelSdk()
    register_audio8_tts(sdk.registry)
    model_id = "audio8/audio8-tts-preview"
    options = {"model_home": Path("models")}
    await sdk.resources.download_source(model_id, variant="0.1b-preview",
                                       options=options | {"runtime": "coreai"})
    async with await sdk.load(model_id, variant="0.1b-preview", runtime="coreai",
                             options=options | {"device": "auto"}) as handle:
        result = await handle.require(SpeechSynthesis).synthesize(
            SpeechSynthesisRequest(text="你好，这是一个语音合成测试。", max_new_tokens=128)
        )
        Path("audio8-coreai.f32le").write_bytes(result.audio)

asyncio.run(main())

Voice cloning accepts both reference_audio and reference_text. Non-native-rate audio is resampled with SciPy. Sampling uses the upstream filtering and repetition-aware strategy, but NumPy random draws differ from PyTorch. Loading never downloads or converts files. If Hub large-file transfer stalls, set HF_HUB_DISABLE_XET=1 before starting Python.

Conversion and validation

Exported using coreai-core 1.0.0b2, coreai-torch 0.4.2, PyTorch/torchaudio 2.8.0 and Transformers 4.57.5 in a separate environment; tested on Apple Silicon macOS 27. These PyTorch dependencies are needed only for re-conversion or reference comparisons.

Checks include native Slow AR against original PyTorch multi-token prefill and continuous decode, state reset between requests, native Fast AR and waveform decoder parity, and exact reference-encoder code equality at 1, 8 and 17 frames. End-to-end plain synthesis, repeated synthesis, and reference voice cloning succeeded in a fresh Python process that rejected imports of torch, torchaudio, and transformers.