florianvoss/kokoro-sima

Kokoro for SiMa Modalix

Kokoro v1.0 speech synthesis on Modalix using Neat. ALBERT and the vocoder run on MLA in BF16; the remaining neural stages run on CPU through ONNX Runtime. Output is 24 kHz mono audio. All 54 original voices are included. Requires Neat Library v0.6.1 installed on Modalix.

Prerequisite: Neat Library v0.6.1

Install or update the general Neat Library on Modalix before setting up Kokoro:

sima-cli neat install core@v0.6.1

Skip this step if Neat Library v0.6.1 is already installed. See SiMa's Neat Library installation guide for SiMa CLI setup and authentication. This is the device runtime requirement, separate from the Python bindings installed in Kokoro's venv below.

Setup

Create a separate Kokoro environment and install the matching PyNeat 0.6.1 wheel, following SiMa's custom-venv instructions:

hf download florianvoss/kokoro-sima --local-dir kokoro-sima
cd kokoro-sima
python3 -m venv .venv
source .venv/bin/activate
sima-cli neat install core@v0.6.1 -t pyneat
python -m pip install --ignore-requires-python ./pyneat-0.6.1-*.whl -r requirements.txt

The matching Neat 0.6.1 device runtime must already be installed. The -t pyneat command downloads the wheel; it does not update the device runtime. If you already have that wheel, copy it here and skip the download command.

Kokoro dependencies are installed only in .venv; the existing ~/pyneat environment is not modified. For later sessions, run source .venv/bin/activate. Thread defaults are handled by the runtime.

The Python-version override is needed because Misaki 0.9.4's published metadata excludes Python 3.13; that version has been tested with this runtime on Modalix. The Japanese text frontend builds mojimoji from source on Modalix. If Python development headers or a C++ compiler are missing, install them once before the pip command:

sudo apt-get install python3-dev g++

Text → WAV

python synthesize.py "Hello! This is Kokoro on Modalix." --voice af_heart -o speech.wav

Optional arguments: --language, --speed, --seed, --threads (default 16). Language is inferred from the voice when omitted.

For a resident engine in your Python application:

from engine import Kokoro

with Kokoro() as tts:
    tts.save_wav("Hello! How are you?", "speech.wav", voice="af_heart")
    audio, report = tts.synthesize("Another sentence.", voice="af_bella")

audio is a mono float32 NumPy array. Pass on_audio_chunk=callback to synthesize() to receive each PCM block as it is reconstructed. Reuse one engine for interactive applications; construction loads and warms the models. Calls on one engine are serialized.

Web app and HTTP API

python serve.py --host 0.0.0.0 --port 8089

Open http://MODALIX_IP:8089 on your laptop and wait for Modalix connected. Choose a language and voice, enter text, then generate speech. Playback starts when the first PCM block is ready. Stop, replay and WAV download are included.

The server also implements a subset of the OpenAI speech API:

curl http://MODALIX_IP:8089/v1/audio/speech \
  -H 'Content-Type: application/json' \
  -d '{"model":"kokoro","input":"Hello from Modalix!","voice":"af_heart","response_format":"wav"}' \
  --output speech.wav

• model: kokoro or kokoro-sima.
• voice: a bundled Kokoro ID, such as af_heart, bf_emma or ff_siwis.
• response_format: wav (default) or pcm. PCM streams immediately as raw signed 16-bit little-endian samples, mono at 24 kHz. WAV returns a complete file.
• speed: 0.5–2.0, default 1.0. Optional extensions: language, seed.
• /v1/models lists the served model. /api/options lists all voices and languages; /api/health reports readiness and busy state.

OpenAI Python clients can use OpenAI(base_url="http://MODALIX_IP:8089/v1", api_key="local") with model="kokoro", a Kokoro voice and response_format="wav" or "pcm". No API key is checked by this local server. MP3, SSE and free-form voice instructions are unsupported. The webapp uses its own framed float32 PCM endpoint at /api/synthesize. All endpoints share one resident engine; concurrent generation requests receive HTTP 409.

Languages and limits

Tag	Language	Example voice
a	American English	af_heart
b	British English	bf_emma
e	Spanish	ef_dora
f	French	ff_siwis
h	Hindi	hf_alpha
i	Italian	if_sara
j	Japanese	jf_alpha
p	Brazilian Portuguese	pf_dora
z	Mandarin Chinese	zf_xiaobei

Text splits at .?! and their Japanese/Chinese equivalents, preserving decimals and common abbreviations. Each sentence must fit 480 phoneme tokens including boundaries and 12 seconds of predicted audio. Longer sentences return a splitting hint. Each request accepts up to 5,000 characters. The vocoder processes a complete sentence before streaming reconstruction begins.

The web server retains the latest 20 generated WAVs under outputs/web/. Languages whose text dependencies fail to load are disabled in the webapp and reported by the health endpoint.

Repository layout

• engine.py: reusable Python engine.
• synthesize.py: text-to-WAV command.
• serve.py, web/: local webapp and HTTP service.
• runtime/: inference, text processing, PCM conversion and optimized CPU DSP.
• models/: two compiled MPKs, four CPU ONNX graphs, embedding tables and voices.

The ARM64 CPU helper is included. Its source is under runtime/native/; rebuild with bash runtime/native/build.sh on Modalix or in the configured SiMa SDK. Model weights and application code use Apache 2.0; see LICENSE and NOTICE.