aufklarer/MOSS-VoiceGenerator-MLX-bf16

MOSS-VoiceGenerator for MLX Swift

Reference-free speech synthesis from an exact script and a free-form description of the desired voice and delivery. This bundle runs through the native speech-swift MossVoiceGeneratorTTS module. Inference uses no Python subprocess or reference recording.

This is the voice generator, separate from MOSS-Audio analysis models and MOSS-TTS-Nano. It is a speech synthesizer, not an audio judge.

Model

Property	Value
Language network	Qwen3-1.7B backbone, 28 layers, width 2,048
Language checkpoint parameters	2,114,118,656, including extra audio embeddings and untied heads
Synthesis codec parameters	887,217,664 selected source parameters, including weight-normalization factors
Language precision	Original BF16
Codec precision	Original FP32
Quantization	None
Format	Safetensors with converted-bundle manifest
Runtime assets	7,793,146,620 bytes, including verification receipts, excluding this model card
Audio	24 kHz mono, 16 codebooks, 1,920 output samples per frame
Checkpoint context	40,960 language positions
Native input limits	32,768 UTF-8 bytes per text/instruction field; 4,096 tokens in the complete rendered prompt
Default generation limit	750 audio frames
Default sampling	Temperature 1.5, top-k 50, audio top-p 0.6, repetition penalty 1.1

Tensor payloads retain their source precision. The codec encoder and final sixteen unused codebooks are omitted because reference-free synthesis does not use them. Weight normalization is folded in FP32 when loading the Swift runtime. No INT4 conversion is included.

Files

File	Bytes	Purpose
model.safetensors	4,228,278,872	Original BF16 language checkpoint
codec.safetensors	3,548,962,536	FP32 synthesis decoder and first sixteen codebooks
config.json	2,203	Voice-generator architecture and control tokens
codec_config.json	6,601	Pinned audio-tokenizer architecture
tokenizer.json	11,422,691	Qwen text tokenizer
tokenizer_config.json	5,501	Added tokens and tokenizer settings
vocab.json	2,776,833	Tokenizer vocabulary
merges.txt	1,671,853	Tokenizer merges
special_tokens_map.json	631	Special-token definitions
added_tokens.json	704	Model-specific control tokens
mvg_export.json	729	Source revisions, file hashes, sizes and precision
model.safetensors.verified.json	620	Language-file conversion receipt
codec.safetensors.verified.json	5,488	Codec-file conversion receipt
LICENSE	11,358	Apache 2.0 license

The language file SHA-256 is dbe345257ff9f6cc84195bed830a268b39d5e0b728ff3ba90e715150a49b16d4. The codec file SHA-256 is ab37fcc9a7a8f84501f1b7f7d48cc790167ede92ce8ba2a25f8e9d542ff14b5c.

Runtime validation

The native module passes six tiny numerical reference checks for Float32 and BF16 Qwen3 outputs, learned audio padding, cached continuation, causal codec windows, rotary positions, patch ordering and signed weight normalization. Whole-file hashes, tensor shapes, storage types and contiguous payload offsets were verified independently after export.

Five CPU-only cache-ownership tests passed. Loading and model lifetime request a 512 MiB spare-buffer budget through the shared coordinator. Tighter caller limits remain in force and other owners' budgets are restored on release. This allocator setting is not a total process-memory limit: cached bytes can transiently exceed the requested budget by buffer granularity. Generation and unload avoid process-wide cache flushes. Unload releases live model parameters while reusable buffers may remain within the restored shared/caller budget.

Seven real-model tests passed, including synthesis, tokenizer IDs, cancellation before and during generation, recovery, concurrent-request rejection and explicit/concurrent unloading. One native CLI round trip also passed: the synthesis process exited before an independent Parakeet process transcribed its saved WAV and matched the complete nine-word English script.

Performance measurements use Apple M5 Pro with 48 GiB unified memory, a release build and compiled MLX Metal shaders. One English sentence and one Chinese sentence are each repeated four times in a process: one initial request and three warm requests. Warm measurements below are medians of those three requests. Weight files were hashed immediately before loading, so loading used a warm filesystem cache. First PCM and generation time exclude loading and verification.

Input	Audio duration	Initial first PCM	Warm first PCM	Warm generation	Warm real-time factor	Peak physical footprint
English, nine words	3.44 s	561 ms	470 ms	2.098 s	0.610	8.448 GB
Mandarin, two sentences	4.88 s	1,308 ms	457 ms	2.793 s	0.572	8.619 GB

Lower real-time factor is faster. Each language produced identical audio bytes across its four requests. These are short runtime checks and do not establish an instruction-following benchmark score. All four Mandarin outputs matched the expected toneless pinyin sequence. The English timing clip has a recognizer disagreement: Parakeet inserts "and", while Qwen3-ASR returns the exact script. Both results are retained; no claim of zero word error across all clips is made. All eight final WAVs are byte-identical to the earlier profiles from before the cache-ownership change; model math, scripts, options and seeds are unchanged. Live MLX memory fell to 12 bytes after unload in both final processes.

The existing-runtime regression screen produced identical WAV bytes for sixteen seeded before/after pairs; 59 existing runtime source files are unchanged. The 24 planned existing-model timing runs have completed. Some latency results exceeded the check's thresholds. The performance comparison remains inconclusive and does not certify absence of latency regression. Model quality remains experimental and has not yet been scored on InstructTTSEval.

Usage

Build speech-swift from main on Apple Silicon before running the examples below. These engines are not yet included in a tagged release or the Homebrew package.

git clone --branch main https://github.com/soniqo/speech-swift.git
cd speech-swift
make build
export PATH="$PWD/.build/release:$PATH"

make build compiles the CLI and the MLX Metal shaders required for inference. Download this bundle to a local folder before using --moss-voicegenerator-bundle-dir.

speech speak "The little boat crossed the quiet lake before sunset." \
  --engine moss-voicegenerator \
  --moss-voicegenerator-bundle-dir /path/to/model \
  --instruct "A warm adult female voice, speaking slowly and gently." \
  --seed 42 --stream --output speech.wav

Python can invoke the same native executable after it is installed:

from pathlib import Path
import subprocess

bundle = Path("/path/to/model").resolve()
subprocess.run([
    "speech", "speak", "Good morning. Welcome home.",
    "--engine", "moss-voicegenerator",
    "--moss-voicegenerator-bundle-dir", str(bundle),
    "--instruct", "A calm, friendly adult voice.",
    "--seed", "42", "--output", "speech.wav",
], check=True)

Swift applications use MossVoiceGeneratorModel.fromBundle(_:) and generate(text:instruction:options:). The streaming API emits mono PCM chunks and one terminal event after complete-output checks. The model supports cancellation and explicit unloading through ModelMemoryManageable.

Put language, age, accent, gender and delivery requirements in the instruction. No voice is cloned. Script and instruction normalization is disabled by default to preserve exact input; the Swift API can enable the upstream cleanup explicitly. An identical seed is reproducible within this runtime; PyTorch uses a different random-number stream.

Text and instruction must each fit 32,768 UTF-8 bytes. This limit is checked before trimming, normalization, prompt rendering or tokenization. The complete rendered prompt, including instruction and processor tokens, must contain at most 4,096 tokenizer positions before tensor allocation. These native safety limits are separate from the checkpoint's 40,960-position context. Longer inputs throw an input error rather than creating an oversized prefill attention mask.

Sources and license

• MOSS-VoiceGenerator, revision 97521ec2b6f3ec5026ac1f5751f8fc302d82c2d4.
• MOSS-Audio-Tokenizer, revision 3cd226ba2947efa357ef453bcad111b6eafba782.
• OpenMOSS source project.

Both pinned upstream model cards declare Apache 2.0 for their checkpoints, and their processor/model/codec source files carry the Apache 2.0 license and copyright notice. Copyright 2026 OpenMOSS and the HuggingFace Inc. team.

Links

• speech-swift
• Documentation
• soniqo.audio
• Blog
• Speech Studio