aufklarer/MOSS-TTS-Nano-100M-CoreML

MOSS-TTS-Nano-100M Core ML

Compiled streaming Core ML graphs for MOSS-TTS-Nano-100M and the companion MOSS-Audio-Tokenizer-Nano. Generates 48 kHz stereo audio, with optional reference audio conditioning. The host owns SentencePiece tokenization and sampling.

Property	Value
Speech model parameters	117,311,232
Codec inference parameters (weight normalization folded)	21,042,560
Format	Compiled Core ML ML Programs (.mlmodelc)
Precision	Global/codec decoder: FP16 weights/matrix operations, FP32 normalization/rotary phases/residuals; local decoder/encoder: FP32; state: FP16
Graph size	367.6 MB total
Context	1024 prompt + generated rows
Prefill	Up to 256 rows per call; longer prompts use consecutive calls
Audio	48,000 Hz, two channels, 3,840 samples/channel per frame (12.5 frames/s)
Codec tokens	16 codebooks, 1,024 entries each
Deployment	macOS 15+ / iOS 18+; local measurements use macOS only

Files

File	Size	Description
GlobalDecoder.mlmodelc	220.8 MB	mixed_float16_float32 GlobalDecoder
LocalDecoder.mlmodelc	78.7 MB	float32 LocalDecoder
CodecDecoder.mlmodelc	22.6 MB	mixed_float16_float32 CodecDecoder
CodecEncoder.mlmodelc	45.5 MB	float32 CodecEncoder
tokenizer.model	0.47 MB	SentencePiece BPE vocabulary
tokenizer_swift.json	1.01 MB	Original vocabulary, scores and compiled normalization map for native Swift
config.json	Small	Graph contracts, source revisions and measured parity/latency
run_coreml.py	Small	Python synthesis/reference-encoding example
benchmark_coreml.py	Small	Fresh-process memory and first-token/audio benchmark
requirements-runtime.txt	Small	Pinned runtime-only dependencies
LICENSE	Small	Apache License 2.0

The global and codec decoders keep normalization, rotary phases and residual operations in full precision to preserve rotary position accuracy in longer audio. The local decoder keeps all calculations in full precision to avoid drift across local steps. The encoder also uses full precision to protect residual code selection.

Measured checks

These are sequential synthetic graph checks against unmodified upstream PyTorch, including multiple cache updates and two complete local frames. Per-call medians include graph work and Python prediction overhead. They are not an utterance throughput benchmark. Codec calls produce or consume one 80 ms frame.

Machine: macOS-26.6.2-arm64-arm-64bit.

Graph	Compute units	Parity (higher is better)	Median prediction
GlobalDecoder	CPU_ONLY	0.9997798 minimum cosine	8.55 ms
GlobalDecoder	CPU_AND_GPU	0.9999997 minimum cosine	28.67 ms
LocalDecoder	CPU_ONLY	0.9999838 minimum cosine	0.58 ms
LocalDecoder	CPU_AND_GPU	0.9999839 minimum cosine	1.02 ms
CodecDecoder	CPU_ONLY	0.9998652 minimum cosine	3.34 ms
CodecDecoder	CPU_AND_GPU	0.9999966 minimum cosine	9.35 ms
CodecEncoder	CPU_ONLY	100.0% token agreement	3.72 ms
CodecEncoder	CPU_AND_GPU	100.0% token agreement	10.94 ms

Speech smoke checks

38 tests passed, including reference voice cloning, long prompts, rotary positions up to 16,000 and a 10.4-second codec comparison beyond every attention window.

Sample timings below exclude model loading. Time/audio duration below 1 means faster than real time. CPU was faster for this short utterance on the Apple M5 Pro, so the example defaults to CPU_ONLY; --compute-unit CPU_AND_GPU selects GPU execution.

File	Compute units	Audio duration	Time/audio duration	First audio	Whisper tiny.en transcript
samples/hello-gpu.wav	CPU_AND_GPU	2.72 s	0.376	162 ms	Hello from Koi Mill.
samples/hello.wav	CPU_ONLY	1.76 s	0.294	79 ms	Hello from CoreML.
samples/voice-clone.wav	CPU_AND_GPU	3.12 s	0.413	80 ms	The weather is nice today.

These are short smoke checks. The GPU greeting's acronym differs in automatic transcription; the CPU greeting and reference-conditioned sentence match their inputs after case/punctuation/spacing normalization. Full results are in speech_validation.json.

Process memory and token latency

Measured on Apple M5 Pro in a fresh runtime-only Python process per backend. First/warm means the first request and the median of five subsequent requests. Every request has fresh model state. OS/Core ML disk caches were not cleared.

Backend	Peak RSS (MiB)	First audio token, first/warm (ms)	First audio, first/warm (ms)	Warm time/audio duration	Model load (s)
CPU_ONLY	658.0	62 / 44	82 / 57	0.289	0.98
CPU_AND_GPU	635.5	87 / 21	178 / 47	0.422	1.18

Latency includes tokenization, state creation and prefill, and excludes model loading. The first token is the first sampled codec token. First audio is the first decoded 80 ms stereo frame, after all 16 codebooks. RSS includes the complete Python/Core ML worker, with neither PyTorch nor ASR loaded. Lower latency, time/audio duration and RSS are better. CPU generates the full utterance faster; the warmed GPU returns the first token sooner. Full measurements and definitions are in performance.json.

Voice-cloning memory and latency

Same isolated runtime-only Python method, using the saved 2.72-second stereo reference samples/hello-gpu.wav, text The weather is nice today., seed 1. First and warm requests include reading, resampling/padding, loading the codec encoder and encoding the reference. Cached requests reuse reference codes. Warm/cached are medians of five requests each, with fresh generation state. Initial loading of the three synthesis graphs is excluded from latency.

Backend	Peak RSS (MiB)	First token, first/warm (ms)	Cached first token (ms)	First audio, first/warm/cached (ms)	Warm/cached time/audio duration
CPU_ONLY	779.2	542 / 203	52	563 / 216 / 65	0.324 / 0.270
CPU_AND_GPU	731.1	649 / 225	24	741 / 245 / 48	0.471 / 0.400

These measurements exercise the complete reference-conditioning path. They do not measure speaker similarity. CPU and GPU can sample different output durations; compare time/audio duration for throughput. Results describe Python on this Mac, not the native Swift runtime or iPhone. Full data: cloning_performance.json.

Usage

Install coremltools==9.0, numpy==1.26.4, sentencepiece==0.2.1, soundfile==0.13.1, and scipy==1.15.3. Synthesis needs neither PyTorch nor Transformers. Run on macOS with the four compiled directories beside the script:

python run_coreml.py "Hello from Core ML." --output hello.wav
python run_coreml.py "A new sentence." --reference reference.wav --output cloned.wav

from pathlib import Path
import soundfile as sf
from run_coreml import Pipeline

pipeline = Pipeline(Path("."), compute_unit="CPU_ONLY")
audio, codes, report = pipeline.synthesize("Hello from Core ML.")
sf.write("hello.wav", audio, 48000)
print(report)

Reference inputs may be mono or stereo; the example resamples them to 48 kHz and pads the final 80 ms frame. Clone mode uses reference audio without a transcript, matching the released model's prompt format. Use authorized reference recordings. Sampled generation uses a fixed seed of 0, text temperature 1.0, audio temperature 0.8, audio top-k 25, audio top-p 0.95 and repetition penalty 1.2. Use --seed to change the sampled voice/take. --greedy is available for diagnostics; greedy generation can repeat without reaching end-of-speech in the original model too. Generation raises an error if the context or frame limit would truncate speech.

Graph contracts

• GlobalDecoder: input_ids [1,T,17] int32, position [1] int32 -> hidden [1,768] float32. New state per utterance. The first call has position 0; later calls append rows at the next contiguous position.
• LocalDecoder: global_hidden [1,768] float32, previous_token [1] int32, step [1] int32 -> text_logits [1,2], audio_logits [1,1024]. Step 0 clears the local cache and predicts continue/end (IDs 9/7). Steps 1 through 16 predict the audio codebooks in order.
• CodecEncoder: audio [1,2,3840] float32, frame [1] int32 -> codes [1,1,16] int32. New state per reference; frame starts at 0.
• CodecDecoder: codes [1,1,16] int32, frame [1] int32 -> audio [1,2,3840] float32. New state per generated utterance; frame starts at 0.

State shapes and their exported names are in config.json. Keep every state with its utterance/reference. The codec retains the upstream sliding attention windows.

Source

• Requested checkpoint mirror
• Original MOSS-TTS-Nano model
• Companion codec
• Upstream source and architecture

Links

• speech-swift — Apple speech SDK
• Docs — installation and CLI documentation
• soniqo.audio
• Blog
• Speech Studio — local speech generation and voice cloning