acidsound/scyllasband

Scylla's Band: speech and singing

🇰🇷 한국어 지원 (Korean Speech & Singing Support)

본 저장소는 **Scylla's Band (v4-20261106)**의 초경량·고속 로컬 추론 및 멀티플랫폼 호환성을 온전히 유지하면서, 한국어(ko / ko_kr) 음성 합성(Speech)과 노래 합성(Singing)을 완전하게 지원하도록 확장된 엔드투엔드 모델 및 런타임 환경입니다.

🌟 주요 작업 및 구현 내용 (Key Features)

1. 한국어 텍스트 정규화 (Context-Aware Korean Text Normalization)

   • 고유어/한자어 수사 시스템 완전 분리: 25종 이상의 순우리말 분류사(개, 명, 분, 살, 마리, 권, 병, 잔, 송이, 조각, 켤레, 채, 대, 바구니, 봉지, 번, 시간, 달 등) 감지 시 고유어 수사(하나/한, 둘/두, 셋/세... 스물/스무)로 자동 전환되며, 일반 수량·연산·서수는 한자어(영, 일, 이, 삼... 십, 백, 천, 만, 억, 조)로 정확하게 정규화 (10 ➔ 십, 15,000 ➔ 만 오천, 100,000,000 ➔ 일억).
   • 구조화된 엔티티 및 기호 음차: ISO 날짜(2026-11-08 ➔ 이천이십육년 십일월 팔일), 시간(오후 3:30 ➔ 오후 세 시 삼십 분), 다양한 통화(₩15,000 ➔ 만 오천 원, $3.50 ➔ 삼 달러 과 오십 센트, 40€, £20, ¥500), 소수점, 백분율, 온도(-5°C ➔ 마이너스 오 도씨), 수학 기호 및 주요 영문 약어(AI ➔ 에이아이, TTS ➔ 티티에스, GPU, MP3, 4K 등) 처리.
   • Python & Native C++ 100% 동일 구현: Python text_normalizer.py와 Native C++ scyllasband_text.cpp에서 동일한 규칙을 완벽하게 구현하여 런타임 간 불일치를 차단했습니다.

2. 국립국어원 표준 발음법 기반 규칙형 G2P 프론트엔드 (Standard Korean Phonological G2P)

   • 체계적인 음운 변동 규칙 적용:
     • 구개음화 (제17항): 같이 ➔ [가치], 굳이 ➔ [구지], 닫히다 ➔ [다치다]
     • 연음 법칙 및 겹받침 분해 (제13~14항): 한국어 ➔ [한구거], 읽어 ➔ [일거], 없어 ➔ [업써]
     • ㅎ 축약 및 탈락 (제12항): 좋다 ➔ [조타], 입학 ➔ [이팍], 좋아요 ➔ [조아요]
     • 음절의 끝소리 규칙 & 겹받침 축약 (제8~11항): 27개 받침의 7개 대표음(k, n, t, l, m, p, ŋ) 중화, 맑고 ➔ [말꼬], 맑다 ➔ [막따]
     • 유음화 (제20항): 신라 ➔ [실라], 설날 ➔ [설랄]
     • 비음화 (제18~19항): 국물 ➔ [궁물], 독립 ➔ [동닙], 있는 ➔ [인는]
     • 된소리되기 (경음화, 제23항): 학교 ➔ [학꾜], 식당 ➔ [식땅]
     • 문맥적 모음 및 음향 폐쇄 실현: ㅡ(ɯ/ə), ㅢ의 위치별 실현(의사 ➔ ɯ i, 희망 ➔ i, 나의 ➔ e), 동일 조음 위치 파열음 폐쇄 병합.
   • 어휘집(Vocabulary) 100% 호환성 유지: 기존 87개 고정 Phone Vocab(phone_vocab.json) 범위를 엄격하게 준수하여 재학습 없이도 모든 한국어 음소가 정확한 IPA 토큰으로 변환됩니다.

3. 최적의 어쿠스틱 컨디셔닝 탐색 (Acoustic Conditioning)

   • 기존 모델의 14개 음성 × 7개 언어 임베딩 중 음절 기반(Syllable-timed) 순수 모음 특성을 지닌 Romance 어쿠스틱 컨디셔닝(language_id = 3)을 매핑하여, schwa 축약이나 엉뚱한 성조 왜곡 없이 14개 전 음성에서 가장 또렷한 한국어 모음 포먼트와 자연스러운 발화 타이밍을 확보했습니다.

4. 한국어 노래 합성 지원 (Korean Singing Voice Synthesis)

   • 악보(JSON score: 가사, 음계, 박자) 기반 한국어 가창 완벽 지원 (melodies/phrase_ko_twinkle.json).
   • Vocos(초고속 음성/가창) 및 BigVGAN v2(고품질 가창) 듀얼 디코더 지원.

5. 형식 검증(TLA+) 및 완벽한 품질 검증 (Formal Verification & Quality Assurance)

   • TLA+ 정형 기법 검증: 음운 변환기 및 레이아웃/스트리밍 라이프사이클에 대한 TLA+ 명세를 작성하고 상태 공간 모델 검사를 수행하여 34,276개 상태 전수 검사에서 0개의 invariant 위반 입증.
   • 네이티브 C++ & 파이썬 100% 패리티 (Bit-exact Parity): CTest 16/16 전원 통과, Pytest 131/131 전원 통과, C++ 및 Python 간 프론트엔드/추론 오차 0.0 (SNR 무한대) 달성.
   • 음성 인식(ASR) 검증: faster-whisper (large-v3-turbo) 평가 결과, 주요 한국어 문장에서 CER 0.0%의 정확한 음소 전달력 확인.
   • 로컬 실시간 속도: C++ 네이티브 추론(scyllasband_speak.exe) 기준 CPU에서 약 18.9배~42.9배 실시간 속도(RTF 0.02~0.05) 기록.

6. 원클릭 웹 UI (webui.py) 제공

   • 브라우저에서 14가지 음성, 억양/속도/음높이 조절, 한국어 음성 합성 및 노래 합성, 텍스트 정규화와 G2P 음운 변환 결과를 한눈에 확인하고 청취할 수 있는 Gradio 기반 WebUI 탑재 (python webui.py).

7. 서버 없는 독립형 WebGPU 데모 (webgpu_demo/)

   • 별도의 Python 백엔드나 모델 서버 없이, 사용자의 웹 브라우저에서 **WebGPU (ONNX Runtime Web)**를 통해 100% 클라이언트 사이드로 동작하는 스탠드얼론 데모 탑재.
   • webgpu/ 번들의 8-bit INT8 가중치(MatMulNBits)와 4 Euler steps를 활용하여 브라우저 내부에서 실시간 배율을 능가하는 고속 추론 수행.
   • 순수 JavaScript로 작성된 한국어 텍스트 정규화 및 국립국어원 표준 발음법 G2P 음운 변환 엔진(korean_g2p.js) 내장.
   • 브라우저 Cache API를 적용하여 743MB의 공유 가중치를 최초 1회만 캐싱하고 이후 즉각적인 오프라인/로컬 로딩 지원.
   • 로컬 원클릭 실행: python serve_webgpu_demo.py 또는 GitHub Pages / Hugging Face Spaces 정적 호스팅 가능.

---

Release v4-20261106: Scylla's Band speaks and now sings. Speech and singing share one model: one flow renders both, selected by a delivery mode, with two waveform decoders. The runtime (C ABI 3.1.0, with the sing command and the v4 graph contract) runs it on Linux, Windows, macOS, iOS, visionOS and Android. The previous release, v3-20261104, stays available at its tags (v3-20261104, v3-20261104-apple) for runtimes built for v3.

What changed from v3-20261104:

• Singing. Pop and ballad delivery from a melody score (lyrics, notes and beats as JSON) in all fourteen voices. The flow is v3's distilled flow continued by reflow on singing material: spoken output starts identical to v3 and the sung and ballad paths are distilled from a singing-trained teacher.
• Singing at the speech cost (v4-20261106). The sung and ballad paths are distilled a second time from the teacher's classifier-free-guided field (melody and mode against both switched off, guidance 2), so the flow reproduces the guided answer unguided: 4 Euler steps and one evaluation per step, as for speech, against 8 Heun steps with guidance (32 evaluations) for the first singing candidate (v4-20261105, not published here). Lyric clarity and pitch accuracy are unchanged (below).
• Second decoder. Sung targets render through a BigVGAN v2 decoder fine-tuned on the voices; speech keeps the Vocos decoder. The runtime picks the decoder by delivery mode.
• Graph contract scyllasband_measured_controls_v4. The flow graphs take one more input, delivery_mode_id. Runtimes built for v3 refuse this bundle and keep downloading v3-20261104-apple, which stays available here.

Delivery for speech is unchanged: intensity, pace and pitch, each from −2 to 2.

Quick start

Install the runtime (pip install -e . from a checkout), then:

python -m scyllasband download --flavor litert
python -m scyllasband speak --voice ariadne --language en_us --intensity 1 --pace -0.5 \
    -o hello.wav "It is a pleasure to meet you."
python -m scyllasband speak --voice scylla --language ko \
    -o hello_ko.wav "안녕하세요, 오늘 날씨가 정말 좋네요."
python -m scyllasband sing --voice gwen --score scyllasband/models/melodies/phrase_pop_00.json -o ring.wav
python -m scyllasband sing --voice scylla --language ko --score melodies/phrase_ko_twinkle.json -o twinkle_ko.wav

# 서버 없이 브라우저 WebGPU로만 동작하는 스탠드얼론 데모 실행 (기본 0.0.0.0:8000)
python serve_webgpu_demo.py --host 0.0.0.0 --port 8000

download installs the bundle and the example scores (melodies/) from this repository at the release tag. On macOS --flavor coreml or --flavor coreai fetch the Apple bundles. python -m scyllasband build-native builds the native runtime that the commands then use (much faster and lighter than the Python engine, the same plans).

Singing

A song is a JSON score: the lyric text, a tempo, a style and the notes in order, each [label, pitch, beats].

{
  "text": "give me a ring baby make me feel so",
  "bpm": 108,
  "style": "pop",
  "notes": [
    ["give", "C4", 0.5], ["me", "D4", 0.5], ["a", "E4", 0.5], ["ring", "G4", 1.0],
    ["-", "rest", 0.5],
    ["baby~1", "E4", 0.5], ["baby~2", "D4", 0.5],
    ["make", "C4", 0.5], ["me", "D4", 0.5], ["feel", "E4", 1.0], ["so", "C4", 1.5]
  ]
}

• style is pop (the sung delivery token; the pop material was ingested under it) or ballad.
• A word over several notes is numbered (baby~1 baby~2, also for a syllable held over two pitches); rests use the label -.
• Phrases split at rests of a quarter second or more and render one at a time; the runtime shifts the whole song by octaves into the voice's sung range (--no-register keeps the written octave).
• Sung targets take the melody from the score, skip the pitch predictor and the spoken pitch limits, and render with 4 Euler steps and no guidance through the BigVGAN decoder (--decoder vocos selects the speech decoder; --steps, --sampler, --guidance override the manifest defaults). The model never free-sings: a score, written by the user or by a planner, is required.

The melodies/ folder holds twenty example scores: four whole songs (two pop, two ballads) and sixteen phrases.

Controls

Control	Range	Default	Effect
Intensity	−2 to 2	0	Calm (negative) to intense (positive): how much the voice puts into the line
Pace	−2 to 2	0	Slower (negative) to faster (positive): the words and the pauses between them
Pitch	−2 to 2	0	Shifts the voice's pitch, in semitones

0 is each voice's own delivery. Intensity and pace are measured in each voice's own range, in standard deviations around its average, so the same value is a comparable change for every voice and language. An angry line sounds angrier at higher intensity and more restrained at lower intensity; intensity changes how much of the emotion comes through, not which emotion it is. Requests outside the range are clamped into it, and unspecified controls take the voice's default (controls.delivery.defaults; 0 for every voice in this release).

How the runtime applies them:

• The duration and flow graphs take intensity and pace as value / 5 in delivery slots 0 and 1, with presence bits; the other slots are always absent.
• Pace also scales the pauses between words by exp(−0.13 × pace) (controls.pace_pauses).
• A passage with fewer than four spoken phones, such as a lone "Yes.", keeps the voice's own intensity and pace, as the model was trained (controls.delivery.min_spoken_phones). Its pitch shift still applies.
• Pitch shifts the predicted contour in semitones; a soft limit keeps it inside the voice's measured range (controls.pitch.edges).

Controls are global within each request. This release does not support control curves or emotion labels; singing takes its melody from a score (see Singing).

Model and inference

Property	Value
Public model version	4 (the v3 speech models, with a delivery mode added to the flow for singing)
Architecture	Phone-duration predictor → pitch predictor (rectified flow) → continuous rectified-flow latent generator that follows the pitch contour → acoustic adapter/Vocos
Audio	Mono 24 kHz; 100 mel features; 24 latent channels
Mel / latent hop	256 / 512 samples
Duration model	Hidden size 320; 8 layers; reads the neighbouring text as context
Pitch predictor	Hidden size 384; 8 layers; a pitch and voicing contour per latent frame
Vector model	Hidden size 512; 12 layers; adaptive layer normalization and QK normalization; reads the pitch contour
Maximum graph phones per sentence	512
Trained target length	64–420 latent frames (about 1.4–9 s) per pass
Sampler	Flow: Euler, 4 steps. Pitch predictor: Euler, 4 steps
Voice conditioning	Fixed per-voice/per-locale identity and prosody references embedded in the graphs
Text frontend	Trained Scylla's Band G2P with its matched tokenizer and vocabulary

For each passage the pitch predictor integrates a pitch and voicing contour from noise, the pitch control shifts it, and the flow generates the latents along it (graph input expanded_melody, [1, frames, 3]). Each passage draws its pitch noise ([2, frames]) and then its latent noise ([24, frames]) from the request's seed.

The flow and the pitch predictor are distilled for few-step sampling: each was trained to reproduce its base model's 32-step Heun output, so Euler with 4 steps, four evaluations of each model per passage, is the default for both.

Long text is spoken as passages within the trained target lengths (64 to 420 latent frames, about 1.4 to 9 seconds): a short first passage so audio starts quickly, then passages as long as the text allows, cut at a sentence end, else a clause, else between words, and measured with the requested delivery. Each passage sees the surrounding text as context and continues from the acoustics of the one before it, and consecutive passages are decoded as one continuous waveform. Durations are predicted deterministically; different controls can alter them.

Performance

Measured with the default samplers (flow and pitch predictor: Euler, 4 steps each) through the native runtime: scyllasband_speak from libscyllasband on the Mac and Android, and the iOS sample's benchmark mode on the iPhone. The text is three sentences, 10.3 s of audio: "The last train leaves at midnight, so pack light. Bring a warm coat, and don't forget the map. If we miss it, we walk, and nobody wants to walk that far in the rain." (voice Scylla, seed 7, the voice's own delivery).

• First run: from launch to first audio on the first launch after installing the bundle. Core ML compiles and Core AI specializes the graphs for the device then (Core AI caches them until the next OS update); LiteRT writes its packed weights to a cache file beside each graph.
• Cold start: from launch to first audio on a later launch, including loading the bundle.
• Warm first audio: from a request to its first audio in a process that has already spoken (median of three requests).
• Real-time speed: seconds of audio generated per second when warm, rounded.

Device	Bundle	Compute	First run	Cold start	Warm first audio	Real-time speed
MacBook Pro, M5 Max (macOS 27)	Core ML	GPU (default)	2.73 s	1.34 s	30 ms	~170×
	Core ML	CPU	2.17 s	0.33 s	221 ms	~20×
	Core AI	GPU (default)	5.80 s	0.46 s	32 ms	~150×
	Core AI	Neural Engine	34.2 s	0.60 s	316 ms	~20×
	Core AI	CPU	3.54 s	0.93 s	844 ms	~5.0×
	LiteRT	CPU	0.41 s	0.30 s	105 ms	~34×
	ONNX Runtime	CPU	0.88 s	0.85 s	216 ms	~21×
iPhone 17 Pro Max (iOS 27.2)	Core ML	GPU (default)	15.5 s	1.79 s	77 ms	~65×
	Core ML	CPU	2.87 s	0.49 s	294 ms	~15×
	Core AI	GPU (default)	18.5 s	0.59 s	70 ms	~68×
	Core AI	Neural Engine	8.00 s	0.74 s	367 ms	~17×
	Core AI	CPU	7.58 s	1.16 s	1010 ms	~4.1×
Galaxy Z Fold 8, Snapdragon SM8850 (Android 17)	LiteRT	CPU	0.81 s	0.52 s	120 ms	~27×
	ONNX Runtime	CPU	1.85 s	1.89 s	461 ms	~9.2×

The GPU is the default on Apple platforms. The Neural Engine placement runs the flow and the pitch predictor on the Neural Engine and the other graphs on the CPU, for apps that speak in the background, where iOS does not allow GPU work. Core ML reloads its graphs on every launch, which accounts for most of its cold start on the GPU. Each passage runs the pitch predictor 4 times, then the flow 4 times. Against v3-20261103, whose pitch predictor ran 16 times, warm first audio is 30–45 % shorter on every device and bundle (12–13 % with the Neural Engine placement).

Singing cost

Singing is heavier than speech because the BigVGAN decoder is about 90 times the work of Vocos per second of audio; from v4-20261106 the flow itself costs the same as for speech (4 Euler evaluations per phrase, where v4-20261105 ran 8 Heun steps with guidance, 32 evaluations). --decoder vocos ("turbo") sings through the speech decoder: much faster, with some artefacts on held notes in some voices. Phrases render one at a time and the runtime streams each as it finishes, so first audio is the first phrase (up to 10 s of song).

Measured with v4-20261106 (v4-20261106-apple on Apple platforms) on the 71 s pop example (melodies/song_pop_gwen.json, voice Gwen, seed 7) through the native runtime, warm; first audio in brackets:

Device	Bundle	Compute	BigVGAN: time (× real time)	Vocos: time (× real time)	Peak memory (BigVGAN)
MacBook Pro, M5 Max	Core ML	GPU	0.98 s (73×) [0.14 s]	0.14 s (510×)	1.0 GB
	Core AI	GPU	1.3 s (56×) [0.18 s]	0.16 s (440×)	1.1 GB
	Core AI	Neural Engine	1.6 s (45×) [0.22 s]	0.56 s (126×)	1.2 GB
	LiteRT	CPU	20.2 s (3.5×) [2.9 s]	1.2 s (59×)	1.1 GB
iPhone 17 Pro Max	Core AI	GPU	12.5 s (5.7×) [1.8 s]	0.95 s (74×)	920 MB
	Core AI	Neural Engine	14.0 s (5.1×) [2.0 s]	0.9 s (80×)	850 MB
	Core ML	GPU, BigVGAN on the CPU	31 s (2.3×) [4.5 s]	0.9 s (78×)	300 MB
Galaxy Z Fold 8	LiteRT	GPU for BigVGAN (AUTO)	19.2 s (3.7×) [1.9 s]	1.7 s (42×) [0.26 s]	764 MB
	LiteRT	CPU	71 s (1.0×) [9.5 s]	1.7 s (42×)	1.85 GB

With v4-20261105 the same song took 15.7 s (Core AI GPU) and 4.8 s (Turbo) on the iPhone, and 33 s and 15.6 s on the Fold.

• On iOS and visionOS the Core ML bundle runs BigVGAN on the CPU (controls.coreml.compute_units_ios): Core ML's GPU path for its largest bucket runs out of an iPhone's GPU memory, which aborts the app. Core AI runs it on the GPU, or on the Neural Engine with that placement, so songs can render in the background.
• The LiteRT bundle names BigVGAN as its one GPU graph (controls.litert.gpu), so the AUTO accelerator sings through the GPU and speaks on the CPU; on the Fold the first launch after an install compiles the GPU program (first audio about 5 s), later launches load it from a cache beside the bundle.
• On v4-20261105, Vocos with 4 Heun steps and no guidance (--steps 4 --guidance 1) sang the same song in 3.0 s on the Fold's CPU (24×); it kept more of the lyric (word coverage 98 % against 92 % over 14 voices and 3 phrases) and held pitch a little less tightly (96.7 % of notes within a semitone against 99.1 %). That trade-off is what the v4-20261106 distillation removes: the unguided 4-step flow keeps both the clarity and the pitch, at 1.7 s for the song.

Voices and language coverage

Fourteen voices: Ariadne, Felix, Gwen, Ink, Kincade, Max, Orpheus, Poppy, Priya, Rex, Scylla, Stone, Thistle, Tuesday.

All fourteen have Spanish (es), Italian (it), French (fr), German (de), Vietnamese (vi) and Korean (ko). Ink, Orpheus and Tuesday use British English (en_gb); the other eleven use American English (en_us). This is 98 supported voice/locale pairs (84 natively trained pairs plus 14 Korean pairs routed through the Standard Korean phonological G2P frontend and syllable-timed acoustic conditioning index 3).

Artifacts

Every bundle carries the BigVGAN decoder (112 M parameters) beside the Vocos decoder. Tags: v4-20261106 and v4-20261106-apple both name this release with every bundle below; download uses v4-20261106-apple, following the earlier releases' naming. Earlier releases keep their tags (v3-20261104, v3-20261104-apple, ...).

• litert/ (471 MB, of which the BigVGAN decoder is 253 MB): LiteRT bundle for Linux, Windows, Android and Intel Macs. Dynamic-range INT8 weights on the fully connected, convolution and embedding layers of the speech graphs as in v3; the Vocos vocoder's depthwise convolutions stay in full precision. Latent buckets every 64 frames up to 512, then 768 and 1024; flow graphs without the 96 prefix frames for a request's first passage; narrower duration, span-context and pitch-context widths and G2P widths beside the full ones. BigVGAN (vocoder_bigvgan.tflite) is one 256-frame graph with FP16 weights, written channels-last so it has no layout shuffles and runs on LiteRT's GPU delegate; the runtime decodes longer phrases through it in windows with 16 frames of overlap either side of each cut (controls.decoders.bigvgan.window; audio within 68 dB of whole-phrase decoding). Against v4-20261105's first LiteRT bundle (one full-precision BigVGAN graph per bucket, 524 MB), with the v4-20261105 flow, it rendered the 71 s example song in 90 s instead of 165 s on the Fold's CPU at half the memory, and in 33 s on its GPU; with the v4-20261106 flow, 71 s and 19 s. Its decoder window needs a runtime with ABI 3.1.0 (native or Python engine).
• coreml/ (406 MB): Core ML bundle for iOS 18, visionOS 2 and macOS 15 and later. FP16 ML Programs; BigVGAN in buckets of 128 to 512 frames, decoded whole (no window); on iOS and visionOS it runs on the CPU (controls.coreml.compute_units_ios).
• coreai/ (418 MB): Core AI bundle for iOS, visionOS and macOS 27 and later. BigVGAN is listed in neural_engine_assets: it runs on the Neural Engine with that placement.
• onnx/ (1.1 GB): the full-precision ONNX bundle, for ONNX Runtime on the CPU and the reference for the release checks and the Apple exports: one shared weight file, latent buckets of 128, 256, 384, 512, 768 and 1024 frames for the flow, the pitch predictor and both decoders.
• webgpu/ (782 MB): ONNX Runtime bundle for ONNX Runtime Web's WebGPU backend and for NVIDIA GPUs (CUDA). Transformer weights are INT8 with float compute (MatMulNBits), which runs entirely on the GPU; both decoders stay in full precision (BigVGAN is 450 MB of the bundle).
• pytorch/ (1.1 GB): duration, pitch predictor, flow, Vocos adapter/vocoder, BigVGAN adapter and BigVGAN v2 (bigvgan_adapter.pt, bigvgan_v2.pt) and G2P weights with their model configurations (no training state or training settings), and the release configuration assets (scyllasband_pytorch_release_v4).
• melodies/: twenty example scores (four whole songs, sixteen phrases) with a README on the format.
• SHA256SUMS.json: published-file hashes.

Every runtime bundle includes the G2P model in a matching precision.

Validation and limitations

Release validation separates graph accuracy, frontend/timing behaviour and perceived speech quality:

• The full-precision ONNX bundle run through the runtime against the PyTorch model, end to end with all three controls (Scylla, Ink and Gwen, two targets each): identical durations, and latents within 8.5e-5 (values of magnitude 16–28), the same as v3.
• The C++ runtime against the Python runtime on the same bundle: identical G2P output, phone durations and passage plan; latents within 2e-5 relative error; decoded audio at 46 dB SNR (worst sentence) and 55 dB over a long passage.
• The distilled flow at 4 Euler steps compared with the undistilled flow by listening on paired lines across the voices, and by ASR word error rate and a MOS predictor on a 168-clip sweep of all fourteen voices in English, French and Spanish.
• The distilled pitch predictor at 4 Euler steps compared with the undistilled predictor at 8 Heun steps (release v3-20261103) by blind listening on paired lines across the voices and languages, and on the same 168-clip sweep: ASR word error rate 0.062 for both, and a MOS-predictor score 0.12 higher.
• Quantized bundles compared with full precision through the runtime on lines covering every voice, each control at its limits and each language. The WebGPU bundle reproduces the full-precision take (identical frame counts on 15 of 17 lines, 0.68 dB median mel-cepstral distance, against 29 dB for the full-precision bundle at another seed; a sung pop phrase comes out the same length with the same pitch). Dynamic INT8 moves some phone durations by a frame, which redraws a passage's noise and gives a different take, so the LiteRT bundle is judged on the same sweep (measured on v3's speech models): ASR word error rate 0.062 against 0.062 for full precision, and a MOS-predictor score within 0.02 of full precision.
• Every LiteRT graph gives identical output with and without XNNPACK's weight cache.
• Korean (ko / ko_kr) frontend and acoustic synthesis. Korean text normalization (Sino-Korean and Native Korean numbers with 25 counter classifiers, dates, times, currencies, fractions, percentages, symbols and mixed Latin/Hangul tokens) and Standard Korean (표준발음법) phonological G2P (liaison, palatalization, nasalization, liquidization, h-aspiration/elision, post-obstruent tensification and complex coda reduction) are implemented identically in the Python and C++ runtimes, formally verified with TLA+ across 34,276 reachable states. On the LiteRT bundle, Python and C++ produce identical normalized text (15/15), identical phone streams (30/30), identical phone durations and song phrase plans, 0.0 relative latent error, and infinite SNR audio parity, running at ~18.9× real-time for speech (138 ms warm first audio) and ~42.9× for Vocos singing on CPU. Because the acoustic graphs were not trained on native Korean speech corpora, Korean synthesis reuses syllable-timed acoustic conditioning index 3 and the 87-symbol IPA phone inventory, so some non-native vowel/prosodic coloration remains on complex consonant clusters.
• Core ML and Core AI bundles compared with the PyTorch model on every graph and through the runtime on 14 passages (every voice, each control at both limits, six languages, narration), on each compute unit they are used with (GPU, CPU, and for Core AI the Neural Engine): identical phonemes, fewer changed phone durations than the LiteRT bundle (0.2–0.7 % against 1.9 %), and smaller pitch-contour (0.016–0.045 semitones RMS against 0.051), latent and spectral differences. The Apple exports rewrite three computations into equivalent forms that FP16 and the Neural Engine evaluate exactly (span pooling, the modifier bit decoding, the duration model's combined phone/segment index), and scale the residual stream of the flow and the pitch predictor; in FP32 the rewritten graphs match the trained ones (identical outputs; the span encoders within 1e-6).

Listening is decisive; automatic scores support it. Training audio is not distributed.

Known problems include occasional pronunciation errors and occasional roughness on fast, wide pitch movements in some voices.

The system supports managed voices, not arbitrary speaker cloning. Intended uses include local speech, narration, character dialogue and delivery-control research. Do not use it for impersonation or deceptive attribution of speech.

Singing validation

The singing path was validated on the PyTorch model through the release renderer (adapter + BigVGAN v2) and on the built bundles through the runtime's sing command:

• Score tests, all fourteen voices (v4-20261105 flow, 8 Heun steps, guidance 2). Eight hand-written pop phrases and eight ballad phrases per voice (lyrics and notes, each phrase transposed into the voice's register), scored by Whisper lyric precision and coverage and by the sung pitch against the score (per-note median absolute error in cents, share of notes within 100 cents):

  Mode	Lyric precision (mean / lowest voice)	Coverage (mean)	Pitch error (median, cents)	Notes within 100 cents
  Pop (sung)	0.981 / 0.933 (Orpheus)	0.976	15	1.000 (every voice)
  Ballad	0.981 / 0.934 (Rex)	0.972	11	1.000 (every voice)

  Eleven voices score 0.96 or better in both modes; Orpheus, Poppy and Rex are the lowest at 0.93–0.97.

• Shared flow against v3. On the four check voices (Felix, Gwen, Max, Scylla) the reflow-distilled release flow matches the undistilled singing teacher's lyric precision (0.987 against 0.987). Spoken output was compared with the v3 release flow on the 17 release check lines (every voice, each control at its limits, six languages): the same durations, a median mel-cepstral distance of 2.4 dB, and no audible difference in paired listening. Distilling the spoken rows from the v3 base (a second teacher) rather than from the singing teacher brought that distance down from 3.4 dB.

• Guidance distillation (v4-20261106). The second distillation was scored on eight pieces (four phrases, the two example songs, the Ghost in the Machine chorus and a sixteenth-note patter at 112 bpm) rendered through the full-precision ONNX bundle with the same seed, against the v4-20261105 flow at its shipped and its fast settings:

  Flow and setting	Flow evaluations	Lyric precision	Pitch error (median, cents)	Notes within 100 cents
  v4-20261105, 8 Heun steps, guidance 2 (shipped)	32	0.979	9.4	0.993
  v4-20261105, 4 Heun steps, no guidance	8	0.986	15.0	0.966
  v4-20261106, 4 Euler steps, no guidance (default)	4	0.969	9.4	0.991
  v4-20261106, 4 Heun steps, no guidance	8	0.960	9.4	0.992

  The v4-20261106 word scores carry one Whisper insertion ("and" before a Scylla phrase, also at step 8000 of the distillation, where the set scored 0.984) rather than a dropped lyric; pitch matches the guided setting. On the sung validation set the distilled flow's latent output at 4 Euler steps sits 0.547 (RMS) from the teacher's guided 32-step Heun output, against 0.726 for the v4-20261105 flow at 8 Heun steps with guidance. Spoken output moved by a median mel-cepstral distance of 2.3 dB from v4-20261105 on the 17 release check lines (2.7 dB from v3, against 2.4 dB for v4-20261105; the same durations), and export parity against PyTorch is unchanged (worst 8.5e-5).

• Renderer. Blind paired listening on Scylla, Felix, Max, Gwen and Poppy preferred the fine-tuned adapter + BigVGAN v2 renderer over the stock BigVGAN v2 and over Vocos for sung targets; Vocos adds harmonic artefacts on sustained notes in some voices (Scylla most), which the BigVGAN path does not.

• Built bundles through the runtime. The sing command on the v4-20261106 LiteRT bundle at its defaults (4 Euler steps, no guidance, windowed BigVGAN), every voice on one pop and one ballad phrase and the four example songs, scored the same way: pop phrase lyric precision 1.00 for all fourteen voices (median pitch error 10 cents, coverage 1.00); ballad phrase 0.92 mean (0.875 lowest, one inserted leading syllable on the long first note of that phrase, coverage 1.00); songs 0.92–0.98 precision with 9–19 cents median pitch error and 95–100 % of notes within 100 cents. The v4-20261105 bundle at 8 Heun steps with guidance scored 1.00 / 0.89 (0.67 lowest) / 0.87–0.94 on the same set. The same phrases through the full-precision ONNX bundle give identical scores, and the LiteRT take sits 1.4 dB (mel-cepstral) from the full-precision take against 27–32 dB for a different seed: the dynamic-INT8 flow reproduces the full-precision singing take.

• Whole songs. Four example songs (melodies/song_*.json, 2 pop and 2 ballads, 20–40 s) render phrase by phrase through the runtime; on the PyTorch path through the singing teacher they scored a lyric precision of 0.90–0.93.

Known limitations of the singing path: the model never free-sings (a score is required); a rap delivery row was trained but is not exposed as a style in this release; the runtime transposes a score into the voice's register by whole octaves only; a syllable held over many notes is rendered as one vowel across them; and Orpheus, Poppy and Rex drop the occasional word.

• Core ML and Core AI bundles (v4-20261106-apple). Against the FP32 PyTorch model on 20 sung cases (every voice on a pop phrase, four ballads, two more), with the same noise: identical phrase plans (20/20) on every compute unit; flow latents within 0.9–1.2 % (LiteRT 1.8 %); BigVGAN within 0.017–0.019 log-mel on the GPU and Neural Engine and 0.090 on Core ML's CPU path; end to end 0.034–0.10 (a reseeded take of the same score is 0.90 away). The release's sing check on both bundles (every voice on pop and ballad phrase 02, the four songs; faster-whisper and pYIN): lyric coverage 1.00 on all 28 phrases, median pitch error about 10 cents, songs 0.87–1.00 coverage with 95–100 % of notes within 100 cents. Speech stays closer to FP32 than the LiteRT bundle on every Apple compute unit, as for v4-20261105.

License and acknowledgments

Apache 2.0. LiteRT and ONNX Runtime provide inference; Vocos (charactr/vocos-mel-24khz) is the base of the 24 kHz speech waveform decoder, and BigVGAN v2 (nvidia/bigvgan_v2_24khz_100band_256x, NVIDIA, MIT) is the base of the singing waveform decoder; both are fine-tuned on the voices, with notices in THIRD_PARTY_NOTICES.md. DeepPhonemizer-derived tooling, Wiktionary-derived vocabulary, eSpeak phonemization and Montreal Forced Aligner contributed to the frontend and alignment workflow. Vocos and DeepPhonemizer are MIT-licensed; their notices are in THIRD_PARTY_NOTICES.md.