VoxCPM2-ONNX-WebGPU

OpenBMB's VoxCPM2 text-to-speech model (2B parameters, 30 languages, 48 kHz) as ONNX graphs with quantized weights for running in a web browser with ONNX Runtime Web's WebGPU backend, as used by Quick Inference. It writes continuous audio latents, a patch of four frames at a time, with no audio tokens.

Changes from the original

Exported from the voxcpm package (OpenBMB/VoxCPM, Apache 2.0) by tools/export/export_voxcpm.py in Quick Inference, split into the graphs below, and quantized without calibration (round to nearest, symmetric, blocks of 32): matrix multiplications to 4-bit, the AudioVAE decoder's convolutions to 8-bit. The float exports matched PyTorch before quantization. Only generation without a reference voice is exported: no voice cloning.

These are modified files: their outputs are close to, but not identical to, the original model's. On four test sentences (classifier-free guidance 2, 10 timesteps), Whisper small.en heard them with an 11.9% word error rate (ONNX Runtime CPU, and the same in Chrome) against 4.8% for the original float32 model; the slips are mostly in numbers ("555-142", number formatting). The voice is the model's own choice with no reference voice: it can change from one generation to the next.

Files

All graphs take and give float32.

File	Inputs → outputs
onnx/text_embed_q4.onnx	ids int64 [1, S] → embeds [1, S, 2048]
onnx/base_lm_q4.onnx	MiniCPM-4, 28 layers (LongRoPE): inputs_embeds [1, S, 2048], past_key_values.{0..27}.{key,value} [1, 2, P, 128] → hidden [1, S, 2048] (after the final norm), present.*; positions continue from the cache
onnx/residual_lm_q4.onnx	The 8-layer residual LM (no RoPE), the same inputs and outputs
onnx/bridge_q4.onnx	hidden, feat_embed [1, S, 2048], audio_mask [1, S] (1 on audio positions) → lm_hidden, residual_input [1, S, 2048], stop_logits [1, S, 2]
onnx/feat_encoder_q4.onnx	feats [1, T, 4, 64] → embeds [1, T, 2048]
onnx/dit_step_q4.onnx	One local-DiT Euler step: x [1, 64, 4], lm_hidden, residual_hidden [1, 2048], cond [1, 4, 64] (the previous patch), t [1], cfg_value [] → v [1, 64, 4] (classifier-free guidance with CFG-zero*)
onnx/vae_decoder_q4c8.onnx	latents [1, 64, F] → audio [1, 1, 1920 F] at 48 kHz

tokenizer.json, tokenizer_config.json, tokenization_voxcpm2.py, config.json: the original's.

Generation (VoxCPM2Model._inference, no reference voice)

1. ids = the text's tokens (no BOS; tokens made only of Chinese characters split into single characters, as the voxcpm package's mask_multichar_chinese_tokens) + [101] (audio start). At most min(6 × tokens + 10, 2000) patches.
2. raw = base_lm(text_embed(ids)); lm_h, res_in, stop = bridge(raw, zeros, mask 0); res_h = residual_lm(res_in); keep the last rows. prev = zeros [1, 4, 64].
3. Per patch i: x ~ N(0, 1) [1, 64, 4]; t = s + (cos(π s / 2) − 1 + s) for s = linspace(1, 0, 11); for k = 1..9: x −= (t[k] − t[k+1]) × dit_step(x, lm_h, res_h, prev, t[k], 2.0) (step 0 has no velocity, as CFG-zero*); patch = xᵀ [1, 4, 64]. Keep it; stop if i > 2 and argmax(stop) == 1 (the stop of the state used for this patch). Otherwise e = feat_encoder(patch), raw = base_lm(e), lm_h, res_in, stop = bridge(raw, e, mask 1), res_h = residual_lm(res_in), prev = patch.
4. The patches, concatenated to [1, 64, 4 × patches], through vae_decoder.

Licence

Apache 2.0 (LICENSE), as VoxCPM2 and its AudioVAE.