GoodnotesLtd/wespeaker-resnet221-lm-coreml

WeSpeaker ResNet221-LM: COREML conversion

Apache-2.0, as tagged by the upstream VoxCeleb2 weights model card. Credit: WeSpeaker (wenet-e2e/wespeaker). Source: https://huggingface.co/Wespeaker/wespeaker-voxceleb-resnet221-LM, revision 26732f43821f238bfb258c5ef54924a798e62118. Code revision: 9fecd6cb4f47475d01761d87c826298dff4ef18c. Preserve VoxCeleb2 attribution: WeSpeaker's general pretrained-model recipe also says weights follow their dataset licence. The upstream tag is retained verbatim; it does not resolve that broader attribution caveat.

Source	SHA-256	Bytes
voxceleb_resnet221_LM.pt	47d76239f1e865b273e31e30f691959061ff6565ed6c4e7be9639a65d9662eb5	114410390
voxceleb_resnet221_LM.onnx	e6b8c14c022288ca186de39f038a9bd9404be4f174ac8a6a3b8dbf229c8f607d	95037357

Conversion and contracts

Model revision wespeaker-resnet221-lm-v1, model ID wespeaker-resnet221-lm-256, 256 dimensions, 16 kHz mono, kaldi-fbank80-v1 (80-bin 25/10ms Hamming, no dither, per-mel CMN, PCM16 scale). Apply L2 normalization before cosine comparison.

ONNX opset 14. Dynamic voxceleb_resnet221_LM.onnx is the original, hash-identical upstream graph: fp32 feats[B,T,80] -> raw embs[B,256]. Static voxceleb_resnet221_LM_static_unmasked.onnx is exported from the same pinned source/checkpoint with ordinary unmasked source TSTP pooling: fp32 feats[1,998,80] -> raw embs[1,256]. This is the single-input ResNet34 contract on current asr-cpp-node main after #143. Windows ML/QNN uses valid-frame CMN followed by feature zero-padding to 998, with no mask input. CPU, Linux and Android continue using the dynamic graph; the native CPU path does not pad inputs for fixed graphs. Windows ML accepts at most 10 seconds per extraction, as for ResNet34. No native padding or loader changes are included. Windows ML/QNN hardware placement remains untested; shape/constant bookkeeping needs ORT folding before placement. CPU ORT basic optimization was checked: Shape/Gather fold away, leaving 13 standard tensor operator types. This is not hardware qualification.

Unmasked static artifact: 95,082,680 bytes; SHA-256 389b4d2c4d1159b66f79f94a1ab4de320ecaefc095c8960f4b2cb8f637f043f8.

Retained historical artifact: voxceleb_resnet221_LM_static.onnx (SHA-256 286cf5a1f932017d942f9abaebad4b7f6d65b3cdbda5e7b3e26463a1f7be7f12) takes feats plus weights. It requires #142-style feature-mask support, which is unavailable on current main because #143 reverted it. It is no longer selected by the catalog. The published file and its original immutable revision 659b3739be8452c22cc7cdf8075d20f954c0c9dd are preserved.

Core ML is fixed 10s NeuralNetwork, fp16 stored weights and fp32 I/O and pooling arithmetic: fbank_features[1,1,80,998], weights[1,589] -> normalized embedding[1,256] and raw embedding_raw[1,256]. Identity mask interpolation is linear; the raw branch is nearest/floor, matching the existing ResNet34 dual-output convention. Reuse the existing FBank model and native valid-prefix CMN, zero padding and multi-window path. The portable Embedding.mlmodel can be compiled with the target iOS SDK; Embedding.mlmodelc was compiled on macOS 27.0.1. macOS uses CPUAndNeuralEngine. iOS runtime is not qualified by macOS compilation.

Why this variant: the bake-off's fixed NN passed 204 masked comparisons; dynamic MLProgram produced NaNs and an fp16 graph failed scalar-subtraction compilation. Compressing only stored NN weights avoids that failure. A separate direct MLProgram experiment was slightly faster in Python but emitted fp16 outputs and was rejected by the unchanged native fp32 tensor contract. It is not distributed. NN is the native-compatible validated lane.

Parity and performance

216 local clips: the original 204 fit-only stratified clips (34 each of headset/far-field x 2/10/30s), plus 12 duration-boundary cases. Direct graph/reference checks use each clip's first <=10s window with valid-prefix CMN and masking. All values are finite. Minimum/median cosine:

Comparison	Minimum	Median
NN normalized vs PyTorch	0.999878287315	0.999977231026
NN raw vs PyTorch	0.999878287315	0.999977111816
Official ONNX vs PyTorch	0.999999821186	1.000000000000

All three source-conversion gates (>=200 clips, cosine >=0.999) pass. Historical production backends (before #143) ran 97 clip comparisons including both conditions at 1/2/5/9.9/10/30s. On 70 conversation segments, Core ML vs dynamic ONNX minimum/mean cosine is 0.890734/0.987775; minima at 2/5/9.9/10/30s are 0.983472/0.980562/0.988460/0.990590/0.995792. The 1s minimum fails the upgrade spec's 0.95 gate. Padding behaviour is shared with ResNet34 on main; this asset port does not change it. This is an opt-in revision, not release qualification or permission to reinterpret existing speaker vectors.

Native macOS M2 Max warm median RTF (34 10s clips; busy shared host; production frontend included): Core ML 0.005877, dynamic ONNX CPU-2 0.230177, historical masked static ONNX CPU-2 0.231359 (not a measurement of the new unmasked graph). Separate bake-off Python RTF: CoreML EP 0.00399, CPU-4 0.09248; those measurements use a different runtime/protocol. Windows x64/arm64, Windows ML/QNN placement, Linux, iOS and Android were not run locally. The user will test Windows and Android later.

Current parity after #143

Core ML artifacts and dynamic ONNX bytes are unchanged. The current native Core ML/dynamic rerun passes at identity admission durations: minimum/mean cosine 0.981987/0.992617 for >=5s; minima at 5/9.9/10/30s are 0.981987/0.988404/0.990590/0.988620. Values at 1/2/3s are informational (0.844409/0.965467/0.847563), excluded from minimum/mean gates. Frame agreement is 0.998361 and cross-backend DER is 0.00222676.

The new unmasked static graph is validated against unpadded pinned PyTorch source and dynamic ONNX on the same conversation segments as ResNet34, with the current Windows ML padding contract. Only the unmasked static 5s comparison has the conditionally approved >=0.94 floor: ResNet34's 5s min/median/p5 are 0.948945/0.960176/0.950547; ResNet221's are 0.944648/0.951946/0.945921. ResNet34's minimum is below 0.95, satisfying the user's condition. Values at 1/2/3s remain informational. Static 9.9/10/30s retain >=0.95; 30s is a graph check averaging normalized 10s windows against full dynamic extraction, not new native Windows support. All durations must also match identically padded PyTorch source at >=0.999. Dynamic ONNX/Core ML retain >=0.95 from 5s. The user will validate Windows x64 and arm64 on hardware. Receipt: speaker-resnet221-unmasked-static-2026-11-06.json in the asr-cpp-node repository's docs/benchmarks/ directory. Native CPU loading/extraction at 10s also passes on headset and far-field clips. Native 30s graph aggregation over seven conversation segments has min/median/p5 0.986523/0.991487/0.986901 against full dynamic extraction, passing the unchanged >=0.95 floor. This is graph/CPU validation, not Windows ML/QNN hardware qualification.

Reproduction and files

Export: prepare-speaker-resnet221.py. Python 3.12, torch 2.5.1, coremltools 9.0, onnx 1.23.2; source validation used ONNX Runtime 1.31.1; native parity used ORT 1.27.0. See manifest.json for exact per-file SHA-256, sizes, contracts and source URLs.

Modifications: export single-input unmasked fixed-shape ONNX; convert the source network to fixed NeuralNetwork with dual normalized/raw outputs and fp16 stored weights. Dynamic ONNX parameters/bytes are unchanged. Only this model revision's files, card and manifest are published; no audio, datasets, individual speaker vectors or personal data are included.