mlboydaisuke/ECAPA-TDNN-Speaker-ExecuTorch

ECAPA-TDNN speaker embedding — ExecuTorch

Three seconds of audio in, one 192-dimensional speaker vector out. Two vectors from the same voice have a high cosine; two from different voices do not. That is the whole interface for speaker verification, diarisation and voice-matched personalisation.

• Source: speechbrain/spkrec-ecapa-voxceleb — 20.8M parameters, ECAPA-TDNN
• License: apache-2.0
• Input: waveform [1, 48000] fp32 — raw 16 kHz mono in [-1, 1], exactly 3 s
• Output: [1, 192] fp32 — compare two with cosine

Variants

build	file	size (MB)	Mac median (ms)*	cosine vs eager	same/different separation
fp32	speaker_ecapa_xnnpack_fp32.pte	83.9	23.7	1.000000	0.653
Core ML (fp16, iOS)	speaker_ecapa_coreml_all.pte	42.5	2.4	0.998470	0.648

*Mac arm64, single process, median of 10. PyTorch eager fp32 on the same machine is 17.0 ms, so the Core ML build is 7.1x eager at half the size. The XNNPACK build is the portable one and the same file runs on Android.

The feature extraction is inside the graph

speechbrain's pipeline is a mel filterbank, then per-utterance mean subtraction, then ECAPA. Both halves are easy to get wrong from outside — the wrong mel configuration, or forgetting the normalisation, gives a vector that looks fine and matches the wrong speaker. Taking a waveform means there is nothing left for the caller to get wrong.

Getting that into a graph took three rewrites, each checked against the module it replaced:

what	why	checked
STFT as two convolutions	torch.stft returns complex; coremltools refuses the model outright (dtype=... complex ... is unsupported)	relative error 2.0e-05
mel filterbank precomputed	Filterbank.forward rebuilds its triangles every call out of arange/abs/sub	max_abs_diff 0.0
reflect padding as slices	F.pad(mode="reflect") lowers to index+arange+abs+sub, which XNNPACK will not take — 38 convolutions, so 38 index, 39 arange, 76 abs, 76 sub	allclose, identical

The padding rewrite is the one that paid: XNNPACK delegate coverage went from 40.6% to 91.4%, subgraphs from 49 to 17, and the build from 34.1 ms to 23.7 ms — still with correlation 1.000000 against eager.

Verification

The assembled pipeline reproduces speechbrain's own EncoderClassifier.encode_batch at max_abs_diff 0.0, so the control is the upstream model rather than a reimplementation of it.

Then two questions per build. Agreement with eager, as cosine per utterance. And whether the vectors are useful at all, which agreement cannot show — a build returning the same vector for everything would agree perfectly:

same voice: 0.727..0.844 over 12 pairs
different : -0.045..0.074 over 16 pairs

Every same-voice pair scores 0.653 above every different-voice pair. The voices are two synthetic speakers from macOS say — a functional check, not an EER on a verification benchmark.

Not shipped

• fp16 comes out at 83.9 MB against fp32's 83.9 MB — no smaller at all. XNNPACK serializes convolution weights as fp32 whatever dtype the graph carries, and this model is convolutions end to end. Its correlation is also 0.9938, below the fp16 bar.
• int8 does not export. Dynamic int8 annotates linear only, and ECAPA does not contain one — exported that way the file was byte-for-byte fp32's size with correlation 1.000000, which is to say nothing had been quantized. Static int8, globally annotated or restricted to conv1d, is rejected by the partitioner with Weight and Input should both be quantized. That is not caused by the padding rewrite: exported with speechbrain's own F.pad(reflect) instead, one variable moved, the failure is identical. The cause is not established, so it is reported as an observation rather than explained.

Conversion

python convert/export_speaker.py
python convert/check_speaker.py fp32

speechbrain is used only to build the module and load embedding_model.ckpt with strict=True. The .pte carries no dependency on it.

(conversion scripts: executorch-models)