kenrunio/transcriber-speaker-onnx

ECAPA-TDNN speaker embeddings, ONNX

SpeechBrain's ECAPA-TDNN speaker model, converted to ONNX so it runs with ONNX Runtime alone, without PyTorch or SpeechBrain. It is used by Transcriber to tell speakers apart.

Given a short stretch of speech, the model returns a 192-number voiceprint (embedding). Clips of the same voice give similar embeddings; different voices give clearly different ones. It does not identify anyone: it only measures how alike two voices sound.

The files

ecapa-any-length.onnx (84 MB) contains the whole model: the audio features (80-band filterbank), their normalisation and the network, so there is no preprocessing to reproduce. It takes clips of any length; Transcriber uses 2 s clips, and 0.5 s ones to place each change of speaker.

ecapa.onnx is the same model exported for 1.5 s clips only (24,000 samples), used by earlier versions of Transcriber. It is kept so that they still install. On 1.5 s clips the two files give the same embeddings.

• Input wav: float32, shape (batch, samples), mono 16 kHz audio, padded with zeros to the longest clip in the batch.
• Input lens: float32, shape (batch,), the share of each row that is audio (1.0 = no padding).
• Output emb: float32, shape (batch, 1, 192). Normalise to unit length and compare with cosine similarity.

import numpy as np, onnxruntime as ort
sess = ort.InferenceSession("ecapa-any-length.onnx")
emb = sess.run(None, {"wav": clips.astype(np.float32), "lens": lens.astype(np.float32)})[0][:, 0]
emb /= np.linalg.norm(emb, axis=1, keepdims=True)

How it was made, and checked

Exported with torch.onnx.export(..., dynamo=True, opset_version=18) by tools/export_models.py from speechbrain/spkrec-ecapa-voxceleb. On a 90-minute recording (5,024 windows of 1.5 s) every ONNX embedding had cosine similarity ≥ 0.999999 to PyTorch's, and speaker clustering assigned every window to the same speaker. ecapa-any-length.onnx was exported the same way with the clip length left open; on a 25-minute recording (1,517 windows of 1.5 s) its embeddings had cosine similarity 1.000000 to ecapa.onnx's, with the same speaker for every window.

Licence and credit

Apache-2.0, as the original (LICENSE). This is a format conversion of the original weights; the weights themselves are unchanged. The model was trained by the SpeechBrain project on VoxCeleb 1 and 2. If you use it in research, please cite SpeechBrain and ECAPA-TDNN:

• M. Ravanelli et al., "SpeechBrain: A General-Purpose Speech Toolkit", 2021. arXiv:2106.04624
• B. Desplanques, J. Thienpondt, K. Demuynck, "ECAPA-TDNN: Emphasized Channel Attention, Propagation and Aggregation in TDNN Based Speaker Verification", Interspeech 2020.