androiddrew/google-speech-embedding-onnx

Google speech_embedding/1, ONNX

An ONNX conversion of Google's speech_embedding/1 (Lin et al., 2020), made from the PyTorch port androiddrew/google-speech-embedding-pytorch. It maps 16 kHz audio to one 96-dimensional embedding every 80 ms. Those embeddings describe speech sounds; the model does not produce transcripts.

It is the frozen audio feature extractor of androiddrew/naptrap-phonmatchnet-en-onnx and is embedded in go-phonmatchnet.

This conversion is not affiliated with or endorsed by Google. The original model is published by Google under the Apache License 2.0. This conversion keeps that license; see LICENSE and NOTICE.

Files

• model.onnx: opset 17, float32. Input waveform float32 [N]; output embeddings [floor(N / 1280), 96]. The padding and log-mel frontend are inside the graph.
• config.json: the preprocessing contract and SHA-256 of model.onnx.
• conversion/: the export script, its locked environment and README.

import onnxruntime as ort

session = ort.InferenceSession("model.onnx")
(embeddings,) = session.run(None, {"waveform": clip})   # clip: mono 16 kHz float32 [N]

Preprocessing contract (inside the graph)

Step	Value
Input	mono 16 kHz float32 in [-1, 1]; the keyword model uses 0.5–2 s clips
Padding	11,120 leading zero samples, no trimming
STFT	400-sample Hann window, hop 160, 512-point FFT, no centering, magnitude
Mel	32 bins, natural log(mel + 0.001)
Output	floor(samples / 1280) frames × 96

The keyword models were trained on features computed this way, one clip at a time. Slicing features computed over a longer recording gives different values.

Conversion and accuracy

conversion/export.py loads the PyTorch port at commit ad7a3dd93b02f83797a040857425d6343d98680a with naptrap 0.1.0 and traces it. All trained parameters, BatchNorm statistics, the window and the mel matrix are unchanged. The FFT (torch.fft.rfft of 400-sample frames padded to 512) is a strided convolution with a fixed DFT basis, the window folded in.

The script fails unless the graph matches the PyTorch port within 1e-3 over 17 clips of speech and noise from 8,000 to 32,000 samples long. The measured maximum difference is 4e-5, smaller than the PyTorch port's own difference from the TensorFlow original (7.2e-5 on real speech). With the locked environment it reproduces model.onnx byte for byte.

Training data and limitations

Google reports training the original on roughly 200 million two-second YouTube clips covering about 5,000 keywords (Lin et al., 2020). No clip-level inventory is available. This repository adds no training. Behaviour on non-English speech, far-field audio or unusual microphones is unknown.

Citation

@article{lin2020training,
  title   = {Training Keyword Spotters with Limited and Synthesized Speech Data},
  author  = {Lin, James and Kilgour, Kevin and Roblek, Dominik and Sharifi, Matthew},
  journal = {arXiv preprint arXiv:2002.01322},
  year    = {2020}
}