DimQ1/sortformer-4spk-v2-onnx-fp32-cpu

Sortformer 4spk-v2 — ONNX FP32 (CPU)

FP32 ONNX export of NVIDIA's Sortformer 4spk-v2 speaker diarization model for CPU inference via ONNX Runtime.

Model Details

• Base model: nvidia/diar_streaming_sortformer_4spk-v2
• Format: ONNX opset 21, FP32
• Parameters: 117.7 M
• File size: 469 MB (single-file ONNX)
• Input: processed_signal — log-mel spectrogram [B, 128, mel_frames]
• Output: speaker_logits — per-frame sigmoid probabilities [B, diar_frames, 4]
• Diar frame rate: 80 ms (10 ms mel stride / 8× subsampling)

Architecture

Audio (16 kHz mono) → STFT → 128 mel bins
  → ConformerEncoder (17 layers, d=512)
    → SortformerModules (4 speakers)
      → TransformerEncoder (18 layers, d=192)
        → Sigmoid → [B, diar_frames, 4] speaker logits

Performance (CPU, 12 threads, ORT 1.27.1)

Metric	Value
RTF	0.037 (27× real-time)
DER	17.82% (LibriCSS test set)
Size	469 MB

Benchmarked on 100-file LibriCSS subset (1954.8 s audio). ORT 1.27.1 with ORT_PARALLEL + memory optimizations.

Quantized Variants

Variant	Size	DER	RTF	Link
INT8	129 MB	21.64%	0.034	sortformer-4spk-v2-onnx-int8-cpu
INT4	135 MB	19.11%	0.043	sortformer-4spk-v2-onnx-int4-cpu

Preprocessing

Before inference, convert raw 16 kHz mono audio to log-mel spectrogram:

• Sample rate: 16000 Hz
• Window: Hann 25 ms
• Stride: 10 ms
• FFT size: 512
• Mel bins: 128
• Log scaling

Usage

Python (ONNX Runtime)

import onnxruntime as ort
import numpy as np

session = ort.InferenceSession("sortformer.onnx", providers=["CPUExecutionProvider"])

# mel: log-mel spectrogram [1, 128, mel_frames]
mel = np.random.randn(1, 128, 500).astype(np.float32)  # 5s @ 10ms stride
speaker_logits = session.run(None, {"processed_signal": mel})[0]
# shape: [1, 63, 4]

C# (.NET 10, ONNX Runtime)

using var session = new InferenceSession("sortformer.onnx");
var inputs = new List<NamedOnnxValue>
{
    NamedOnnxValue.CreateFromTensor("processed_signal", melTensor),
};
using var results = session.Run(inputs);
var logits = results.First().AsTensor<float>();
// logits shape: [1, diar_frames, 4]

Conversion

Exported from NeMo to ONNX (opset 21) using DiarizationConverter:

python scripts/export_fp32_opset21.py

References

• NVIDIA Sortformer Paper
• NeMo Diarization Models
• ONNX Runtime