moorlee/gender-voice-classifier-ecapa

Gender Voice Classifier — English / Hindi / Tamil

Binary speaker gender classifier (Female / Male) that works across English, Hindi, and Tamil in a single model. Built to fix the female-recall problem in existing models like audEERING's wav2vec2, which uses a 3-class head (child / female / male) causing high-F0 female voices to be misclassified as child.

Performance

Test set: speaker-disjoint held-out split — speakers never seen during training.

Language	Accuracy	Female Recall	Male Recall
English	99.5%	99.5%	99.5%
Hindi	99.2%	100.0%	98.4%
Tamil	99.2%	100.0%	98.3%
Overall	99.3%	99.8%	99.1%

1,044 test clips across 3 languages, balanced male/female, speaker-disjoint from training.

Quick Start

pip install speechbrain torchaudio soundfile scikit-learn joblib huggingface_hub

import json
import numpy as np
import soundfile as sf
import torch
import joblib
from huggingface_hub import hf_hub_download
from speechbrain.inference.speaker import EncoderClassifier

REPO = "moorlee/gender-voice-classifier-ecapa"

# Download the 3 lightweight head files (total < 10 KB)
scaler    = joblib.load(hf_hub_download(REPO, "scaler.pkl"))
clf       = joblib.load(hf_hub_download(REPO, "logreg.pkl"))
threshold = json.load(open(hf_hub_download(REPO, "thresholds.json")))["female_optimized"]

# Load frozen ECAPA backbone — downloads ~100 MB once, cached after that
encoder = EncoderClassifier.from_hparams(
    source="speechbrain/spkrec-ecapa-voxceleb",
    run_opts={"device": "cpu"},
)
encoder.eval()

def predict_gender(path: str) -> str:
    arr, sr = sf.read(path, dtype="float32")
    if arr.ndim > 1:
        arr = arr.mean(axis=1)          # stereo → mono
    if sr != 16000:
        import torchaudio
        arr = torchaudio.transforms.Resample(sr, 16000)(
            torch.tensor(arr).unsqueeze(0)
        ).squeeze(0).numpy()
    with torch.no_grad():
        emb = encoder.encode_batch(torch.tensor(arr).unsqueeze(0)).squeeze().numpy()
    p_female = float(clf.predict_proba(scaler.transform(emb.reshape(1, -1)))[0, 1])
    return "female" if p_female >= threshold else "male"

print(predict_gender("your_voice.wav"))  # → "female" or "male"

How It Works

Audio (any format / sample rate)
  → resample to 16 kHz, mono
  → ECAPA-TDNN  (frozen, speechbrain/spkrec-ecapa-voxceleb)
  → 192-dim speaker embedding
  → StandardScaler  (fit on training set)
  → LogisticRegression  (193 parameters: 192 weights + 1 bias)
  → P(female) vs calibrated threshold
  → "female" / "male"

The backbone is frozen — we don't fine-tune it. ECAPA-TDNN speaker embeddings encode gender as a strongly linearly separable feature, so a logistic regression head is sufficient. The model converged in 11 L-BFGS iterations.

Decision threshold: calibrated on the validation set to maximise female recall subject to female precision ≥ 0.88. Default is 0.05 — very sensitive to females. Raise to 0.35–0.50 for more conservative predictions.

Training Data

Dataset	Languages	Clips used	Gender balance
Google FLEURS	en_us, hi_in, ta_in	4,878	50 / 50 per language

Speaker-disjoint splits: speakers are grouped by ID and split 70/15/15 (train/val/test) so no speaker appears in more than one set. This prevents the model from learning voice identity instead of gender.

Limitations

• Trained on FLEURS read speech (clean, studio-quality). Accuracy may drop on noisy audio, phone calls, or highly accented speech.
• ECAPA backbone was pretrained on VoxCeleb (English celebrities). Hindi/Tamil generalisation relies on the physiological universality of gender cues (F0 and vocal tract length), not language-specific fine-tuning.
• Binary only — does not distinguish child voices. A child's voice may be classified as female due to similar pitch range.
• Not suited for non-binary or gender-nonconforming speakers.

Why Not audEERING?

audeering/wav2vec2-large-robust-24-ft-age-gender is 1.2 GB and uses a 3-class gender head (child / female / male). High-F0 female voices are frequently absorbed into the child class, causing poor female recall. This model uses a binary head only, eliminating that confusion, at a fraction of the size.

Citation

If you use this model, please cite the ECAPA-TDNN backbone:

@inproceedings{desplanques2020ecapa,
  title     = {ECAPA-TDNN: Emphasized Channel Attention, Propagation and Aggregation in TDNN Based Speaker Verification},
  author    = {Desplanques, Brecht and Thienpondt, Jenthe and Demuynck, Kris},
  booktitle = {Interspeech},
  year      = {2020}
}