DigiGreen/pyannote-segmentation-agri-indic

Agri Diarization Segmenter (pyannote segmentation, fine-tuned for Indic farmer field audio)

A speaker-segmentation model for noisy, multi-speaker agricultural voice queries in Hindi, Telugu, and Odia. It is pyannote/segmentation-3.0 fine-tuned on human-annotated FarmerChat field recordings, and it is the one fine-tuned component of Digital Green's model-agnostic voice pipeline for agricultural speech. It is used to isolate the farmer's own speech (diarization → best-speaker selection) before an unmodified ASR model transcribes it.

Why fine-tune

No off-the-shelf diarizer transferred cleanly to open-air, low-end-phone farmer recordings across these three languages: for every stock system the dominant error is false alarm (hearing speech in background noise). Fine-tuning the segmenter on this audio collapses that false alarm.

Results (leakage-free, matched held-out split)

Scored on a frozen 411-clip test split the model never trained on, against the same human reference at a 0.25 s collar, alongside the stock and cloud diarizers on the same clips:

Diarizer	DER @0.25 (lower is better)
This model (fine-tuned pyannote)	0.214
Azure (cloud)	0.249
pyannote 3.1 (stock)	0.417

The improvement over stock is almost entirely a reduction in false alarm (0.364 → 0.118), at the cost of a small rise in missed speech (0.065 → 0.111). The validation-checkpoint DER was 0.1908 (epoch 82).

Intended use

• Speaker segmentation / diarization for agricultural voice pipelines, as the front end to target-speaker selection before ASR.
• CPU-inferable (~0.4 s per clip) — no GPU required at inference.

Training data

Fine-tuned on human-annotated FarmerChat field recordings (Hindi / Telugu / Odia), with speaker turns and timestamps placed by human annotators. Clip-level split: 1,884 train / 401 dev / 411 test. The raw training audio and speaker-turn files are not redistributed here because they are farmer field recordings (privacy); this repository ships the model weights and recipe only.

Limitations

• Adapted on Hindi, Telugu, and Odia field audio; transfer to unseen languages is not established.
• The false-alarm reduction trades a little recall on the rare clips with three or more speakers.
• The split is by clip, not demonstrated to be speaker- or session-disjoint.

Usage

from pyannote.audio import Model
from pyannote.audio.pipelines import SpeakerDiarization

# load the fine-tuned segmentation model from the local checkpoint
seg = Model.from_pretrained("seg-ft-epoch82-DER0.1908.ckpt")

pipeline = SpeakerDiarization(segmentation=seg, clustering="AgglomerativeClustering")
pipeline.instantiate({
    "clustering": {"method": "centroid", "threshold": 0.7},
    "segmentation": {"min_duration_off": 0.0},
})
diarization = pipeline("clip.wav")
for turn, _, speaker in diarization.itertracks(yield_label=True):
    print(f"{turn.start:.2f}-{turn.end:.2f} {speaker}")

Tune the clustering threshold for your audio. The checkpoint is a pyannote/Lightning segmentation model.

The accompanying paper, Model-Agnostic and Language-Agnostic Voice Pipeline Improvement for the Agriculture Domain, is included in this repository (arXiv:2609.20504).

License and attribution

MIT, following the base model pyannote/segmentation-3.0 (Bredin, pyannote.audio). Please cite pyannote.audio and Digital Green's agricultural voice-pipeline paper if you use this model.