Voice activity detection trained on DIHARD III development set

Relies on pyannote.audio 2.0 (which is still in development):

$ pip install https://github.com/pyannote/pyannote-audio/archive/develop.zip

Usage

MODEL = "hbredin/VoiceActivityDetection-PyanNet-DIHARD"

The simplest way of getting voice activity detection results is to use the pretrained pipeline:

from pyannote.audio import Pipeline
vad = Pipeline.from_pretrained(MODEL, device="cuda")
speech_regions = vad("audio.wav")
for (start_time, end_time) in speech_regions.get_timeline():
    pass

speech_regions is a pyannote.core.Annotation instance.

If you need more control (e.g. to lower the detection threshold for better recall), speech probabilites can be obtained by running the pretrained model:

from pyannote.audio import Inference
vad = Inference(MODEL, device="cuda")
speech_prob = vad("audio.wav")

speech_prob is a pyannote.core.SlidingWindowFeature instance

Note: audio files can also be provided as a {"waveform": (num_channels, num_samples) numpy array, "sample_rate": int} dictionary.

If you need even more control (e.g. to fine-tune the model), the model can be loaded like that:

from pyannote.audio import Model
model = Model.from_pretrained(MODEL)

Citations

Model

@inproceedings{Lavechin2020,
  Title = {{End-to-end Domain-Adversarial Voice Activity Detection}},
  Author = {{Lavechin}, Marvin and {Gill}, Marie-Philippe and {Bousbib}, Ruben and {Bredin}, Herv{\'e} and {Garcia-Perera}, Leibny Paola},
  Booktitle = {ICASSP 2020, IEEE International Conference on Acoustics, Speech, and Signal Processing},
  Address = {Barcelona, Spain},
  Month = {May},
  Year = {2020},
}

pyannote.audio toolkit

@inproceedings{Bredin2020,
  Title = {{pyannote.audio: neural building blocks for speaker diarization}},
  Author = {{Bredin}, Herv{\'e} and {Yin}, Ruiqing and {Coria}, Juan Manuel and {Gelly}, Gregory and {Korshunov}, Pavel and {Lavechin}, Marvin and {Fustes}, Diego and {Titeux}, Hadrien and {Bouaziz}, Wassim and {Gill}, Marie-Philippe},
  Booktitle = {ICASSP 2020, IEEE International Conference on Acoustics, Speech, and Signal Processing},
  Address = {Barcelona, Spain},
  Month = {May},
  Year = {2020},
}
Downloads last month
9,720
Hosted inference API
Voice Activity Detection

Inference API has been turned off for this model.