Package: pyannote.audio
Usingpyannote.audioopen-source toolkit in production?
Make the most of it thanks to ourconsulting services.pyannote.audiospeaker diarization toolkitpyannote.audiois an open-source toolkit written in Python for speaker diarization. Based onPyTorchmachine learning framework, it comes with state-of-the-artpretrained models and pipelines, that can be further finetuned to your own data for even better performance.TL;DRInstallpyannote.audiowithpip install pyannote.audioAcceptpyannote/segmentation-3.0user conditionsAcceptpyannote/speaker-diarization-3.1user conditionsCreate access token athf.co/settings/tokens.frompyannote.audioimportPipelinepipeline=Pipeline.from_pretrained("pyannote/speaker-diarization-3.1",use_auth_token="HUGGINGFACE_ACCESS_TOKEN_GOES_HERE")# send pipeline to GPU (when available)importtorchpipeline.to(torch.device("cuda"))# apply pretrained pipelinediarization=pipeline("audio.wav")# print the resultforturn,_,speakerindiarization.itertracks(yield_label=True):print(f"start={turn.start:.1f}s stop={turn.end:.1f}s speaker_{speaker}")# start=0.2s stop=1.5s speaker_0# start=1.8s stop=3.9s speaker_1# start=4.2s stop=5.7s speaker_0# ...Highlights:hugs: pretrainedpipelines(andmodels) on:hugs: model hub:exploding_head: state-of-the-art performance (seeBenchmark):snake: Python-first API:zap: multi-GPU training withpytorch-lightningDocumentationChangelogFrequently asked questionsModelsAvailable tasks explainedApplying a pretrained modelTraining, fine-tuning, and transfer learningPipelinesAvailable pipelines explainedApplying a pretrained pipelineAdapting a pretrained pipeline to your own dataTraining a pipelineContributingAdding a new modelAdding a new taskAdding a new pipelineSharing pretrained models and pipelinesBlog2022-12-02 >"How I reached 1st place at Ego4D 2022, 1st place at Albayzin 2022, and 6th place at VoxSRC 2022 speaker diarization challenges"2022-10-23 >"One speaker segmentation model to rule them all"2021-08-05 >"Streaming voice activity detection with pyannote.audio"VideosIntroduction to speaker diarization/ JSALT 2023 summer school / 90 minSpeaker segmentation model/ Interspeech 2021 / 3 minFirst releaase of pyannote.audio/ ICASSP 2020 / 8 minBenchmarkOut of the box,pyannote.audiospeaker diarizationpipelinev3.1 is expected to be much better (and faster) than v2.x.
Those numbers are diarization error rates (in %):Benchmarkv2.1v3.1PremiumAISHELL-414.112.311.9AliMeeting (channel 1)27.424.522.5AMI (IHM)18.918.816.6AMI (SDM)27.122.620.9AVA-AVD66.350.039.8CALLHOME (part 2)31.628.422.2DIHARD 3 (full)26.921.417.2Ego4D (dev.)61.551.243.8MSDWild32.825.419.8REPERE (phase2)8.27.87.6VoxConverse (v0.3)11.211.29.4Diarization error rate(in %)CitationsIf you usepyannote.audioplease use the following citations:@inproceedings{Plaquet23,author={Alexis Plaquet and Hervé Bredin},title={{Powerset multi-class cross entropy loss for neural speaker diarization}},year=2023,booktitle={Proc. INTERSPEECH 2023},}@inproceedings{Bredin23,author={Hervé Bredin},title={{pyannote.audio 2.1 speaker diarization pipeline: principle, benchmark, and recipe}},year=2023,booktitle={Proc. INTERSPEECH 2023},}DevelopmentThe commands below will setup pre-commit hooks and packages needed for developing thepyannote.audiolibrary.pipinstall-e.[dev,testing]pre-commitinstallTestpytest