Hervé BREDIN
commited on
Commit
•
2c6a571
1
Parent(s):
af4b976
fix: link to tag 2.1.1
Browse files
README.md
CHANGED
|
@@ -89,19 +89,19 @@ Processing is fully automatic:
|
|
| 89 |
* evaluation of overlapped speech
|
| 90 |
|
| 91 |
|
| 92 |
-
| Benchmark | [DER%](. "Diarization error rate") | [FA%](. "False alarm rate") | [Miss%](. "Missed detection rate") | [Conf%](. "Speaker confusion rate") | Expected output
|
| 93 |
-
| ------------------------------------------------------------------------------------------------------------------------------------------- | ---------------------------------- | --------------------------- | ---------------------------------- | ----------------------------------- |
|
| 94 |
-
| [AISHELL-4](http://www.openslr.org/111/) | 14.09 | 5.17 | 3.27 | 5.65 | [RTTM](https://huggingface.co/pyannote/speaker-diarization/blob/2.1/reproducible_research/2.1.1/AISHELL.test.rttm) | [eval](https://huggingface.co/pyannote/speaker-diarization/blob/2.1/reproducible_research/2.1.1/AISHELL.test.eval) |
|
| 95 |
-
| [Albayzin (*RTVE 2022*)](http://catedrartve.unizar.es/albayzindatabases.html) | 25.60 | 5.58 | 6.84 | 13.18 | [RTTM](https://huggingface.co/pyannote/speaker-diarization/blob/2.1/reproducible_research/2.1.1/Albayzin2022.test.rttm) | [eval](https://huggingface.co/pyannote/speaker-diarization/blob/2.1/reproducible_research/2.1.1/Albayzin2022.test.eval) |
|
| 96 |
-
| [AliMeeting (*channel 1*)](https://www.openslr.org/119/) | 27.42 | 4.84 | 14.00 | 8.58 | [RTTM](https://huggingface.co/pyannote/speaker-diarization/blob/2.1/reproducible_research/2.1.1/AliMeeting.test.rttm) | [eval](https://huggingface.co/pyannote/speaker-diarization/blob/2.1/reproducible_research/2.1.1/AliMeeting.test.eval) |
|
| 97 |
-
| [AMI (*headset mix,*](https://groups.inf.ed.ac.uk/ami/corpus/) [*only_words*)](https://github.com/BUTSpeechFIT/AMI-diarization-setup) | 18.91 | 4.48 | 9.51 | 4.91 | [RTTM](https://huggingface.co/pyannote/speaker-diarization/blob/2.1/reproducible_research/2.1.1/AMI.test.rttm) | [eval](https://huggingface.co/pyannote/speaker-diarization/blob/2.1/reproducible_research/2.1.1/AMI.test.eval) |
|
| 98 |
-
| [AMI (*array1, channel 1,*](https://groups.inf.ed.ac.uk/ami/corpus/) [*only_words)*](https://github.com/BUTSpeechFIT/AMI-diarization-setup) | 27.12 | 4.11 | 17.78 | 5.23 | [RTTM](https://huggingface.co/pyannote/speaker-diarization/blob/2.1/reproducible_research/2.1.1/AMI-SDM.test.rttm) | [eval](https://huggingface.co/pyannote/speaker-diarization/blob/2.1/reproducible_research/2.1.1/AMI-SDM.test.eval) |
|
| 99 |
-
| [CALLHOME](https://catalog.ldc.upenn.edu/LDC2001S97) [(*part2*)](https://github.com/BUTSpeechFIT/CALLHOME_sublists/issues/1) | 32.37 | 6.30 | 13.72 | 12.35 | [RTTM](https://huggingface.co/pyannote/speaker-diarization/blob/2.1/reproducible_research/2.1.1/CALLHOME.test.rttm) | [eval](https://huggingface.co/pyannote/speaker-diarization/blob/2.1/reproducible_research/2.1.1/CALLHOME.test.eval) |
|
| 100 |
-
| [DIHARD 3 (*Full*)](https://arxiv.org/abs/2012.01477) | 26.94 | 10.50 | 8.41 | 8.03 | [RTTM](https://huggingface.co/pyannote/speaker-diarization/blob/2.1/reproducible_research/2.1.1/DIHARD.test.rttm) | [eval](https://huggingface.co/pyannote/speaker-diarization/blob/2.1/reproducible_research/2.1.1/DIHARD.test.eval) |
|
| 101 |
-
| [Ego4D *v1 (validation)*](https://arxiv.org/abs/2110.07058) | 63.99 | 3.91 | 44.42 | 15.67 | [RTTM](https://huggingface.co/pyannote/speaker-diarization/blob/2.1/reproducible_research/2.1.1/Ego4D.development.rttm) | [eval](https://huggingface.co/pyannote/speaker-diarization/blob/2.1/reproducible_research/2.1.1/Ego4D.development.eval) |
|
| 102 |
-
| [REPERE (*phase 2*)](https://islrn.org/resources/360-758-359-485-0/) | 8.17 | 2.23 | 2.49 | 3.45 | [RTTM](https://huggingface.co/pyannote/speaker-diarization/blob/2.1/reproducible_research/2.1.1/REPERE.test.rttm) | [eval](https://huggingface.co/pyannote/speaker-diarization/blob/2.1/reproducible_research/2.1.1/REPERE.test.eval) |
|
| 103 |
-
| [This American Life](https://arxiv.org/abs/2005.08072) | 20.82 | 2.03 | 11.89 | 6.90 | [RTTM](https://huggingface.co/pyannote/speaker-diarization/blob/2.1/reproducible_research/2.1.1/ThisAmericanLife.test.rttm) | [eval](https://huggingface.co/pyannote/speaker-diarization/blob/2.1/reproducible_research/2.1.1/ThisAmericanLife.test.eval) |
|
| 104 |
-
| [VoxConverse (*v0.3*)](https://github.com/joonson/voxconverse) | 11.24 | 4.42 | 2.88 | 3.94 | [RTTM](https://huggingface.co/pyannote/speaker-diarization/blob/main/reproducible_research/2.1.1/VoxConverse.test.rttm)
|
| 105 |
|
| 106 |
## Technical report
|
| 107 |
|
|
|
|
| 89 |
* evaluation of overlapped speech
|
| 90 |
|
| 91 |
|
| 92 |
+
| Benchmark | [DER%](. "Diarization error rate") | [FA%](. "False alarm rate") | [Miss%](. "Missed detection rate") | [Conf%](. "Speaker confusion rate") | Expected output | File-level evaluation |
|
| 93 |
+
| ------------------------------------------------------------------------------------------------------------------------------------------- | ---------------------------------- | --------------------------- | ---------------------------------- | ----------------------------------- | ----------------------------------------------------------------------------------------------------------------------------- | ----------------------------------------------------------------------------------------------------------------------------- |
|
| 94 |
+
| [AISHELL-4](http://www.openslr.org/111/) | 14.09 | 5.17 | 3.27 | 5.65 | [RTTM](https://huggingface.co/pyannote/speaker-diarization/blob/2.1.1/reproducible_research/2.1.1/AISHELL.test.rttm) | [eval](https://huggingface.co/pyannote/speaker-diarization/blob/2.1.1/reproducible_research/2.1.1/AISHELL.test.eval) |
|
| 95 |
+
| [Albayzin (*RTVE 2022*)](http://catedrartve.unizar.es/albayzindatabases.html) | 25.60 | 5.58 | 6.84 | 13.18 | [RTTM](https://huggingface.co/pyannote/speaker-diarization/blob/2.1.1/reproducible_research/2.1.1/Albayzin2022.test.rttm) | [eval](https://huggingface.co/pyannote/speaker-diarization/blob/2.1.1/reproducible_research/2.1.1/Albayzin2022.test.eval) |
|
| 96 |
+
| [AliMeeting (*channel 1*)](https://www.openslr.org/119/) | 27.42 | 4.84 | 14.00 | 8.58 | [RTTM](https://huggingface.co/pyannote/speaker-diarization/blob/2.1.1/reproducible_research/2.1.1/AliMeeting.test.rttm) | [eval](https://huggingface.co/pyannote/speaker-diarization/blob/2.1.1/reproducible_research/2.1.1/AliMeeting.test.eval) |
|
| 97 |
+
| [AMI (*headset mix,*](https://groups.inf.ed.ac.uk/ami/corpus/) [*only_words*)](https://github.com/BUTSpeechFIT/AMI-diarization-setup) | 18.91 | 4.48 | 9.51 | 4.91 | [RTTM](https://huggingface.co/pyannote/speaker-diarization/blob/2.1.1/reproducible_research/2.1.1/AMI.test.rttm) | [eval](https://huggingface.co/pyannote/speaker-diarization/blob/2.1.1/reproducible_research/2.1.1/AMI.test.eval) |
|
| 98 |
+
| [AMI (*array1, channel 1,*](https://groups.inf.ed.ac.uk/ami/corpus/) [*only_words)*](https://github.com/BUTSpeechFIT/AMI-diarization-setup) | 27.12 | 4.11 | 17.78 | 5.23 | [RTTM](https://huggingface.co/pyannote/speaker-diarization/blob/2.1.1/reproducible_research/2.1.1/AMI-SDM.test.rttm) | [eval](https://huggingface.co/pyannote/speaker-diarization/blob/2.1.1/reproducible_research/2.1.1/AMI-SDM.test.eval) |
|
| 99 |
+
| [CALLHOME](https://catalog.ldc.upenn.edu/LDC2001S97) [(*part2*)](https://github.com/BUTSpeechFIT/CALLHOME_sublists/issues/1) | 32.37 | 6.30 | 13.72 | 12.35 | [RTTM](https://huggingface.co/pyannote/speaker-diarization/blob/2.1.1/reproducible_research/2.1.1/CALLHOME.test.rttm) | [eval](https://huggingface.co/pyannote/speaker-diarization/blob/2.1.1/reproducible_research/2.1.1/CALLHOME.test.eval) |
|
| 100 |
+
| [DIHARD 3 (*Full*)](https://arxiv.org/abs/2012.01477) | 26.94 | 10.50 | 8.41 | 8.03 | [RTTM](https://huggingface.co/pyannote/speaker-diarization/blob/2.1.1/reproducible_research/2.1.1/DIHARD.test.rttm) | [eval](https://huggingface.co/pyannote/speaker-diarization/blob/2.1.1/reproducible_research/2.1.1/DIHARD.test.eval) |
|
| 101 |
+
| [Ego4D *v1 (validation)*](https://arxiv.org/abs/2110.07058) | 63.99 | 3.91 | 44.42 | 15.67 | [RTTM](https://huggingface.co/pyannote/speaker-diarization/blob/2.1.1/reproducible_research/2.1.1/Ego4D.development.rttm) | [eval](https://huggingface.co/pyannote/speaker-diarization/blob/2.1.1/reproducible_research/2.1.1/Ego4D.development.eval) |
|
| 102 |
+
| [REPERE (*phase 2*)](https://islrn.org/resources/360-758-359-485-0/) | 8.17 | 2.23 | 2.49 | 3.45 | [RTTM](https://huggingface.co/pyannote/speaker-diarization/blob/2.1.1/reproducible_research/2.1.1/REPERE.test.rttm) | [eval](https://huggingface.co/pyannote/speaker-diarization/blob/2.1.1/reproducible_research/2.1.1/REPERE.test.eval) |
|
| 103 |
+
| [This American Life](https://arxiv.org/abs/2005.08072) | 20.82 | 2.03 | 11.89 | 6.90 | [RTTM](https://huggingface.co/pyannote/speaker-diarization/blob/2.1.1/reproducible_research/2.1.1/ThisAmericanLife.test.rttm) | [eval](https://huggingface.co/pyannote/speaker-diarization/blob/2.1.1/reproducible_research/2.1.1/ThisAmericanLife.test.eval) |
|
| 104 |
+
| [VoxConverse (*v0.3*)](https://github.com/joonson/voxconverse) | 11.24 | 4.42 | 2.88 | 3.94 | [RTTM](https://huggingface.co/pyannote/speaker-diarization/blob/main/reproducible_research/2.1.1/VoxConverse.test.rttm) | [eval](https://huggingface.co/pyannote/speaker-diarization/blob/main/reproducible_research/2.1.1/VoxConverse.test.eval) |
|
| 105 |
|
| 106 |
## Technical report
|
| 107 |
|