File size: 2,475 Bytes
09597a3 15b443f b21b5f6 09597a3 703fa02 09597a3 c331491 862c1da 09597a3 862c1da 09597a3 caa0f76 09597a3 |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 |
---
license: apache-2.0
tags:
- audio
- speech
- speaker
- speaker-recognition
- speaker-embedding
- speaker-verification
- speaker-identification
- speaker-encoder
- tflite
- voice
library_name: sidlingvo
---
# Conformer based multilingual speaker encoder
## Summary
This is a massively multilingual conformer-based speaker recognition model.
The model was trained with public data only, using the GE2E loss.
Papers:
* Multilingual: https://arxiv.org/abs/2104.02125
* GE2E loss: https://arxiv.org/abs/1710.10467
```
@inproceedings{chojnacka2021speakerstew,
title={{SpeakerStew: Scaling to many languages with a triaged multilingual text-dependent and text-independent speaker verification system}},
author={Chojnacka, Roza and Pelecanos, Jason and Wang, Quan and Moreno, Ignacio Lopez},
booktitle={Prod. Interspeech},
pages={1064--1068},
year={2021},
doi={10.21437/Interspeech.2021-646},
issn={2958-1796},
}
@inproceedings{wan2018generalized,
title={Generalized end-to-end loss for speaker verification},
author={Wan, Li and Wang, Quan and Papir, Alan and Moreno, Ignacio Lopez},
booktitle={International Conference on Acoustics, Speech and Signal Processing (ICASSP)},
pages={4879--4883},
year={2018},
organization={IEEE}
}
```
## Usage
Run use this model, you will need to use the `siglingvo` library: https://github.com/google/speaker-id/tree/master/lingvo
Since lingvo does not support Python 3.11 yet, make sure your Python is up to 3.10.
Install the library:
```
pip install sidlingvo
```
Example usage:
```Python
import os
from sidlingvo import wav_to_dvector
from huggingface_hub import hf_hub_download
repo_id = "tflite-hub/conformer-speaker-encoder"
model_path = "models"
hf_hub_download(repo_id=repo_id, filename="vad_long_model.tflite", local_dir=model_path)
hf_hub_download(repo_id=repo_id, filename="vad_long_mean_stddev.csv", local_dir=model_path)
hf_hub_download(repo_id=repo_id, filename="conformer_tisid_medium.tflite", local_dir=model_path)
enroll_wav_files = ["your_first_wav_file.wav"]
test_wav_file = "your_second_wav_file.wav"
runner = wav_to_dvector.WavToDvectorRunner(
vad_model_file=os.path.join(model_path, "vad_long_model.tflite"),
vad_mean_stddev_file=os.path.join(model_path, "vad_long_mean_stddev.csv"),
tisid_model_file=os.path.join(model_path, "conformer_tisid_medium.tflite"))
score = runner.compute_score(enroll_wav_files, test_wav_file)
print("Speaker similarity score:", score)
``` |