File size: 1,526 Bytes
c92d3b5 |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 |
---
datasets:
- librispeech_asr
language:
- en
metrics:
- wer
tags:
- hubert
- tts
---
# voidful/mhubert-unit-tts
voidful/mhubert-unit-tts
This repository provides a text to unit model form mhubert and trained with bart model.
The model was trained on the LibriSpeech ASR dataset for the English language and
Train epoch 13: `WER:30.41` `CER: 20.22`
Hubert Code TTS Example
```python
import asrp
import nlp2
import IPython.display as ipd
from transformers import AutoTokenizer, AutoModelForSeq2SeqLM
nlp2.download_file(
'https://dl.fbaipublicfiles.com/fairseq/speech_to_speech/vocoder/code_hifigan/mhubert_vp_en_es_fr_it3_400k_layer11_km1000_lj/g_00500000',
'./')
tokenizer = AutoTokenizer.from_pretrained("voidful/mhubert-unit-tts")
model = AutoModelForSeq2SeqLM.from_pretrained("voidful/mhubert-unit-tts")
model.eval()
cs = asrp.Code2Speech(tts_checkpoint='./g_00500000', vocoder='hifigan')
inputs = tokenizer(["The quick brown fox jumps over the lazy dog."], return_tensors="pt")
code = tokenizer.batch_decode(model.generate(**inputs,max_length=1024))[0]
code = [int(i) for i in code.replace("</s>","").replace("<s>","").split("v_tok_")[1:]]
print(code)
ipd.Audio(data=cs(code), autoplay=False, rate=cs.sample_rate)
```
Datasets
The model was trained on the LibriSpeech ASR dataset for the English language.
Language
The model is trained for the English language.
Metrics
The model's performance is evaluated using Word Error Rate (WER).
Tags
The model can be tagged with "hubert" and "tts".
|