Spaces:
Running
Running
# Few-Shot Voice Cloning | |
This repository is an implementation of the pipeline for few-short voice cloning based on SpeechT5 architecture introduced in [ SpeechT5: Unified-Modal Encoder-Decoder Pre-Training for Spoken Language Processing](https://arxiv.org/abs/2110.07205). | |
It is able to clone a voice from 15-30 seconds of audio recording in English (another languages are planned). | |
# Getting Started | |
Clone repository | |
```angular2html | |
git clone https://github.com/konverner/deep-voice-cloning.git | |
``` | |
Install the modules | |
```angular2html | |
pip install . | |
``` | |
Run traning specifying arguments using config file `training_config.json` or the console command, for example | |
```angular2html | |
python scripts/train.py --audio_path scripts/input/hank.mp3 --output_dir /content/deep-voice-cloning/models | |
``` | |
Resulting model will be saved in `output_dir` directory. It will be used in the next step. | |
Run inference specifying arguments using config file `inference_config.json` or the console command, for example | |
```angular2html | |
python scripts/cloning_inference.py --model_path "/content/deep-voice-cloning/models/microsoft_speecht5_tts_hank"\ | |
--input_text 'do the things, not because they are easy, but because they are hard'\ | |
--output_path "scripts/output/do_the_things.wav" | |
``` | |
Resulting audio file will be saved as `output_path` file. | |
# Docker | |
To build docker image: | |
``` | |
docker build -t deep-voice-cloning . | |
``` | |
To pull docker image from Hub: | |
```angular2html | |
docker pull konverner/deep-voice-cloning:latest | |
``` | |
To run image in a container: | |
``` | |
docker run -it --entrypoint=/bin/bash konverner/deep-voice-cloning | |
``` | |
To run training in a container for example: | |
``` | |
python scripts/train.py --audio_path scripts/input/hank.mp3 --output_dir models | |
``` | |
To run inference in a container for example: | |
``` | |
python scripts/cloning_inference.py --model_path models/microsoft_speecht5_tts_hank --input_text "do the things, not because they are easy, but because they are hard" --output_path scripts/output/do_the_things.wav | |
``` | |
# Notebook Examples | |
Example of using CLI for training and inference can be found in [notebook](https://github.com/konverner/deep-voice-cloning/blob/main/notebooks/CLI_Example.ipynb) | |