File size: 5,623 Bytes
a9384d7 |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 |
# Fine-tuning a 🐸 TTS model
## Fine-tuning
Fine-tuning takes a pre-trained model and retrains it to improve the model performance on a different task or dataset.
In 🐸TTS we provide different pre-trained models in different languages and different pros and cons. You can take one of
them and fine-tune it for your own dataset. This will help you in two main ways:
1. Faster learning
Since a pre-trained model has already learned features that are relevant for the task, it will converge faster on
a new dataset. This will reduce the cost of training and let you experiment faster.
2. Better results with small datasets
Deep learning models are data hungry and they give better performance with more data. However, it is not always
possible to have this abundance, especially in specific domains. For instance, the LJSpeech dataset, that we released most of
our English models with, is almost 24 hours long. It takes weeks to record this amount of data with
the help of a voice actor.
Fine-tuning comes to the rescue in this case. You can take one of our pre-trained models and fine-tune it on your own
speech dataset and achieve reasonable results with only a couple of hours of data.
However, note that, fine-tuning does not ensure great results. The model performance is still depends on the
{ref}`dataset quality <what_makes_a_good_dataset>` and the hyper-parameters you choose for fine-tuning. Therefore,
it still takes a bit of tinkering.
## Steps to fine-tune a 🐸 TTS model
1. Setup your dataset.
You need to format your target dataset in a certain way so that 🐸TTS data loader will be able to load it for the
training. Please see {ref}`this page <formatting_your_dataset>` for more information about formatting.
2. Choose the model you want to fine-tune.
You can list the available models in the command line with
```bash
tts --list_models
```
The command above lists the the models in a naming format as ```<model_type>/<language>/<dataset>/<model_name>```.
Or you can manually check the `.model.json` file in the project directory.
You should choose the model based on your requirements. Some models are fast and some are better in speech quality.
One lazy way to test a model is running the model on the hardware you want to use and see how it works. For
simple testing, you can use the `tts` command on the terminal. For more info see {ref}`here <synthesizing_speech>`.
3. Download the model.
You can download the model by using the `tts` command. If you run `tts` with a particular model, it will download it automatically
and the model path will be printed on the terminal.
```bash
tts --model_name tts_models/es/mai/tacotron2-DDC --text "Ola."
> Downloading model to /home/ubuntu/.local/share/tts/tts_models--en--ljspeech--glow-tts
...
```
In the example above, we called the Spanish Tacotron model and give the sample output showing use the path where
the model is downloaded.
4. Setup the model config for fine-tuning.
You need to change certain fields in the model config. You have 3 options for playing with the configuration.
1. Edit the fields in the ```config.json``` file if you want to use ```TTS/bin/train_tts.py``` to train the model.
2. Edit the fields in one of the training scripts in the ```recipes``` directory if you want to use python.
3. Use the command-line arguments to override the fields like ```--coqpit.lr 0.00001``` to change the learning rate.
Some of the important fields are as follows:
- `datasets` field: This is set to the dataset you want to fine-tune the model on.
- `run_name` field: This is the name of the run. This is used to name the output directory and the entry in the
logging dashboard.
- `output_path` field: This is the path where the fine-tuned model is saved.
- `lr` field: You may need to use a smaller learning rate for fine-tuning to not lose the features learned by the
pre-trained model with big update steps.
- `audio` fields: Different datasets have different audio characteristics. You must check the current audio parameters and
make sure that the values reflect your dataset. For instance, your dataset might have a different audio sampling rate.
Apart from the parameters above, you should check the whole configuration file and make sure that the values are correct for
your dataset and training.
5. Start fine-tuning.
Whether you use one of the training scripts under ```recipes``` folder or the ```train_tts.py``` to start
your training, you should use the ```--restore_path``` flag to specify the path to the pre-trained model.
```bash
CUDA_VISIBLE_DEVICES="0" python recipes/ljspeech/glow_tts/train_glowtts.py \
--restore_path /home/ubuntu/.local/share/tts/tts_models--en--ljspeech--glow-tts/model_file.pth
```
```bash
CUDA_VISIBLE_DEVICES="0" python TTS/bin/train_tts.py \
--config_path /home/ubuntu/.local/share/tts/tts_models--en--ljspeech--glow-tts/config.json \
--restore_path /home/ubuntu/.local/share/tts/tts_models--en--ljspeech--glow-tts/model_file.pth
```
As stated above, you can also use command-line arguments to change the model configuration.
```bash
CUDA_VISIBLE_DEVICES="0" python recipes/ljspeech/glow_tts/train_glowtts.py \
--restore_path /home/ubuntu/.local/share/tts/tts_models--en--ljspeech--glow-tts/model_file.pth
--coqpit.run_name "glow-tts-finetune" \
--coqpit.lr 0.00001
```
|