Package: hifi-gan
NOTE: I (@zachwe) didn't create this project and deserve none of the credit for
it. All of the credit for this package should go to Jungil Kong, Jaehyeon
Kim, Jaekyoung Bae. Many thanks to them for creating such an awesome vocoder.
All I did was package it to make it easier to use.HiFi-GAN: Generative Adversarial Networks for Efficient and High Fidelity Speech SynthesisJungil Kong, Jaehyeon Kim, Jaekyoung BaeIn ourpaper,
we proposed HiFi-GAN: a GAN-based model capable of generating high fidelity speech efficiently.We provide our implementation and pretrained models as open source in this repository.Abstract :Several recent work on speech synthesis have employed generative adversarial networks (GANs) to produce raw waveforms.
Although such methods improve the sampling efficiency and memory usage,
their sample quality has not yet reached that of autoregressive and flow-based generative models.
In this work, we propose HiFi-GAN, which achieves both efficient and high-fidelity speech synthesis.
As speech audio consists of sinusoidal signals with various periods,
we demonstrate that modeling periodic patterns of an audio is crucial for enhancing sample quality.
A subjective human evaluation (mean opinion score, MOS) of a single speaker dataset indicates that our proposed method
demonstrates similarity to human quality while generating 22.05 kHz high-fidelity audio 167.9 times faster than
real-time on a single V100 GPU. We further show the generality of HiFi-GAN to the mel-spectrogram inversion of unseen
speakers and end-to-end speech synthesis. Finally, a small footprint version of HiFi-GAN generates samples 13.4 times
faster than real-time on CPU with comparable quality to an autoregressive counterpart.Visit ourdemo websitefor audio samples.Pre-requisitesPython >= 3.6Clone this repository.Install python requirements. Please referrequirements.txtDownload and extract theLJ Speech dataset.
And move all wav files toLJSpeech-1.1/wavsTrainingpython train.py --config config_v1.jsonTo train V2 or V3 Generator, replaceconfig_v1.jsonwithconfig_v2.jsonorconfig_v3.json.Checkpoints and copy of the configuration file are saved incp_hifigandirectory by default.You can change the path by adding--checkpoint_pathoption.Validation loss during training with V1 generator.Pretrained ModelYou can also use pretrained models we provide.Download pretrained modelsDetails of each folder are as in follows:Folder NameGeneratorDatasetFine-TunedLJ_V1V1LJSpeechNoLJ_V2V2LJSpeechNoLJ_V3V3LJSpeechNoLJ_FT_T2_V1V1LJSpeechYes (Tacotron2)LJ_FT_T2_V2V2LJSpeechYes (Tacotron2)LJ_FT_T2_V3V3LJSpeechYes (Tacotron2)VCTK_V1V1VCTKNoVCTK_V2V2VCTKNoVCTK_V3V3VCTKNoUNIVERSAL_V1V1UniversalNoWe provide the universal model with discriminator weights that can be used as a base for transfer learning to other datasets.Fine-TuningGenerate mel-spectrograms in numpy format usingTacotron2with teacher-forcing.The file name of the generated mel-spectrogram should match the audio file and the extension should be.npy.Example:Audio File : LJ001-0001.wav
Mel-Spectrogram File : LJ001-0001.npyCreateft_datasetfolder and copy the generated mel-spectrogram files into it.Run the following command.python train.py --fine_tuning True --config config_v1.jsonFor other command line options, please refer to the training section.Inference from wav fileMaketest_filesdirectory and copy wav files into the directory.Run the following command.python inference.py --checkpoint_file [generator checkpoint file path]Generated wav files are saved ingenerated_filesby default.You can change the path by adding--output_diroption.Inference for end-to-end speech synthesisMaketest_mel_filesdirectory and copy generated mel-spectrogram files into the directory.You can generate mel-spectrograms usingTacotron2,Glow-TTSand so forth.Run the following command.python inference_e2e.py --checkpoint_file [generator checkpoint file path]Generated wav files are saved ingenerated_files_from_melby default.You can change the path by adding--output_diroption.AcknowledgementsWe referred toWaveGlow,MelGANandTacotron2to implement this.