Package: crank-vc
crankNon-parallel voice conversion based on vector-quantized variational autoencoder with adversarial learningSetupInstall Python dependency$gitclonehttps://github.com/k2kobayashi/crank.git
$cdcrank/tools
$makeinstall dependency for mosnet$sudoaptinstallffmpeg# mosnet dependencyRecipesEnglishVCC2020VCC2018 (Thanks to@unilight)Japanesejsv_ver1Conversion samplesYou can access several converted audio samples of VCC 2018 dataset in theURL.vcc2020v1vcc2018v1Run VCC2020 recipecrank has prepared recipe for Voice Conversion Challenge 2020.
In crank recipe, there are 7 stages to implement non-parallel voice conversion.stage 0download datasetstage 1initializationgenerate scp files and figures to be determine speaker-dependent parametersstage 2feature extractionextract mlfb and mcep featuresstage 3trainingstage 4reconstuctiongenerate reconstructed feature for fine-tuning of neural vocoderstage 5evaluationconvert evaluation waveformstage 6synthesissynthesis waveform by pre-trainedParallelWaveGANsynthesis waveform by GriffinLimstage 7objective evalutionmel-cepstrum distortionmosnetPut dataset to downloadsNote that dataset is only released for the participants (2020/05/26).$ cd egs/vaevc/vcc2020v1
$ mkdir downloads && cd downloads
$ mv <path_to_zip>/vcc2020_{training,evaluation}.zip downloads
$ unzip vcc2020_training.zip
$ unzip vcc2020_evaluation.zipRun feature extraction and model trainingBecause the challenge defines its training and evaluation set, we have initially put configuration files.
So, you need to run from 2nd stage.$./run.sh--n_jobs10--stage2--stop_stage5where then_jobsindicates the number of CPU cores used in the training.ConfigurationConfigurations are defined inconf/mlfb_vqvae.yml.
Followings are explanation of representative parameters.featureWhen you create your own recipe, be carefull to set parameters for feature extraction such asfs,fftl,hop_size,framems,shiftms, andmcep_alpha. These parameters depend on sampling frequency.feat_typeYou can choosefeat_typeeithermlfbormcep.
If you choosemlfb, the converted waveforms are generated by either GllifinLim vocoder or ParallelWaveGAN vocoder.
If you choosemcep, the converted waveforms are generated by world vocoder (i.e., excitation generation and MLSA filtering).trainer_typeWe support training withvqvae,lsgan,cyclegan, andstarganusing same generator network.vqvae: default vqvae settinglsgan: vqvae with adversarial learningcyclegan: vqvae with adevesarial learning and cyclic constraintsstargan: vqvae with adevesarial learning similar to cycleganCreate your recipeCopy recipe templatePlease copy template directory to start creation of your recipe.$cp-regs/vaevc/templateegs/vaevc/<new_recipe>
$cdegs/vaevc/<new_recipe>Put .wav filesYou need to put wav files appropriate directory.
You can choose either modifyingdownload.shor putting wav files.
In either case, the wav files should be located in each speaker like following<new_recipe>/downloads/wav/{spkr1, spkr2, ..., spkr3}/*.wav.If you modifydownaload.sh,$vimlocal/download.shIf you put wav files,$mkdirdownloads
$mv<path_to_your_wav_directory>downloads/wav
$touchdownloads/.doneRun initializationThe initialization process generates kaldi-like scp files.$./run.sh--stage0--stop_stage1Then you modify speaker-dependent parameters inconf/spkr.ymlusing generated figures.
Page 20~22 inslidehelp you how to set these parameters.Run feature extraction, train, reconstruction, and evaluationAfter preparing configuration, you run it.$./run.sh--stage2--stop_stage7CitationPlease cite this paper when you use crank.K. Kobayashi, W-C. Huang, Y-C. Wu, P.L. Tobing, T. Hayashi, T. Toda,
"crank: an open-source software for nonparallel voice conversion based on vector-quantized variational autoencoder",
Proc. ICASSP, 2021. (accepted)AchknowledgementsThank you@kan-bayashifor lots of contributions and encouragement helps.Who we areKazuhiro Kobayashi@k2kobayashi[maintainer, design and development]Wen-Chin Huang@unilight[maintainer, design and development]Tomoki Toda[advisor]