Package: espnet-model-zoo
ESPnet Model ZooUtilities managing the pretrained models created byESPnet. This function is inspired by theAsteroid pretrained model function.From version 0.1.0, the huggingface models can be also used:https://huggingface.co/models?filter=espnetZenodo community:https://zenodo.org/communities/espnet/Registered models:table.csvInstallpip install torch
pip install espnet_model_zooPython API for inferencemodel_namein the following section should behuggingface_idor one of the tags in thetable.csv.
Or you can directly provide zenodo URL (e.g.,https://zenodo.org/record/xxxxxxx/files/hogehoge.zip?download=1).ASRimportsoundfilefromespnet2.bin.asr_inferenceimportSpeech2Textspeech2text=Speech2Text.from_pretrained("model_name",# Decoding parameters are not included in the model filemaxlenratio=0.0,minlenratio=0.0,beam_size=20,ctc_weight=0.3,lm_weight=0.5,penalty=0.0,nbest=1)# Confirm the sampling rate is equal to that of the training corpus.# If not, you need to resample the audio data before inputting to speech2textspeech,rate=soundfile.read("speech.wav")nbests=speech2text(speech)text,*_=nbests[0]print(text)TTSimportsoundfilefromespnet2.bin.tts_inferenceimportText2Speechtext2speech=Text2Speech.from_pretrained("model_name")speech=text2speech("foobar")["wav"]soundfile.write("out.wav",speech.numpy(),text2speech.fs,"PCM_16")Speech separationimportsoundfilefromespnet2.bin.enh_inferenceimportSeparateSpeechseparate_speech=SeparateSpeech.from_pretrained("model_name",# for segment-wise process on long speechsegment_size=2.4,hop_size=0.8,normalize_segment_scale=False,show_progressbar=True,ref_channel=None,normalize_output_wav=True,)# Confirm the sampling rate is equal to that of the training corpus.# If not, you need to resample the audio data before inputting to speech2textspeech,rate=soundfile.read("long_speech.wav")waves=separate_speech(speech[None,...],fs=rate)This API allows processing both short audio samples and long audio samples. For long audio samples, you can set the value of arguments segment_size, hop_size (optionally normalize_segment_scale and show_progressbar) to perform segment-wise speech enhancement/separation on the input speech. Note that the segment-wise processing is disabled by default.For old ESPnet (<=10.1)ASRimportsoundfilefromespnet_model_zoo.downloaderimportModelDownloaderfromespnet2.bin.asr_inferenceimportSpeech2Textd=ModelDownloader()speech2text=Speech2Text(**d.download_and_unpack("model_name"),# Decoding parameters are not included in the model filemaxlenratio=0.0,minlenratio=0.0,beam_size=20,ctc_weight=0.3,lm_weight=0.5,penalty=0.0,nbest=1)TTSimportsoundfilefromespnet_model_zoo.downloaderimportModelDownloaderfromespnet2.bin.tts_inferenceimportText2Speechd=ModelDownloader()text2speech=Text2Speech(**d.download_and_unpack("model_name"))Speech separationimportsoundfilefromespnet_model_zoo.downloaderimportModelDownloaderfromespnet2.bin.enh_inferenceimportSeparateSpeechd=ModelDownloader()separate_speech=SeparateSpeech(**d.download_and_unpack("model_name"),# for segment-wise process on long speechsegment_size=2.4,hop_size=0.8,normalize_segment_scale=False,show_progressbar=True,ref_channel=None,normalize_output_wav=True,)Instruction for ModelDownloaderfromespnet_model_zoo.downloaderimportModelDownloaderd=ModelDownloader("~/.cache/espnet")# Specify cachedird=ModelDownloader()# <module_dir> is used as cachedir by defaultTo obtain a model, you need to give ahuggingface_idmodel` or a tag , which is listed intable.csv.>>>d.download_and_unpack("kamo-naoyuki/mini_an4_asr_train_raw_bpe_valid.acc.best"){"asr_train_config":<configpath>,"asr_model_file":<modelpath>,...}You can specify the revision if it's huggingface_id giving with@:>>>d.download_and_unpack("kamo-naoyuki/mini_an4_asr_train_raw_bpe_valid.acc.best@<revision>"){"asr_train_config":<configpath>,"asr_model_file":<modelpath>,...}Note that if the model already exists, you can skip downloading and unpacking.You can also get a model with certain conditions.d.download_and_unpack(task="asr",corpus="wsj")If multiple models are found with the condition, the last model is selected.
You can also specify the condition using "version" option.d.download_and_unpack(task="asr",corpus="wsj",version=-1)# Get the last modeld.download_and_unpack(task="asr",corpus="wsj",version=-2)# Get previous modelYou can also obtain it from the URL directly.d.download_and_unpack("https://zenodo.org/record/...")If you need to use a local model file using this API, you can also give it.d.download_and_unpack("./some/where/model.zip")In this case, the contents are also expanded in the cache directory,
but the model is identified by the file path,
so if you move the model to somewhere and unpack again,
it's treated as another model,
thus the contents are expanded again at another place.Query model namesYou can view the model names from our Zenodo community,https://zenodo.org/communities/espnet/,
or usingquery().  All information are written intable.csv.d.query("name")You can also show them with specifying certain conditions.d.query("name",task="asr")Command line toolsespnet_model_zoo_query# Query model nameespnet_model_zoo_querytask=asrcorpus=wsj# Show all model nameespnet_model_zoo_query# Query the other keyespnet_model_zoo_query--keyurltask=asrcorpus=wsjespnet_model_zoo_downloadespnet_model_zoo_download<model_name># Print the path of the downloaded fileespnet_model_zoo_download--unpacktrue<model_name># Print the path of unpacked filesespnet_model_zoo_uploadexportACCESS_TOKEN=<access_token>
espnet_zenodo_upload\--file<packed_model>\--title<title>\--description<description>\--creator_name<your-git-account>Use pretrained model in ESPnet recipe# e.g. ASR WSJ taskgitclonehttps://github.com/espnet/espnet
pipinstall-e.cdegs2/wsj/asr1
./run.sh--skip_data_prepfalse--skip_traintrue--download_modelkamo-naoyuki/wsjRegister your modelHuggingfaceUpload your model using huggingface APIComing soon...Create a Pull Request to modifytable.csvThe models registered in thistable.csv, the model are tested in the CI.
Indeed, the model can be downloaded without modificationtable.csv.(Administrator does) Increment the third version number ofsetup.py, e.g. 0.0.3 -> 0.0.4(Administrator does) Release new versionZenodo (Obsolete)Upload your model to ZenodoYou need tosignup to Zenodoandcreate an access tokento upload models.
You can upload your own model by usingespnet_model_zoo_uploadcommand freely,
but we normally upload a model usingrecipes.Create a Pull Request to modifytable.csvYou need to append your record at the last line.(Administrator does) Increment the third version number ofsetup.py, e.g. 0.0.3 -> 0.0.4(Administrator does) Release new version