How to migrate a Quartznet 15x5  model from NeMo 0.10.1 to 1.0.0b3?
State: closed | Pull request: no

Greetings!
I've got 4 files of a Quartznet 15x5 model:
JasperDecoderForCTC-STEP-520000.pt
JasperEncoder-STEP-520000.pt
lm.binary
quartznet15x5_ru.yaml
The model was trained using NeMo 0.10.1.

I want to fine tune this model using NeMo 1.0.0b3.
Is there any script to convert a model from old versions to the new format?
I've compared .yaml files format of 0.10.1 and 1.0.0b3 and it seemed to be completely different.

**Environment overview (please complete the following information)**
Docker container nvcr.io/nvidia/nemo:1.0.0b3

**Additional context**
Add any other context about the problem here.
GPU: Nvidia GTX 980ti


You can try this script https://github.com/NVIDIA/NeMo/blob/main/scripts/asr_checkpoint_port.py 

When i run the command:
python ./asr_checkpoint_port.py --config_path='/workspace/nemo/quartznet_ru/quartznet15x5_ru.yaml' --encoder_ckpt='/workspace/nemo/quartznet_ru/JasperEncoder-STEP-520000.pt' --decoder_ckpt='/workspace/nemo/quartznet_ru/JasperDecoderForCTC-STEP-520000.pt' --output_path='/workspace/nemo/quartznet_ru/quartznet15x5_ru.nemo'

I've got an error:
ModuleNotFoundError: No module named 'ruamel'

Changing from _ruamel.yaml import YAML_ to from _ruamel_yaml import YAML_ inside asr_checkpoint_port.py resolved this error.

But got another one (copy and paste full trace):
root@4506e32dbac2:/workspace/nemo/scripts# python ./asr_checkpoint_port.py --config_path='/workspace/nemo/quartznet_ru/quartznet15x5_ru.yaml' --encoder_ckpt='/workspace/nemo/quartznet_ru/JasperEncoder-STEP-520000.pt' --decoder_ckpt='/workspace/nemo/quartznet_ru/JasperDecoderForCTC-STEP-520000.pt' --output_path='/workspace/nemo/quartznet_ru/quartznet15x5_ru.nemo'
[NeMo W 2021-01-07 10:00:50 experimental:28] Module nemo.collections.asr.data.audio_to_text.AudioToCharDataset is experimental, not ready for production and is not fully supported. Use at your own risk.
[NeMo W 2021-01-07 10:00:50 experimental:28] Module nemo.collections.asr.data.audio_to_text.AudioToBPEDataset is experimental, not ready for production and is not fully supported. Use at your own risk.
[NeMo W 2021-01-07 10:00:50 experimental:28] Module nemo.collections.asr.data.audio_to_text.AudioLabelDataset is experimental, not ready for production and is not fully supported. Use at your own risk.
[NeMo W 2021-01-07 10:00:50 experimental:28] Module nemo.collections.asr.data.audio_to_text.TarredAudioToTextDataset is experimental, not ready for production and is not fully supported. Use at your own risk.
[NeMo W 2021-01-07 10:00:50 experimental:28] Module nemo.collections.asr.data.audio_to_text.TarredAudioToCharDataset is experimental, not ready for production and is not fully supported. Use at your own risk.
[NeMo W 2021-01-07 10:00:50 experimental:28] Module nemo.collections.asr.data.audio_to_text.TarredAudioToBPEDataset is experimental, not ready for production and is not fully supported. Use at your own risk.
[NeMo W 2021-01-07 10:00:50 experimental:28] Module <class 'nemo.collections.asr.losses.ctc.CTCLoss'> is experimental, not ready for production and is not fully supported. Use at your own risk.
[NeMo W 2021-01-07 10:00:50 experimental:28] Module <class 'nemo.collections.asr.data.audio_to_text_dali.AudioToCharDALIDataset'> is experimental, not ready for production and is not fully supported. Use at your own risk.

WARNING, path does not exist: KALDI_ROOT=/mnt/matylda5/iveselyk/Tools/kaldi-trunk
(please add 'export KALDI_ROOT=<your_path>' in your $HOME/.profile)
(or run as: KALDI_ROOT=<your_path> python <your_script>.py)


[NeMo I 2021-01-07 10:00:50 asr_checkpoint_port:52] Creating ASR NeMo 1.0 model
Traceback (most recent call last):
  File "./asr_checkpoint_port.py", line 71, in <module>
    main(args.config_path, args.encoder_ckpt, args.decoder_ckpt, args.output_path, args.model_type)
  File "./asr_checkpoint_port.py", line 53, in main
    model = nemo_asr.models.EncDecCTCModel(cfg=DictConfig(params['model']))
  File "/opt/conda/lib/python3.6/site-packages/omegaconf/dictconfig.py", line 81, in __init__
    self._set_value(content)
  File "/opt/conda/lib/python3.6/site-packages/omegaconf/dictconfig.py", line 549, in _set_value
    raise ValidationError(msg=msg)  # pragma: no cover
omegaconf.errors.ValidationError


looks like ruamel isn't properly installed in your environment

I'm getting the same error of "pragma: no cover omegaconf.errors.ValidationError"
and "Unsupported value type : QuartzNet"

I figured out the problem -- I changed the yaml file to be like this: https://github.com/NVIDIA/NeMo/blob/main/examples/asr/conf/quartznet_15x5.yaml

I believe I was using an old version of the yaml file that includes "AudioToTextDataLayer", "AudioToMelSpectrogramPreprocessor", "JasperEncoder", etc.