Repository: NVIDIA/NeMo

Issue:
Cannot run r1.6.1 fastpitch_finetune.ipynb from scratch using Colab
**Describe the bug**

Trying to run tts/tutorials/fastpitch_finetune.ipynb from Colab, the first cell give the following error:

`Collecting nemo_toolkit[tts]
  Cloning https://github.com/NeMo/NeMo.git (to revision r1.6.1) to /tmp/pip-install-dikw_cg4/nemo-toolkit_dde310e4647c4385a9c5f59aa12c3537
  Running command git clone -q https://github.com/NeMo/NeMo.git /tmp/pip-install-dikw_cg4/nemo-toolkit_dde310e4647c4385a9c5f59aa12c3537
  fatal: could not read Username for 'https://github.com/': No such device or address
WARNING: Discarding git+https://github.com/NeMo/NeMo.git@r1.6.1#egg=nemo_toolkit[tts]. Command errored out with exit status 128: git clone -q https://github.com/NeMo/NeMo.git /tmp/pip-install-dikw_cg4/nemo-toolkit_dde310e4647c4385a9c5f59aa12c3537 Check the logs for full command output.
ERROR: Could not find a version that satisfies the requirement nemo-toolkit (unavailable) (from versions: 0.11.0b6.linux-x86_64, 0.8, 0.8.1, 0.8.2, 0.9.0, 0.10.0b0, 0.10.0b1, 0.10.0b2, 0.10.0b3, 0.10.0b4, 0.10.0b5, 0.10.0b6, 0.10.0b7, 0.10.0b8, 0.10.0b9, 0.10.0b10, 0.10.0, 0.10.1, 0.11.0b1, 0.11.0b2, 0.11.0b3, 0.11.0b4, 0.11.0b5, 0.11.0b8, 0.11.0b10, 0.11.0b11, 0.11.0b12, 0.11.0b14, 0.11.0, 1.0.0a4, 1.0.0b0, 1.0.0b1, 1.0.0b2, 1.0.0b3, 1.0.0rc1, 1.0.0, 1.0.1, 1.0.2, 1.1.0, 1.2.0, 1.3.0, 1.4.0, 1.5.0, 1.5.1, 1.6.0, 1.6.1)
ERROR: No matching distribution found for nemo-toolkit (unavailable)`

In addition, I've been using fastpitch_finetune.ipynb with my own dataset since months ago (Branch Main), and now I cannot run it anymore **since 3 days ago**. It gives me an error when finetuning hifigan in line 32 (sorry for the long output):

`env: HYDRA_FULL_ERROR=1 [NeMo W 2022-02-04 14:08:27 optimizers:52] Apex was not found. Using the lamb or fused_adam optimizer will error out. ################################################################################ ### WARNING, path does not exist: KALDI_ROOT=/mnt/matylda5/iveselyk/Tools/kaldi-trunk ### (please add 'export KALDI_ROOT=<your_path>' in your $HOME/.profile) ### (or run as: KALDI_ROOT=<your_path> python <your_script>.py) ################################################################################ [NeMo W 2022-02-04 14:08:28 experimental:28] Module <function get_argmin_mat at 0x7f6324ecccb0> is experimental, not ready for production and is not fully supported. Use at your own risk. [NeMo W 2022-02-04 14:08:28 experimental:28] Module <function getMultiScaleCosAffinityMatrix at 0x7f6324eccd40> is experimental, not ready for production and is not fully supported. Use at your own risk. [NeMo W 2022-02-04 14:08:28 experimental:28] Module <function parse_scale_configs at 0x7f6324ecc950> is experimental, not ready for production and is not fully supported. Use at your own risk. [NeMo W 2022-02-04 14:08:28 experimental:28] Module <function get_embs_and_timestamps at 0x7f6324ecc290> is experimental, not ready for production and is not fully supported. Use at your own risk. [NeMo W 2022-02-04 14:08:28 nemo_logging:349] /usr/local/lib/python3.7/dist-packages/hydra/_internal/defaults_list.py:251: UserWarning: In 'hifigan.yaml': Defaults list is missing `_self_`. See https://hydra.cc/docs/upgrades/1.0_to_1.1/default_composition_order for more information warnings.warn(msg, UserWarning) Traceback (most recent call last): File "hifigan_finetune.py", line 32, in <module> main() # noqa pylint: disable=no-value-for-parameter File "/usr/local/lib/python3.7/dist-packages/nemo/core/config/hydra_runner.py", line 105, in wrapper config_name=config_name, File "/usr/local/lib/python3.7/dist-packages/hydra/_internal/utils.py", line 378, in _run_hydra lambda: hydra.run( File "/usr/local/lib/python3.7/dist-packages/hydra/_internal/utils.py", line 214, in run_and_report raise ex File "/usr/local/lib/python3.7/dist-packages/hydra/_internal/utils.py", line 211, in run_and_report return func() File "/usr/local/lib/python3.7/dist-packages/hydra/_internal/utils.py", line 381, in <lambda> overrides=args.overrides, File "/usr/local/lib/python3.7/dist-packages/hydra/_internal/hydra.py", line 92, in run run_mode=RunMode.RUN, File "/usr/local/lib/python3.7/dist-packages/hydra/_internal/hydra.py", line 563, in compose_config from_shell=from_shell, File "/usr/local/lib/python3.7/dist-packages/hydra/_internal/config_loader_impl.py", line 145, in load_configuration from_shell=from_shell, File "/usr/local/lib/python3.7/dist-packages/hydra/_internal/config_loader_impl.py", line 262, in _load_configuration_impl ConfigLoaderImpl._apply_overrides_to_config(config_overrides, cfg) File "/usr/local/lib/python3.7/dist-packages/hydra/_internal/config_loader_impl.py", line 342, in _apply_overrides_to_config f"Could not delete from config. '{override.key_or_group}' does not exist." hydra.errors.ConfigCompositionException: Could not delete from config. 'model.sched' does not exist.`


**Steps/Code to reproduce bug**

Simply trying to execute the latest tts/tutorials/fastpitch_finetune.ipynb.

**Environment details**

Google Colab and Colab PRO +


Any help on this will be appreciated.

Thanks!


Discussion:
It looks like a bug on Github? Found some info on google but its from many years ago 
About first error: try to use file from main branch: https://github.com/NVIDIA/NeMo/blob/main/tutorials/tts/FastPitch_Finetuning.ipynb. 

About second error: you run this one, right?
```
python examples/tts/hifigan_finetune.py --config_name=hifigan.yaml model.train_ds.dataloader_params.batch_size=32 model.max_steps=1000 ~model.sched model.optim.lr=0.0001 train_dataset=./hifigan_train_ft.json validation_datasets=./hifigan_val_ft.json exp_manager.exp_dir=hifigan_ft +init_from_nemo_model=tts_hifigan.nemo trainer.check_val_every_n_epoch=10 model/train_ds=train_ds_finetune
```

Please, try this one:

```
python examples/tts/hifigan_finetune.py --config_name=hifigan.yaml model.train_ds.dataloader_params.batch_size=32 model.max_steps=1000 ~model.optim.sched model.optim.lr=0.0001 train_dataset=./hifigan_train_ft.json validation_datasets=./hifigan_val_ft.json exp_manager.exp_dir=hifigan_ft +init_from_nemo_model=tts_hifigan.nemo trainer.check_val_every_n_epoch=10 model/train_ds=train_ds_finetune
```

The difference in place where `sched` is to be.
Thanks @Oktai15.

Have tried with the second one, and now I get this error:

usage: hifigan_finetune.py [--help] [--hydra-help] [--version] [--cfg {job,hydra,all}] [--resolve] [--package PACKAGE] [--run] [--multirun] [--shell-completion] [--config-path CONFIG_PATH] [--config-name CONFIG_NAME] [--config-dir CONFIG_DIR] [--info [{all,config,defaults,defaults-tree,plugins,searchpath}]] [overrides [overrides ...]] hifigan_finetune.py: error: unrecognized arguments: --config_name=hifigan.yaml

It sounds weird because the argument remained the same.


It sounds very strange. Please, check carefully how you run script or provide minimal example which will produce this error  
Here are the lines:

```
from nemo.collections.tts.models import HifiGanModel
HifiGanModel.from_pretrained("tts_hifigan")

nemo_files = [p for p in Path("/root/.cache/torch/NeMo/").glob("**/tts_hifigan.nemo")]
print(f"Copying {nemo_files[0]} to ./")
Path("./tts_hifigan.nemo").write_bytes(nemo_files[0].read_bytes())`
!pip install youtokentome
!pip install pynini
!cp /content/conf/hifigan/hifigan.yaml ./
!python hifigan_finetune.py --config_name=hifigan.yaml model.train_ds.dataloader_params.batch_size=32 model.max_steps=1000 ~model.optim.sched model.optim.lr=0.0001 train_dataset=./hifigan_train_ft.json validation_datasets=./hifigan_val_ft.json exp_manager.exp_dir=hifigan_ft +init_from_nemo_model=tts_hifigan.nemo trainer.check_val_every_n_epoch=10 model/train_ds=train_ds_finetune
```
@gedefet, okay, I got the problem. Use `--config-name` instead of `--config_name`. 

You run should be:
```
python examples/tts/hifigan_finetune.py --config-name=hifigan.yaml model.train_ds.dataloader_params.batch_size=32 model.max_steps=1000 ~model.optim.sched model.optim.lr=0.0001 train_dataset=./hifigan_train_ft.json validation_datasets=./hifigan_val_ft.json exp_manager.exp_dir=hifigan_ft +init_from_nemo_model=tts_hifigan.nemo trainer.check_val_every_n_epoch=10 model/train_ds=train_ds_finetune model/validation_ds=val_ds_finetune
```
Thanks @Oktai15 . That worked.

After executing the command, I have another one. It seems like something with the hifigan.py:

```
[NeMo W 2022-02-08 20:54:01 optimizers:55] Apex was not found. Using the lamb or fused_adam optimizer will error out.
################################################################################
### WARNING, path does not exist: KALDI_ROOT=/mnt/matylda5/iveselyk/Tools/kaldi-trunk
### (please add 'export KALDI_ROOT=<your_path>' in your $HOME/.profile)
### (or run as: KALDI_ROOT=<your_path> python <your_script>.py)
################################################################################



[NeMo W 2022-02-08 20:54:01 nemo_logging:349] /usr/local/lib/python3.7/dist-packages/hydra/_internal/defaults_list.py:251: UserWarning: In 'hifigan.yaml': Defaults list is missing `_self_`. See https://hydra.cc/docs/upgrades/1.0_to_1.1/default_composition_order for more information
warnings.warn(msg, UserWarning)

[NeMo W 2022-02-08 20:54:02 nemo_logging:349] /usr/local/lib/python3.7/dist-packages/pytorch_lightning/trainer/connectors/callback_connector.py:148: LightningDeprecationWarning: Setting `Trainer(checkpoint_callback=False)` is deprecated in v1.5 and will be removed in v1.7. Please consider using `Trainer(enable_checkpointing=False)`.
f"Setting `Trainer(checkpoint_callback={checkpoint_callback})` is deprecated in v1.5 and will "

GPU available: True, used: True
TPU available: False, using: 0 TPU cores
IPU available: False, using: 0 IPUs
[NeMo W 2022-02-08 20:54:02 nemo_logging:349] /usr/local/lib/python3.7/dist-packages/pytorch_lightning/trainer/connectors/logger_connector/logger_connector.py:60: LightningDeprecationWarning: Setting `Trainer(flush_logs_every_n_steps=200)` is deprecated in v1.5 and will be removed in v1.7. Please configure flushing in the logger instead.
f"Setting `Trainer(flush_logs_every_n_steps={flush_logs_every_n_steps})` is deprecated in v1.5 "

[NeMo I 2022-02-08 20:54:02 exp_manager:283] Experiments will be logged at hifigan_ft/HifiGan/2022-02-08_20-54-02
[NeMo I 2022-02-08 20:54:02 exp_manager:648] TensorboardLogger has been set up
[NeMo W 2022-02-08 20:54:02 exp_manager:902] The checkpoint callback was told to monitor a validation value and trainer's max_steps was set to 4500. Please ensure that max_steps will run for at least 10 epochs to ensure that checkpointing will not error out.
[NeMo W 2022-02-08 20:54:02 nemo_logging:349] /usr/local/lib/python3.7/dist-packages/pytorch_lightning/callbacks/model_checkpoint.py:244: LightningDeprecationWarning: `ModelCheckpoint(every_n_val_epochs)` is deprecated in v1.4 and will be removed in v1.6. Please use `every_n_epochs` instead.
"`ModelCheckpoint(every_n_val_epochs)` is deprecated in v1.4 and will be removed in v1.6."

[NeMo I 2022-02-08 20:54:02 data:758] Loading dataset from ./hifigan_train_ft.json.
210it [00:00, 107166.79it/s]
[NeMo I 2022-02-08 20:54:02 data:782] Loaded dataset with 210 files.
[NeMo I 2022-02-08 20:54:02 data:784] Dataset contains 0.15 hours.
[NeMo I 2022-02-08 20:54:02 data:292] Pruned 37 files. Final dataset contains 173 files
[NeMo I 2022-02-08 20:54:02 data:295] Pruned 0.00 hours. Final dataset contains 0.15 hours.
[NeMo I 2022-02-08 20:54:02 data:758] Loading dataset from ./hifigan_val_ft.json.
7it [00:00, 46019.01it/s]
[NeMo I 2022-02-08 20:54:02 data:782] Loaded dataset with 7 files.
[NeMo I 2022-02-08 20:54:02 data:784] Dataset contains 0.00 hours.
[NeMo I 2022-02-08 20:54:02 data:292] Pruned 5 files. Final dataset contains 2 files
[NeMo I 2022-02-08 20:54:02 data:295] Pruned 0.00 hours. Final dataset contains 0.00 hours.
[NeMo I 2022-02-08 20:54:02 features:255] PADDING: 0
[NeMo I 2022-02-08 20:54:02 features:272] STFT using torch
[NeMo I 2022-02-08 20:54:02 features:274] STFT using exact pad
[NeMo I 2022-02-08 20:54:02 features:255] PADDING: 0
[NeMo I 2022-02-08 20:54:02 features:272] STFT using torch
[NeMo I 2022-02-08 20:54:02 features:274] STFT using exact pad
[NeMo W 2022-02-08 20:54:06 modelPT:148] If you intend to do training or fine-tuning, please call the ModelPT.setup_training_data() method and provide a valid configuration file to setup the train data loader.
Train config :
dataset:
_target_: nemo.collections.tts.data.datalayers.MelAudioDataset
manifest_filepath: /home/fkreuk/data/train_finetune.txt
min_duration: 0.75
n_segments: 8192
dataloader_params:
drop_last: false
shuffle: true
batch_size: 64
num_workers: 4

[NeMo W 2022-02-08 20:54:06 modelPT:155] If you intend to do validation, please call the ModelPT.setup_validation_data() or ModelPT.setup_multiple_validation_data() method and provide a valid configuration file to setup the validation data loader(s).
Validation config :
dataset:
_target_: nemo.collections.tts.data.datalayers.MelAudioDataset
manifest_filepath: /home/fkreuk/data/val_finetune.txt
min_duration: 3
n_segments: 66150
dataloader_params:
drop_last: false
shuffle: false
batch_size: 5
num_workers: 4

[NeMo W 2022-02-08 20:54:06 features:233] Using torch_stft is deprecated and will be removed in 1.1.0. Please set stft_conv and stft_exact_pad to False for FilterbankFeatures and AudioToMelSpectrogramPreprocessor. Please set exact_pad to True as needed.
[NeMo I 2022-02-08 20:54:06 features:255] PADDING: 0
[NeMo I 2022-02-08 20:54:06 features:272] STFT using torch
[NeMo W 2022-02-08 20:54:06 features:233] Using torch_stft is deprecated and will be removed in 1.1.0. Please set stft_conv and stft_exact_pad to False for FilterbankFeatures and AudioToMelSpectrogramPreprocessor. Please set exact_pad to True as needed.
[NeMo I 2022-02-08 20:54:06 features:255] PADDING: 0
[NeMo I 2022-02-08 20:54:06 features:272] STFT using torch
[NeMo I 2022-02-08 20:54:07 save_restore_connector:154] Model HifiGanModel was successfully restored from /content/tts_hifigan.nemo.
[NeMo I 2022-02-08 20:54:07 modelPT:977] Model checkpoint restored from nemo file with path : `tts_hifigan.nemo`
[NeMo W 2022-02-08 20:54:07 nemo_logging:349] /usr/local/lib/python3.7/dist-packages/pytorch_lightning/trainer/configuration_validator.py:292: LightningDeprecationWarning: Base `Callback.on_train_batch_start` hook signature has changed in v1.5. The `dataloader_idx` argument will be removed in v1.7.
f"Base `Callback.{hook}` hook signature has changed in v1.5."

[NeMo W 2022-02-08 20:54:07 nemo_logging:349] /usr/local/lib/python3.7/dist-packages/pytorch_lightning/trainer/configuration_validator.py:292: LightningDeprecationWarning: Base `Callback.on_train_batch_end` hook signature has changed in v1.5. The `dataloader_idx` argument will be removed in v1.7.
f"Base `Callback.{hook}` hook signature has changed in v1.5."

initializing distributed: GLOBAL_RANK: 0, MEMBER: 1/1
Added key: store_based_barrier_key:1 to store for rank: 0
Rank 0: Completed store-based barrier for key:store_based_barrier_key:1 with 1 nodes.
----------------------------------------------------------------------------------------------------
distributed_backend=nccl
All distributed processes registered. Starting with 1 processes
----------------------------------------------------------------------------------------------------



LOCAL_RANK: 0 - CUDA_VISIBLE_DEVICES: [0]



| Name | Type | Params
------------------------------------------------------------------------
0 | audio_to_melspec_precessor | FilterbankFeatures | 0
1 | trg_melspec_fn | FilterbankFeatures | 0
2 | generator | Generator | 13.9 M
3 | mpd | MultiPeriodDiscriminator | 41.1 M
4 | msd | MultiScaleDiscriminator | 29.6 M
5 | feature_loss | FeatureMatchingLoss | 0
6 | discriminator_loss | DiscriminatorLoss | 0
7 | generator_loss | GeneratorLoss | 0
------------------------------------------------------------------------
84.7 M Trainable params
0 Non-trainable params
84.7 M Total params
338.643 Total estimated model params size (MB)
Validation sanity check: 0it [00:00, ?it/s][NeMo W 2022-02-08 20:54:13 nemo_logging:349] /usr/local/lib/python3.7/dist-packages/pytorch_lightning/trainer/data_loading.py:133: UserWarning: The dataloader, val_dataloader 0, does not have many workers which may be a bottleneck. Consider increasing the value of the `num_workers` argument` (try 8 which is the number of cpus on this machine) in the `DataLoader` init to improve performance.
f"The dataloader, {name}, does not have many workers which may be a bottleneck."

Validation sanity check: 0% 0/1 [00:00<?, ?it/s]Error executing job with overrides: ['model.train_ds.dataloader_params.batch_size=24', 'model.max_steps=4500', '~model.optim.sched', 'model.optim.lr=0.0001', 'train_dataset=./hifigan_train_ft.json', 'validation_datasets=./hifigan_val_ft.json', 'exp_manager.exp_dir=hifigan_ft', '+init_from_nemo_model=tts_hifigan.nemo', 'trainer.check_val_every_n_epoch=10', 'model/train_ds=train_ds_finetune']
Traceback (most recent call last):
File "hifigan_finetune.py", line 32, in <module>
main() # noqa pylint: disable=no-value-for-parameter
File "/usr/local/lib/python3.7/dist-packages/nemo/core/config/hydra_runner.py", line 105, in wrapper
config_name=config_name,
File "/usr/local/lib/python3.7/dist-packages/hydra/_internal/utils.py", line 378, in _run_hydra
lambda: hydra.run(
File "/usr/local/lib/python3.7/dist-packages/hydra/_internal/utils.py", line 214, in run_and_report
raise ex
File "/usr/local/lib/python3.7/dist-packages/hydra/_internal/utils.py", line 211, in run_and_report
return func()
File "/usr/local/lib/python3.7/dist-packages/hydra/_internal/utils.py", line 381, in <lambda>
overrides=args.overrides,
File "/usr/local/lib/python3.7/dist-packages/hydra/_internal/hydra.py", line 111, in run
_ = ret.return_value
File "/usr/local/lib/python3.7/dist-packages/hydra/core/utils.py", line 233, in return_value
raise self._return_value
File "/usr/local/lib/python3.7/dist-packages/hydra/core/utils.py", line 160, in run_job
ret.return_value = task_function(task_cfg)
File "hifigan_finetune.py", line 28, in main
trainer.fit(model)
File "/usr/local/lib/python3.7/dist-packages/pytorch_lightning/trainer/trainer.py", line 741, in fit
self._fit_impl, model, train_dataloaders, val_dataloaders, datamodule, ckpt_path
File "/usr/local/lib/python3.7/dist-packages/pytorch_lightning/trainer/trainer.py", line 685, in _call_and_handle_interrupt
return trainer_fn(*args, **kwargs)
File "/usr/local/lib/python3.7/dist-packages/pytorch_lightning/trainer/trainer.py", line 777, in _fit_impl
self._run(model, ckpt_path=ckpt_path)
File "/usr/local/lib/python3.7/dist-packages/pytorch_lightning/trainer/trainer.py", line 1199, in _run
self._dispatch()
File "/usr/local/lib/python3.7/dist-packages/pytorch_lightning/trainer/trainer.py", line 1279, in _dispatch
self.training_type_plugin.start_training(self)
File "/usr/local/lib/python3.7/dist-packages/pytorch_lightning/plugins/training_type/training_type_plugin.py", line 202, in start_training
self._results = trainer.run_stage()
File "/usr/local/lib/python3.7/dist-packages/pytorch_lightning/trainer/trainer.py", line 1289, in run_stage
return self._run_train()
File "/usr/local/lib/python3.7/dist-packages/pytorch_lightning/trainer/trainer.py", line 1311, in _run_train
self._run_sanity_check(self.lightning_module)
File "/usr/local/lib/python3.7/dist-packages/pytorch_lightning/trainer/trainer.py", line 1375, in _run_sanity_check
self._evaluation_loop.run()
File "/usr/local/lib/python3.7/dist-packages/pytorch_lightning/loops/base.py", line 145, in run
self.advance(*args, **kwargs)
File "/usr/local/lib/python3.7/dist-packages/pytorch_lightning/loops/dataloader/evaluation_loop.py", line 110, in advance
dl_outputs = self.epoch_loop.run(dataloader, dataloader_idx, dl_max_batches, self.num_dataloaders)
File "/usr/local/lib/python3.7/dist-packages/pytorch_lightning/loops/base.py", line 145, in run
self.advance(*args, **kwargs)
File "/usr/local/lib/python3.7/dist-packages/pytorch_lightning/loops/epoch/evaluation_epoch_loop.py", line 122, in advance
output = self._evaluation_step(batch, batch_idx, dataloader_idx)
File "/usr/local/lib/python3.7/dist-packages/pytorch_lightning/loops/epoch/evaluation_epoch_loop.py", line 217, in _evaluation_step
output = self.trainer.accelerator.validation_step(step_kwargs)
File "/usr/local/lib/python3.7/dist-packages/pytorch_lightning/accelerators/accelerator.py", line 236, in validation_step
return self.training_type_plugin.validation_step(*step_kwargs.values())
File "/usr/local/lib/python3.7/dist-packages/pytorch_lightning/plugins/training_type/ddp.py", line 444, in validation_step
return self.model(*args, **kwargs)
File "/usr/local/lib/python3.7/dist-packages/torch/nn/modules/module.py", line 1102, in _call_impl
return forward_call(*input, **kwargs)
File "/usr/local/lib/python3.7/dist-packages/torch/nn/parallel/distributed.py", line 886, in forward
output = self.module(*inputs[0], **kwargs[0])
File "/usr/local/lib/python3.7/dist-packages/torch/nn/modules/module.py", line 1102, in _call_impl
return forward_call(*input, **kwargs)
File "/usr/local/lib/python3.7/dist-packages/pytorch_lightning/overrides/base.py", line 92, in forward
output = self.module.validation_step(*inputs, **kwargs)
File "/usr/local/lib/python3.7/dist-packages/nemo/collections/tts/models/hifigan.py", line 226, in validation_step
audio, audio_len, audio_mel = batch
ValueError: not enough values to unpack (expected 3, got 2)
```

By the way, https://github.com/NVIDIA/NeMo/blob/main/tutorials/tts/FastPitch_Finetuning.ipynb still throw me an error if I use it right out of the box. 

@gedefet, please, give me your command. It looks like you forgot to add this one: model/validation_ds=val_ds_finetune as I added in my command in my preview comment 
@Oktai15 thanks for that. The command and now the error are the following:

```
!python hifigan_finetune.py --config-name=hifigan.yaml model.train_ds.dataloader_params.batch_size=32 model.max_steps=1000 

~model.optim.sched model.optim.lr=0.0001 train_dataset=./hifigan_train_ft.json validation_datasets=./hifigan_val_ft.json 

exp_manager.exp_dir=hifigan_ft +init_from_nemo_model=tts_hifigan.nemo trainer.check_val_every_n_epoch=10 

model/train_ds=train_ds_finetune model/validation_ds=val_ds_finetune

```
```

Error executing job with overrides: ['model.train_ds.dataloader_params.batch_size=32', 'model.max_steps=1000', '~model.optim.sched', 'model.optim.lr=0.0001', 'train_dataset=./hifigan_train_ft.json', 'validation_datasets=./hifigan_val_ft.json', 'exp_manager.exp_dir=hifigan_ft', '+init_from_nemo_model=tts_hifigan.nemo', 'trainer.check_val_every_n_epoch=10', 'model/train_ds=train_ds_finetune', 'model/validation_ds=val_ds_finetune']

Traceback (most recent call last):

  File "/usr/local/lib/python3.7/dist-packages/hydra/_internal/utils.py", line 585, in _locate

    import_module(mod)

  File "/usr/lib/python3.7/importlib/__init__.py", line 127, in import_module

    return _bootstrap._gcd_import(name[level:], package, level)

  File "<frozen importlib._bootstrap>", line 1006, in _gcd_import

  File "<frozen importlib._bootstrap>", line 983, in _find_and_load

  File "<frozen importlib._bootstrap>", line 962, in _find_and_load_unlocked

 

ModuleNotFoundError: No module named 'nemo.collections.tts.torch.data.VocoderDataset'; 'nemo.collections.tts.torch.data' is not a package

```

Thanks,
Use main branch 
I first have to try with 1.6.2. Then, I can use main and it install _nemo-toolkit-1.7.0rc0_. If I use main branch directly, I cannot execute it from scratch. Note that in the last line, I've changed NeMo for NVIDIA, it's the only way I've found to execute it.

```
BRANCH = 'main'
# # If you're using Google Colab and not running locally, uncomment and run this cell.
!apt-get install sox libsndfile1 ffmpeg
!pip install wget unidecode
!python -m pip install git+https://github.com/NVIDIA/NeMo.git@$BRANCH#egg=nemo_toolkit[tts]
```

Almost all examples in https://github.com/NVIDIA/NeMo/tree/main/tutorials/tts cannot be run from scratch (throw an error at some point, including the last FastPitch_MixerTTS_Training.ipynb), I think because the first cell is similar or almost.
Hi @Oktai15 , the way I've found to execute it is the following:

```
BRANCH = 'r1.6.2'
# # If you're using Google Colab and not running locally, uncomment and run this cell.
!apt-get install sox libsndfile1 ffmpeg
!pip install wget unidecode
!python -m pip install git+https://github.com/NVIDIA/NeMo.git@$BRANCH#egg=nemo_toolkit[all]
```
and then:

```
BRANCH = 'main'
# # If you're using Google Colab and not running locally, uncomment and run this cell.
!apt-get install sox libsndfile1 ffmpeg
!pip install wget unidecode
!python -m pip install git+https://github.com/NVIDIA/NeMo.git@$BRANCH#egg=nemo_toolkit[all]
```
Note that I had to change also  `nemo_toolkit[tts]` for `nemo_toolkit[all]`
If I run the second one directly, it throws me a dependencies error. As that cell is almost equal for all colabs in /tutorials/tts, I have to always do the same.

Hope this helps.





Resolving patch:
diff --git a/nemo/collections/tts/models/fastpitch.py b/nemo/collections/tts/models/fastpitch.py
--- a/nemo/collections/tts/models/fastpitch.py
+++ b/nemo/collections/tts/models/fastpitch.py
@@ -197,8 +197,6 @@ def parser(self):
     def parse(self, str_input: str, normalize=True) -> torch.tensor:
         if self.training:
             logging.warning("parse() is meant to be called in eval mode.")
-        if str_input[-1] not in [".", "!", "?"]:
-            str_input = str_input + "."
 
         if normalize and self.text_normalizer_call is not None:
             str_input = self.text_normalizer_call(str_input, **self.text_normalizer_call_kwargs)
diff --git a/nemo/collections/tts/torch/data.py b/nemo/collections/tts/torch/data.py
--- a/nemo/collections/tts/torch/data.py
+++ b/nemo/collections/tts/torch/data.py
@@ -736,7 +736,7 @@ def __init__(
             json. Each line should contain the following:
                 "audio_filepath": <PATH_TO_WAV>,
                 "duration": <Duration of audio clip in seconds> (Optional),
-                "mel_filepath": <PATH_TO_LOG_MEL_PT> (Optional)
+                "mel_filepath": <PATH_TO_LOG_MEL> (Optional, can be in .npy (numpy.save) or .pt (torch.save) format)
             sample_rate (int): The sample rate of the audio. Or the sample rate that we will resample all files to.
             n_segments (int): The length of audio in samples to load. For example, given a sample rate of 16kHz, and
                 n_segments=16000, a random 1 second section of audio from the clip will be loaded. The section will


Test patch: