renwei2024/nemo-github-issues huggingface dataset:
DALI pipeline bug
State: closed | Pull request: no
Labels: name: bug

I use 4 GPU setup and trying to make DALI work as a backend for DataLoader (making slight corrections in existing NeMo code to at least make it without errors till pipeline compilation) but training still fails (fails PTL sanity check)

```
  File "/opt/conda/lib/python3.7/site-packages/torch/nn/modules/module.py", line 722, in _call_impl
    self.padding, self.dilation, self.groups)
**RuntimeError: Expected tensor for argument #1 'input' to have the same device as tensor for argument #2 'weight'; but device 0 does not equal 2 (while checking arguments for cudnn_convolution)**
    result = self.forward(*input, **kwargs)
    result = self.forward(*input, **kwargs)
  File "/opt/conda/lib/python3.7/site-packages/torch/nn/modules/conv.py", line 257, in forward
  File "/opt/conda/lib/python3.7/site-packages/torch/nn/modules/conv.py", line 257, in forward
    self.padding, self.dilation, self.groups)
    self.padding, self.dilation, self.groups)

```

**Expected tensor for argument #1 'input' to have the same device as tensor for argument #2 'weight'; but device 0 does not equal 2 ** is the main part here and it repeats for devices {0,1,3}.

Seems like some of the operations in pipeline are done on cpu, and the others on different gpu or something in the like. Is there a way, that I can work through this problem? 


**Environment details**

Environment location: DGX-1
Torch '1.6.0'
Python 3.76
Ubuntu 18.04.4 LTS"
SGD QA
State: closed | Pull request: yes

SGD QA model
* added ci test
* added documentation

This pull request **introduces 21 alerts** when merging 0af0bc70ad4a47e06067aa02ea4ada5a86168cc2 into 297a8bdbb0370a51cb74a41288025befd9cd2392 - [view on LGTM.com](https://lgtm.com/projects/g/NVIDIA/NeMo/rev/pr-40f8e76f758db09b08253434d7944ca47edbd50f)

**new alerts:**

* 16 for Unused import
* 5 for Unused local variable

This pull request **introduces 21 alerts** when merging 4c9e21e39c492dc583a8b9c7b277d0a83ec01e9d into 297a8bdbb0370a51cb74a41288025befd9cd2392 - [view on LGTM.com](https://lgtm.com/projects/g/NVIDIA/NeMo/rev/pr-c09ce5ec6b80d84c152294588eec12d5dca3d875)

**new alerts:**

* 16 for Unused import
* 5 for Unused local variable

This pull request **introduces 8 alerts** when merging 9dae1ddf19acb06084cbcb1121c03668b76c9908 into 297a8bdbb0370a51cb74a41288025befd9cd2392 - [view on LGTM.com](https://lgtm.com/projects/g/NVIDIA/NeMo/rev/pr-0bff3a176e7685d9e9b349b9cce8e6e60d51c917)

**new alerts:**

* 8 for Unused import

Which paper was implemented here?

@chris-boson  a modification of https://arxiv.org/pdf/2006.09035v1.pdf
EncDecCTCModel can't be pickled due to training_step wrapped with wrapt
State: closed | Pull request: no
Labels: name: bug

**Describe the bug**

EncDecCTCModel can't be pickled due to `training_step` wrappted with [wrapt](https://github.com/GrahamDumpleton/wrapt)

**Steps/Code to reproduce bug**

```
model_names = ["Jasper10x5Dr-En", "QuartzNet15x5NR-En"]
models = [EncDecCTCModel.from_pretrained(model_name=model_name) for model_name in model_names]
```
then try to pickle `models`.

wrapt's FunctionWrapper throws `NotImplementedError: object proxy must define __reduce_ex__()`
see:
https://github.com/GrahamDumpleton/wrapt/blob/develop/src/wrapt/_wrappers.c#L1241
https://github.com/GrahamDumpleton/wrapt/issues/102#issuecomment-456528633

**Expected behavior**

EncDecCTCModel should be pickled and unpickled. 


Currently, quite a few classes in ASR collection cannot be pickled easily. DDP is suggested for any distributed training.
Megatron-lm model parallel gradient clipping
State: closed | Pull request: yes

This PR adds model parallel gradient clipping to `NLPModel`.

This pull request **introduces 3 alerts** when merging 9cc1d68444b748be950f0ae0b4e4c7701320bb71 into 297a8bdbb0370a51cb74a41288025befd9cd2392 - [view on LGTM.com](https://lgtm.com/projects/g/NVIDIA/NeMo/rev/pr-8b83c948558d1688357dbe7f12b05949d53d236d)

**new alerts:**

* 2 for Unused import
* 1 for Except block handles &#39;BaseException&#39;

This pull request **introduces 3 alerts** when merging 531b79ef3d1f704acb529e223241b2fbbe1b1827 into f58ab531376a854e2e94b0fedc05705bc3711086 - [view on LGTM.com](https://lgtm.com/projects/g/NVIDIA/NeMo/rev/pr-17dcebf23511e395c6b1753a47d43d31b05f9635)

**new alerts:**

* 2 for Unused import
* 1 for Except block handles &#39;BaseException&#39;

This pull request **introduces 3 alerts** when merging 5ccfc94ad6fde36dfabebae5d355b0687ef2adbf into 212b1f343aea96bb390a1e00bdcea5b1a9afe1fc - [view on LGTM.com](https://lgtm.com/projects/g/NVIDIA/NeMo/rev/pr-9d071914f4135f4c7fbf73b1dde5dd7054694de2)

**new alerts:**

* 2 for Unused import
* 1 for Except block handles &#39;BaseException&#39;

This pull request **introduces 3 alerts** when merging cc95c613dcb1d2e04e607a82ba898f5c49956db0 into d8150b41bbba0c2ee2e291b309651cef2ed248bf - [view on LGTM.com](https://lgtm.com/projects/g/NVIDIA/NeMo/rev/pr-71feca283519465f203dd8bb933e86050cabce60)

**new alerts:**

* 2 for Unused import
* 1 for Except block handles &#39;BaseException&#39;
RNN-T: Strange type in forward LSTMDropout
State: closed | Pull request: no

https://github.com/NVIDIA/NeMo/blob/297a8bdbb0370a51cb74a41288025befd9cd2392/nemo/collections/common/parts/rnn.py#L196

@titu1994, why does `h` have this type? I suppose it should be `Optional[[Tuple[torch.Tensor, torch.Tensor]]`, just like returnable `h`

Good catch. I was trying our a few things with decoupled lstms and forgot to update signature.
TypeError: stft() got an unexpected keyword argument 'return_complex'
State: closed | Pull request: no
Labels: name: bug

Seems like a problem in a feature extractror module. Training doesn't start, until I comment out line containing that keyword

**Steps/Code to reproduce bug**

start the basic ASR training


 - Environment location: DGX-1
 - Method of NeMo install: pip install -e 


Ubuntu
Torch '1.6.0'
Python 3.76
Ubuntu 18.04.4 LTS"


https://github.com/NVIDIA/NeMo/blob/main/nemo/collections/asr/parts/features.py#L269

if you change to torch>=1.7.0 this should no longer be an issue

yes

I had the issue when using Anaconda Python, but error after switching over to Linux installed python.
[Question] How do you transcribe audio from mic input
State: closed | Pull request: no

**Describe your question**

I want to transcribe audio input from a microphone without saving it as an audio file

**Environment overview (please complete the following information)**

 - Environment location: Local
 - Method of NeMo install: pip install nemo_toolkit[all]==1.0.0b2
 

**Environment details**
OS Version: Ubuntu 18.04
Python version: 3.7
Pytorch Version: 1.7.1
If NVIDIA docker image is used you don't need to specify these.



This is discussed in #1486, there is no built in way to do it but instead you need to write your own code to convert the mic input into a tensor that you pass directly to the model (numpy.frombuffer might be helpful). You then need to pass that tensor as input to your saved nemo model (you wont be able to use the transcribe function but instead will have to decode manually as is done in `validate` in the asr_tutorial). Don't forget to put your model into eval mode (just quartznet.eval() if using quartznet) as the asr_tutorial has a bug and doesn't do this, and it causes training augmentations to be applied during inference.

I'm happy to try to help and answer any more questions if you have them. Good luck. 

Thanks @rbracco for the reply, closing this issue now, I'll try to implement it myself

> This is discussed in #1486, there is no built in way to do it but instead you need to write your own code to convert the mic input into a tensor that you pass directly to the model (numpy.frombuffer might be helpful). You then need to pass that tensor as input to your saved nemo model (you wont be able to use the transcribe function but instead will have to decode manually as is done in `validate` in the asr_tutorial). Don't forget to put your model into eval mode (just quartznet.eval() if using quartznet) as the asr_tutorial has a bug and doesn't do this, and it causes training augmentations to be applied during inference.
> 
> I'm happy to try to help and answer any more questions if you have them. Good luck.

Can you probably elaboate a little bit more and provide with your notebook if already implemented? 

> Thanks @rbracco for the reply, closing this issue now, I'll try to implement it myself

Hi. Have you implemented the conversion audio from mic input into tensor? I need it very much. Can we discuss it? 

Please do not reply to long closed threads, open a new one c

Also, we have not yet implemented audio from mic.
[Question] Question about STFT parameter
State: closed | Pull request: no

Hello, 
I have a question about one parameter used in the config file of quartznet model. Can someone tell me what is the purpose of stft_conv parameter? in the config file it's set to False.
![image](https://user-images.githubusercontent.com/45438779/103083820-aef03980-45dd-11eb-9879-86108b002205.png)
With kind regards.

It's somewhat misleadingly named.
```
stft_conv (bool): If True, uses pytorch_stft and convolutions. If False, uses torch.stft. Defaults to False.
```
[Question] Instantiate ASR model from scratch
State: closed | Pull request: no

Hi all,
I wish to instantiate a brand new ASR Quartznet 15x5 from a .yaml file and train it from scratch.
Actually, what i'm tryin to do is what is shown in 01_ASR_with_Nemo tutorial:

`first_asr_model = nemo_asr.models.EncDecCTCModel(cfg=DictConfig(params['model']), trainer=trainer)`

Even just running the tutorial cell by cell, the following ValueError is returned:

`num_samples should be a positive integer value, but got num_samples=0`

What's wrong? I appreciate your help. Sorry if the question may sound silly.
Thanks

 - Environment location: Google Colab
 - Method of NeMo install: pip install


what is the manifest file you instantiated with?
[Question] Best WAV length for fine-tuning
State: closed | Pull request: no

I'm working on finetuning the ASR to adapt to a specific domain. I am using the CTC segmentation tool ([here](https://github.com/NVIDIA/NeMo/tree/main/tools/ctc_segmentation)) to produce the training data. The wave lengths vary between 2 minutes to 2 seconds. I'm wondering what is the best wave length to use (or range of length to use).

Thanks!

A maximum of 16-20 seconds is recommended for ASR training. Anything more than that will reduce batch size and reduce training efficiency too much
Tacotron2: tokens of pad value confuse decoder at inference
State: closed | Pull request: no

**Describe your question**

When running inference on a batch with a Tacotron2 model, the decoder correctly ends sequences when none of the sequences in the batch have too many tokens with the ```pad``` (which is ```0``` here) value at the end - so inferencing is fine when the batch size is 1 as shown in the examples or the sequences in the batch happen to have very similar length.
However, when there are more than a few such tokens at the end, the decoder does not stop and reaches ```max_decoder_steps``` even for very large values of ```max_decoder_steps```, even on sequences of in-domain length, even for very low values of ```gate_threshold```.

Intuitively, I'd expect the model to have learned that any amount of ```pad``` tokens signal the end, especially since they're preceded by the ```eos``` token. The sequences are not longer than ones encountered during training, so there shouldn't be attention alignment problems, at least not every time as to guarantee this issue.

What could be going wrong? I suspect it may have something to do with either masking of loss during training and/or one of the dropouts being used, but I'm not sure how to tackle this.

Note that this issue is also replicated in the TTS example colab notebook - add as few as 5 zeroes to the end of your parsed input and see what happens. Note that in the example the pad value is equivalent to the token assigned to the whitespace character, which isn't the case in my application, although I'm not sure if that's of any significance given that the issue arises in both cases.

This issue is, in my view, fairly significant because it makes batch inferencing essentially impossible.


Closing. re-open if issue is still relevant.
Error running Speaker Recognition notebook: arguments are located on different GPUs
State: closed | Pull request: no
Labels: name: bug

**Describe the bug**

When running a speaker recognition notebook on multi GPUs machine I get the following error:
RuntimeError: arguments are located on different GPUs at /tmp/pip-req-build-xlj_h8ax/aten/src/THC/generic/THCTensorMathBlas.cu:25

**Steps/Code to reproduce bug**

1) 8-V100 GPUs machine
2) using the latests NeMo docker image 1.0.0b3.
3) run the Speaker_Recognition_Verification.ipynb notebook
4) modified the trainer sets up cell to 
```
  # Let us modify some trainer configs for this demo
  # Checks if we have GPU available and uses it
  cuda = 1 if torch.cuda.is_available() else 0
  config.trainer.gpus = 4

  # Reduces maximum number of epochs to 5 for quick demonstration
  config.trainer.max_epochs = 5

  # Remove distributed training flags
  config.trainer.accelerator = 'dp'
 ```



**Expected behavior**
running the trainer.fit(speaker_model) I get:

```
...

    357         # dot with filterbank energies
--> 358         x = torch.matmul(self.fb.to(x.dtype), x)
    359 
    360         # log features if required

RuntimeError: arguments are located on different GPUs at /tmp/pip-req-build-xlj_h8ax/aten/src/THC/generic/THCTensorMathBlas.cu:25

```
**Environment overview (please complete the following information)**

docker run command used:
```
docker run --gpus all -it --rm  -d -v `pwd`:/NeMo -v /home/nshaposhnikov/datasets:/datasets -v /home/nshaposhnikov/work:/work --shm-size=8g -p 8888:8888 -p 6006:6006 --ulimit memlock=-1 --ulimit stack=67108864 --device=/dev/snd nvcr.io/nvidia/nemo:1.0.0b3
Unable to find image 'nvcr.io/nvidia/nemo:1.0.0b3' locally
1.0.0b3: Pulling from nvidia/nemo
f08d8e2a3ba1: Already exists 
3baa9cb2483b: Already exists 
```


I am also getting the same issue.
https://github.com/NVIDIA/NeMo/issues/1502 this might help.

please use `ddp` instead of `dp` as accelerator. Feel free to open the issue again incase you encounter it again.
Fine-tuning of VAD EncDecClassificationModel fails while computing classification accuracy
State: closed | Pull request: no
Labels: name: bug

**Describe the bug**

I tried to fine-tune EncDecClassificationModel with my labels and encountered `TypeError: only integer tensors of a single element can be converted to an index`. The error came from `NeMo/nemo/collections/common/metrics/classification_accuracy.py`

**Steps/Code to reproduce bug**

```python3
 vad_model = EncDecClassificationModel.from_pretrained(model_name="MatchboxNet-VAD-3x2")
 my_labels = ['a', 'b', 'c']
 vad_model.change_labels(my_labels)
 vad_model.setup_training_data(
            train_data_config={
                'vad_stream': True,
                'sample_rate': 16000,
                'manifest_filepath': my_train_path,
                'labels': my_labels,
                'num_workers': 10,
                'shuffle': True,
                'time_length': 0.67,
                'shift_length': 0.01,
                'trim_silence': False,
            }
        )
 trainer = pl.Trainer(gpus=[0])
 trainer.fit(vad_model)
```

**Environment overview (please complete the following information)**

 - pip install using `reinstall.sh` in the NeMo repository

**Environment details**

- OS version Debian GNU/Linux 10
- PyTorch version 1.7.1
- Python version 3.8.6


This is an issue due to Pytorch 1.7.1 changes regarding how float tensors are cast inside torch.tensor(). There is currently a PR to update NeMo to 1.7.1 here - https://github.com/NVIDIA/NeMo/pull/1579

However, there are other pressing issues before that PR can be merged, and it is unlikely to be merged in a short time. 

In the interim, you could revert to 1.7, use the b3 branch, or you can clone the Nemo main branch repo and apply the one line fix here - https://github.com/NVIDIA/NeMo/pull/1579/files#diff-403adc4a04aa5494044dea63fc59969e52d78c32e6a0ab4518a7c5556c5e6991R89 

@fayejf fyi

Thanks for your quick reply!
I've applied the one line fix and indeed everything looks good.
Tarred dataset support for audio classification models
State: closed | Pull request: yes

- [x] Refactor classification models and label models in audio_to_label 
- [x] Add support for tarred dataset for audio_to_label
- [x] Update classification models
- [x] Update VAD inference (should belong to Speaker Diarization pipelines PR but need to be used here)
- [x] Update training script, config, and instructions to use tarred dataset and improve consistency
- [x] Update converting_to_tarred_dataset script to support label models
- [x] Upload test data
- [x] Add test script
- [x] Ok a new one. enable using the duplicate name for tarred dataloader. (webdataset doesn't allow it) 
       Will try some hacky ways and discuss with people about a cleaner / long-term way after break. 

Updates for label_models will be in another PR 🚢    basically add the tarred option to the configs files.


This pull request **introduces 4 alerts** when merging d616be61a367dbac439f7a42e93a8e2638d23806 into 8c6152787e1eae5c6b23677df1be47041efe2d61 - [view on LGTM.com](https://lgtm.com/projects/g/NVIDIA/NeMo/rev/pr-7804a833bde0dad8ce222e13d982fae64a3c4981)

**new alerts:**

* 4 for Unused import

This pull request **introduces 5 alerts** when merging 00e18b6ff8c903396f82ecfb058f5e7a873ddab6 into c6b90b3dd1321b3e95df8f91b89f55326d52a81c - [view on LGTM.com](https://lgtm.com/projects/g/NVIDIA/NeMo/rev/pr-313d01c6b288c870d3097f6bb5e0cce61a42eb99)

**new alerts:**

* 5 for Unused import
Add labeling your own data section 
State: closed | Pull request: yes


can you please sign your commit? See DCO's bot output on how to sign already committed commits: https://github.com/NVIDIA/NeMo/pull/1575/checks?check_run_id=1569064051 

Thanks!

> can you please sign your commit? See DCO's bot output on how to sign already committed commits: https://github.com/NVIDIA/NeMo/pull/1575/checks?check_run_id=1569064051
> 
> Thanks!

Sorry miss that I use -S instead of -s, updated.

Thanks
Text_Classification_Sentiment_Analysis tutorial is broken
State: closed | Pull request: no
Labels: name: bug

Getting `ModuleNotFoundError: No module named 'transformers.tokenization_bert'` in Text_Classification_Sentiment_Analysis.ipynb notebook:

![image](https://user-images.githubusercontent.com/73861554/102413997-a5961a00-3faa-11eb-8a16-432fccb43eca.png)





Could you please try with `BRANCH='main'` in the first cell of the notebook.

@ekmb Tried `BRANCH='main'` but still getting the same error

I forgot to restart the kernel when trying `BRANCH='main'`. So I restarted the kernel and tried `from nemo.collections import nlp as nemo_nlp` and the original `ModuleNotFoundError: No module named 'transformers.tokenization_bert'` has gone away but there is a different `ModuleNotFoundError: No module named 'webdataset'`: 
![image](https://user-images.githubusercontent.com/73861554/102415438-01fa3900-3fad-11eb-99d9-5a85397dfa81.png)


Could you please restart the kernel and run this one:

BRANCH = 'main'
!python -m pip install git+https://github.com/NVIDIA/NeMo.git@$BRANCH#egg=nemo_toolkit[all]




Tried it and it seems like it's fixed now. Not getting any more errors. There are two APEX warnings though:

![image](https://user-images.githubusercontent.com/73861554/102417301-43401800-3fb0-11eb-98f1-fb15688cbad5.png)



@kshitizgupta21 the warnings are OK (Colab doesn't have APEX).
Readme update
State: closed | Pull request: yes


> can you please replace all references to 1.0.0b2 with 1.0.0b3 in README.md ?

@okuchaiev 1.0.0b3 won't work, the notebooks were not updated from `1.0.0b2` to `1.0.0b3` when 1.0.0b3 was released. for example, this [one with `/blob/v1.0.0b3/`](https://colab.research.google.com/github/NVIDIA/NeMo/blob/v1.0.0b3/tutorials/nlp/Joint_Intent_and_Slot_Classification.ipynb) will install `BRANCH=1.0.0b2`

can you please put paths like https://colab.research.google.com/github/NVIDIA/NeMo/blob/r1.0.0b3/tutorials/nlp/Joint_Intent_and_Slot_Classification.ipynb ?

and we'll update notebooks in r1.0.0b3 to install from r1.0.0b3

> can you please put paths like https://colab.research.google.com/github/NVIDIA/NeMo/blob/r1.0.0b3/tutorials/nlp/Joint_Intent_and_Slot_Classification.ipynb ?
> 
> and we'll update notebooks in r1.0.0b3 to install from r1.0.0b3

But what about bug fixes, like this one https://github.com/NVIDIA/NeMo/pull/1558 ?
Correct ASR issues + Patch for Pytorch 1.8
State: closed | Pull request: yes

# Changelog
- `trim_silence` will now be False by default for all dataloaders in ASR and only active for train_ds (set by flag in config).
- Update ASR stft, ifft ops to be compatible with Pytorch 1.8 deprecation warnings

Note:
- Due to change in `trim_silence`, scores obtained via trainer.test(), or model.transcribe() or any other evaluation process will change slightly. We expect only minor changes to WER due to this bugfix.

Closes https://github.com/NVIDIA/NeMo/issues/1561

@blisc Could you review the TTS side changes to `stft` and `torch.ifft`?

trim_silence default in the model.transcribe method is still True!
This issue can be seen here:
collections/asr/models/ctc_models.py: _setup_transcribe_dataloader
Token_Classification_Named_Entity_Recognition.ipynb specifies specific gpu instead of a single gpu
State: closed | Pull request: no
Labels: name: bug

**Describe the bug**

In the notebook [Token_Classification_Named_Entity_Recognition.ipynb](https://github.com/NVIDIA/NeMo/blob/main/tutorials/nlp/Token_Classification_Named_Entity_Recognition.ipynb), the last cell specifies a specific gpu, which happens to be the second gpu, when it most likely was meant to specify a single gpu.  This causes an error if only one gpu is present on the system.<br>
The error is:
```
MisconfigurationException: 
                You requested GPUs: [1]
                But your machine only has: [0]
```

**Steps/Code to reproduce bug**

Set up a single-GPU box with the v1.0.0b3 version of NeMo running in a container
Upload the [notebook](https://github.com/NVIDIA/NeMo/blob/main/tutorials/nlp/Token_Classification_Named_Entity_Recognition.ipynb)
Run all the cells.

**Expected behavior**

no error when running a single GPU

**Proposed fix for the bug**

In the final cell of the [notebook](https://github.com/NVIDIA/NeMo/blob/main/tutorials/nlp/Token_Classification_Named_Entity_Recognition.ipynb), we see the following:<br>
`trainer = pl.Trainer(gpus=[1], fast_dev_run=fast_dev_run)`<br>
I believe the intent was to run on a only one GPU, but instead we are saying GPU No. 1, which is generally the second GPU in a system where the first GPU is GPU No. 0.  We get an error when there is only one GPU.  To run on a single GPU, the line should be:<br>
`trainer = pl.Trainer(gpus=1, fast_dev_run=fast_dev_run)`<br>

This is explained in the [PyTorch Lighnting gpus documentation](https://pytorch-lightning.readthedocs.io/en/latest/trainer.html#gpus).

**Environment overview (please complete the following information)**

 - Environment location: Docker on AWS V100
 - Method of NeMo install: Docker
 - If method of install is [Docker], provide `docker pull` & `docker run` commands used
    pulled from `nvcr.io/nvidian/nemo/nemo:1.0.0b3`
TalkNet 1.x
State: closed | Pull request: yes

TODOs:
- [x] Pitch predictor (with available models)
- [x] Spect prediction new things (with available models)
- [x] Durs predictor (with available models)
- [x] LJSpeech with durs and f0
- [x] 3x pretrained models (no need)
- [x] Documentation
- [x] Update TTS Notebook inference

Signed-off-by: Stanislav Beliaev <stasbelyaev96@gmail.com>

ping @stasbel  

This pull request **introduces 2 alerts** when merging bb66ad0cb949eab426b3fd510525e4edcf0fdace into fc4017a08db4cf2aaad2288dd158e9c287f7107a - [view on LGTM.com](https://lgtm.com/projects/g/NVIDIA/NeMo/rev/pr-01fe481edaf7d6ac977150b56a2b758a568a4873)

**new alerts:**

* 1 for Unused import
* 1 for Wrong name for an argument in a class instantiation

got some arch mismatches with my pretrained models
going to retrain in following days

This pull request **introduces 2 alerts** when merging bb66ad0cb949eab426b3fd510525e4edcf0fdace into d7ad10435e97024c7c9fdbdd4228e0011dc9e432 - [view on LGTM.com](https://lgtm.com/projects/g/NVIDIA/NeMo/rev/pr-40377bb52a1892ca08cd4ac249299434aa2636b2)

**new alerts:**

* 1 for Unused import
* 1 for Wrong name for an argument in a class instantiation

This pull request **introduces 2 alerts** when merging bb66ad0cb949eab426b3fd510525e4edcf0fdace into db5a787f460d67f0cd89b400cf81826fa2cc9ed6 - [view on LGTM.com](https://lgtm.com/projects/g/NVIDIA/NeMo/rev/pr-6a685435f18b4d1006d383dafd209b308356040a)

**new alerts:**

* 1 for Unused import
* 1 for Wrong name for an argument in a class instantiation

This is not going to RC1. Code freeze was far too long ago

Please top resolving my comments by yourself
Nemo ASR silently trims silence on valid/test set
State: closed | Pull request: no
Labels: name: bug

**Describe the bug**

My assumption would be that `trim_silence` would be False for valid/test unless explicitly turned on. It appears that that is the intention [here in the CTC Models Config](https://github.com/NVIDIA/NeMo/blob/f0378b10cdadc82752d70032bd80219e531519c0/nemo/collections/asr/models/configs/ctc_models_config.py#L66) is to have `trim_silence=True` for training but not validation. Also all the ASR configs for the tutorials have an explicit `trim_silence=True` for train and no entry for validation, implying that the default is False.

I believe the default is set to True [here in audio_to_text_dataset.py](https://github.com/NVIDIA/NeMo/blob/c0fedd91e316a4ba95ac6b14330ff92844c56097/nemo/collections/asr/data/audio_to_text_dataset.py#L46).

**Steps/Code to reproduce bug**

Print out the duration of the audio in the validation loop for some dataset, then set `params["model"]["validation_ds"]["trim_silence"] = False` and try it again and some of the audios will be of different length because they are no longer being trimmed.

**Expected behavior**

Validation silence trimming defaults to False, or the documentation changes to make it more explicit that silence is being trimmed during validation/inference. 

**Environment overview (please complete the following information)**

 -Colab + Nemo 1.0.0b2

Hey @rbracco, thanks for pointing this out! I submitted a PR to patch this bug.

Thank you for all that you do for the repo!
Docker image "v1.0.0b3 not found"
State: closed | Pull request: no
Labels: name: bug

**Steps/Code to reproduce bug**

```bash
docker pull nvcr.io/nvidia/nemo:v1.0.0b3
Error response from daemon: manifest for nvcr.io/nvidia/nemo:v1.0.0b3 not found: manifest unknown: manifest unknown
```

Similar to #1498

Do you mean v1.0.0b2? I haven't seen a b3 version yet, but there is a branch called r1.0.0b3.

Well, I assumed since there is [v1.0.0b3](https://github.com/NVIDIA/NeMo/releases/tag/v1.0.0b3) release, then is should be docker image for that version.

Sorry, I missed the 1.0.0b3 release, you are correct! In the [v1.0.0b3 README](https://github.com/NVIDIA/NeMo/tree/v1.0.0b3) there appear to be updated instructions but it appears your command is correct so the docker container is indeed missing. I was unable to retrieve it as well. 

![image](https://user-images.githubusercontent.com/47190785/102230326-bf583400-3eba-11eb-803f-f30cf7b1bd4f.png)


Try pulling `nvcr.io/nvidia/nemo:1.0.0b3`, it should be live now.
Bio megatron download fix
State: closed | Pull request: yes

1. Branch for Bio-related notebooks changed to `main`, for the rest of the  notebooks to `v1.0.0b3`
2. Fixed Bio-megatron download issue

This pull request **fixes 9 alerts** when merging 4f9ad400d51c3fe596d79169394c70ad936cb80b into d01b7bbc3fdb1bbf14789f71b8f368cf0aa8f86b - [view on LGTM.com](https://lgtm.com/projects/g/NVIDIA/NeMo/rev/pr-80363198ef854198c67fd4639ab1230057dd89b0)

**fixed alerts:**

* 9 for Unused import

This pull request **fixes 9 alerts** when merging 4bcae26d43c55ea4ee017c78054b68f554e13264 into 9da1d27ddd7f1d5d9b6cd75f7308787976680dd8 - [view on LGTM.com](https://lgtm.com/projects/g/NVIDIA/NeMo/rev/pr-baeae19bf39b2e47e8b566507936e07fac2cc74d)

**fixed alerts:**

* 9 for Unused import

This pull request **fixes 9 alerts** when merging 3965be07e3101dea7d42ff0b27a6e9569ce54fe7 into 1c24a5d35a9680b7436f505a8e69dc8f8373f3c2 - [view on LGTM.com](https://lgtm.com/projects/g/NVIDIA/NeMo/rev/pr-7a19d7bc10ec9b44f6aa44d402b8649440267c3c)

**fixed alerts:**

* 9 for Unused import
Tacotron2 bug when batch dimension is 1
State: closed | Pull request: no
Labels: name: bug

**Describe the bug**

Training (or just passing input with ground-truth spectrogram) Tacotron2 with a batch dimension of 1 does not work due to a dimension problem when aggregating the gate outputs.

The line here
https://github.com/NVIDIA/NeMo/blob/d01b7bbc3fdb1bbf14789f71b8f368cf0aa8f86b/nemo/collections/tts/modules/tacotron2.py#L256
should have ```unsqueeze``` instead of ```squeeze``` when the batch dimension is 1.

(I didn't want to submit a pull request as I'm not familiar with the conventions of the project)


Thanks for catching this bug. It should be fixed now
VAD infer script looks broken
State: closed | Pull request: no
Labels: name: bug

I'm trying to perform Voice Activity Detection with NeMo using the script _vad_infer.py_ that is included in the examples. However, the following error comes at first:


```
Traceback (most recent call last):
  File "/home/diego/Sutherland/speechanalytics/speechanalyticsapi/speechanalytics/test.py", line 159, in <module>
    main()
  File "/home/diego/Sutherland/speechanalytics/speechanalyticsapi/speechanalytics/test.py", line 89, in main
    'trim_silence': False,
  File "/home/diego/miniconda3/envs/speechanalytics/lib/python3.6/site-packages/nemo/collections/asr/models/classification_models.py", line 110, in setup_test_data
    self._test_dl = self._setup_dataloader_from_config(config=test_data_config)
  File "/home/diego/miniconda3/envs/speechanalytics/lib/python3.6/site-packages/nemo/collections/asr/models/classification_models.py", line 89, in _setup_dataloader_from_config
    batch_size=config['batch_size'],
KeyError: 'batch_size'
```

I've been able to fix it by including the parameter _batch_size_ in the dict passed to the _vad_model.setup_data_ method. After that, the script runs til the end, but it behaves in an unexpected way as it generates only one prediction value (one float) per audio file, while it should be generating one prediction per frame in every audio file... shouldn't it?

**Environment**

Nemo Toolkit version 1.0.0b2

 

Hi @diego-fustes  Thanks for reporting this. 
You don't need to include the parameter batch_size in the dict passed.  [It's been set here.](https://github.com/NVIDIA/NeMo/blob/9da1d27ddd7f1d5d9b6cd75f7308787976680dd8/nemo/collections/asr/models/classification_models.py#L167) 
There isn't a vad_infer.py in v1.0.0b2. Could you please try main branch and let me know if you still have this issue and share a demo manifests if possible? Thanks! 

Hi @fayejf ,

you are right, I was running the script from the main branch, but my installed NeMo version was v1.0.0b2. Updating to the main branch fixed the issue. Many thanks!

Thanks for letting me know :)
NLP Tutorial notebooks for Biomegatron-bert
State: closed | Pull request: no
Labels: name: bug

**Describe the bug**

Not able to download and initialize the pretrained biomegatron bert model.

config.model.language_model.pretrained_model_name = "biomegatron-bert-345m-cased"
Error Line: nemo_nlp.models.TokenClassificationModel(cfg=config.model, trainer=trainer)

Error: 'OSError: Distant resource does not have an ETag, we won't be able to reliably ensure reproducibility.'

But, I don't get any error if I replace the pre-trained model to a Megatron model ("megatron-bert-345m-cased").
Since the architecture of both models are same I would only assume there is some sort of error in packaging of the biomegatron model.


**Expected behavior**

The notebook to run without any error.

**Environment overview (please complete the following information)**

 - Environment location: Google Colab
 - Tutorial Notebook - https://colab.research.google.com/github/NVIDIA/NeMo/blob/main/tutorials/nlp/Token_Classification-BioMegatron.ipynb
 - Changed transformers version from 4.0.0 to 3.5.0. (I downgraded the version as it seems that the notebook was built on an older version of transformers and I kept getting ModuleNotFoundError because of module name changes in the newer version)

**Environment details**

- Python version = 3.6.9
- Transformers version = 3.5.0


@avikpal00 thank you for reporting the issue. PR #1558 should fix it, no need to change the transformers version.

@ekmb I am still getting this error for the tutorial notebook [Token_Classification-BioMegatron.ipynb](https://colab.research.google.com/github/NVIDIA/NeMo/blob/r1.0.0b3/tutorials/nlp/Token_Classification-BioMegatron.ipynb) when I try to use "biomegatron-bert-345m-cased".

<img width="1086" alt="Screen Shot 2021-01-05 at 7 10 01 PM" src="https://user-images.githubusercontent.com/7240751/103713149-9f7fd180-4f89-11eb-9ec5-b4170ab1773a.png">
[Question] How to setup exp_manager
State: closed | Pull request: no

Question:
    I setup the config.yaml  and run the script. It thorw this message

"The pytorch lightning trainer that was passed to exp_manager contained a logger, and either "
nemo.utils.exp_manager.LoggerMisconfigurationError: The pytorch lightning trainer that was passed to exp_manager contained a logger, and either create_tensorboard_logger: True or create_wandb_logger: False was set to True. These can only be used if trainer does not already have a logger. You can disable lightning's trainer from creating a logger by passing logger=False to its constructor. "

I have disabled the logger in trainer. but it still has this message.  and I saw the code in "nemo/utils/exp_manager.py"
it will throw above following message if trainer is not None . However, I have set the logger to be False in config.yaml . Then, print the 
"logger" in trainer. It shows the object "<pytorch_lightning.loggers.base.LoggerCollection object at 0x7f41b5538550>"

How to disable logger and use exp_manager.py? 

**My yaml configuration (in attached file)** 

**trainer:**
  gpus: 1
  max_epochs: 150
  max_steps: null
  num_nodes: 1
  accelerator: ddp
  accumulate_grad_batches: 1
  checkpoint_callback: **False**
  logger: null
  row_log_interval: 1
  val_check_interval: 0.25

**exp_manager:**
  exp_dir: 
  name: 
  use_datetime_version: **True**
  create_tensorboard_logger: **True**
  create_checkpoint_callback: **True**
  checkpoint_callback_params:
    monitor: "val_wer"
    mode: "min"
  
  create_wandb_logger: **False** 
  wandb_logger_kwargs:
    name: 
    project:


**Environment overview (please complete the following information)**

 - Environment location:  Ubuntu 16.04 + python virtual env (python 3.7)
 - Method of NeMo install:  pip installation [1.0.0.b2]

**Environment details**
python == 3.7.0
torch == 1.7.0


[quartznet_5x3_cer_single.zip](https://github.com/NVIDIA/NeMo/files/5686333/quartznet_5x3_cer_single.zip)



Can you share your training command? As well as the python file that you're using?

Sorry, It is my fault. I found that I was not pass parameter correctly to trainer, so it use default configuration. And solved this problem. I saw the example in "speech_to_text.py" and found I should pass parameters separately not a config dictionary)

pl = trainer(configDict(params['trainer']))   ## original (wrong way)

pl = trainer(gpus=params['trainer']['gpus'], amp_level=params['trainer']['amp_level'], ...)  ## correct way

 Thank you :)

Does the default `trainer = pl.Trainer(**cfg.trainer)` not work for you? What happens if you do `pl = trainer(**params['trainer'])`?

Closing for now as well since it seems to be resolved.

Yes, I test it and the two way you mentioned does work if using hydra decorator on function. Because i need to do some pre-processing for script before training,  I remove the decorator and need to pass all parameter separately.  

Thanks :)
Correcting couple of bugs in example an tutorial
State: closed | Pull request: yes


This pull request **introduces 1 alert** when merging 260e2af8a0d4ea45c2f7ce8cfec9f17fc52a3912 into d01b7bbc3fdb1bbf14789f71b8f368cf0aa8f86b - [view on LGTM.com](https://lgtm.com/projects/g/NVIDIA/NeMo/rev/pr-838abc09fd547a8fab60739d22febc15820ebfe7)

**new alerts:**

* 1 for Unused import

@vladgets  can you please fix CI on this PR?


Correcting it. The problem was in the previous commits exp_manager was removed from testing my model in Jenkins file. And my code was relying for it to save the model.
[Question] Running Nemo with no internet
State: closed | Pull request: no

I'm trying to run nemo with no internet connection - the import statements take quite a long time. I've been digging through the files files to see where, if any, of them are trying to do some kind of `ping` or `curl` on module level, but I haven't found it yet.
If this is the case, do you know where it happens? I can't think of any other reason why it takes so long without an internet connection.

Using the Nemo v1.0.0b2 container.

what is the task or example you were trying to run? Did you first run with internet data and then later without internet? Some examples where we fetch model from NGC needs to be run with the internet, probably your example got stuck there.
Text normalization
State: closed | Pull request: yes

* Added rule-based text normalization for TTS
* added pytests
* added tutorial 

This pull request **introduces 9 alerts** when merging 1f4320a27481466e8223cf8c37375a6f529d7221 into 08d1ccc503e35796e2874b24eca677e9eebad71c - [view on LGTM.com](https://lgtm.com/projects/g/NVIDIA/NeMo/rev/pr-95caec7dea0dda6c9d810101bc940f1b7fcbe7f0)

**new alerts:**

* 3 for Except block handles &#39;BaseException&#39;
* 3 for Unused import
* 2 for Unused local variable
* 1 for Comparison of constants

This pull request **introduces 4 alerts** when merging 8fcb060ad660afba3c0c18f66fadf19b6a591f33 into 08d1ccc503e35796e2874b24eca677e9eebad71c - [view on LGTM.com](https://lgtm.com/projects/g/NVIDIA/NeMo/rev/pr-1caf3d54580f674d744a5163396a7f3dace0978b)

**new alerts:**

* 3 for Except block handles &#39;BaseException&#39;
* 1 for Comparison of constants

This pull request **introduces 4 alerts** when merging a5ac274b377cc86064482e431dd69ee9cb5c4c68 into 08d1ccc503e35796e2874b24eca677e9eebad71c - [view on LGTM.com](https://lgtm.com/projects/g/NVIDIA/NeMo/rev/pr-e358af38a187868e9fee123f45c1a075333c6e29)

**new alerts:**

* 3 for Except block handles &#39;BaseException&#39;
* 1 for Comparison of constants

This pull request **introduces 4 alerts** when merging ba2af6f3b9111367b2513d5e2a84b02eb1db6dd5 into 08d1ccc503e35796e2874b24eca677e9eebad71c - [view on LGTM.com](https://lgtm.com/projects/g/NVIDIA/NeMo/rev/pr-bf9d19eb66ac9a1ea3a08bd19df118bef5be049c)

**new alerts:**

* 3 for Except block handles &#39;BaseException&#39;
* 1 for Comparison of constants

This pull request **introduces 4 alerts** when merging ad0e8cdf9bea1c2db50eb33b1c61bb83a76a36e9 into 9da1d27ddd7f1d5d9b6cd75f7308787976680dd8 - [view on LGTM.com](https://lgtm.com/projects/g/NVIDIA/NeMo/rev/pr-0c0639bdacc1eba624c84b35486129114651b4b3)

**new alerts:**

* 3 for Except block handles &#39;BaseException&#39;
* 1 for Comparison of constants

This pull request **introduces 4 alerts** when merging be92477cd301a13ae6d223a09b490cf2a81508dd into 9aa4ada92b2e6ddd418704dc392e8576d44e5f41 - [view on LGTM.com](https://lgtm.com/projects/g/NVIDIA/NeMo/rev/pr-c6a60db3032019e55dea98b121d785892bd73fb5)

**new alerts:**

* 3 for Except block handles &#39;BaseException&#39;
* 1 for Comparison of constants
Questions about the performance of ASR augmentation
State: closed | Pull request: no

Thank you for sharing the valuable [ASR augmentation code](https://github.com/NVIDIA/NeMo/blob/08d1ccc503e35796e2874b24eca677e9eebad71c/nemo/collections/asr/parts/perturb.py#L596) recently. I would like to ask what is the result of your experiment with these different perturbations. Have you ever done an ablation study on all of the perturbations? Can you share the results with us?



Some details can be found here - https://arxiv.org/abs/2010.12715
--result-dir enabled in TLT
State: closed | Pull request: yes

Modifications enabling TLT text classification train to use ExpManager

- [x] added postfix to ckpt callback structured config + its handling
- [x] minimally polished logic for explicit_log_dir
- [x] minor polish of names
- [x] Bumped versions in requirements: omegaconf, hydra to make Optional[List]] work
- [x] Bumped version of PTL to 1.1 to enable CheckpointConnector overload in TLT
 

This pull request **introduces 1 alert** when merging 340f7482b74ee14f376e15450762888c0536e201 into 16055d86b0be550b87ee5d327e351a1562ce7eb2 - [view on LGTM.com](https://lgtm.com/projects/g/NVIDIA/NeMo/rev/pr-2be8828844b45e4c518c59e6cf4c908a99fc228e)

**new alerts:**

* 1 for Unused import

This pull request **introduces 1 alert** when merging b7e4358cd6bd6e4a2428749fcbf569d393fdfefd into 1c24a5d35a9680b7436f505a8e69dc8f8373f3c2 - [view on LGTM.com](https://lgtm.com/projects/g/NVIDIA/NeMo/rev/pr-29672ebec67da7b0117c1d17394c488f86248496)

**new alerts:**

* 1 for Unused import
Some transformers-based models may not work
State: closed | Pull request: no
Labels: name: bug

**Describe the bug**
With the update of the requirements to transformers >= 4.0.0 on the main branch, calling a model now passes `return_dict=True` by default rather than a tuple previously: https://github.com/huggingface/transformers/releases/tag/v4.0.0

The fix is simple - any transformers model call should explicitly state `return_dict=False` to maintain old functionality.

**Steps/Code to reproduce bug**
I've only looked at the text2sparql model myself - training and evaluating a model will error. I expect some of the other transformer models to do so as well.

**Expected behavior**
Model trains and evaluates properly.

**Environment overview (please complete the following information)**

 - Environment location: Local
 - Method of NeMo install: Installing from main branch:
```
pip install Cython
pip install git+https://github.com/NVIDIA/NeMo.git@main#egg=nemo_toolkit[nlp]
./reinstall.sh
```

**Environment details**
- OS version: Ubuntu 20.04.1
- PyTorch version: 1.8.0.dev20201207+cu110 (I'm using a RTX 3090)
- Python version: 3.8.3
Inference in 01_ASR_with_NeMo fails to put model in eval mode
State: closed | Pull request: no
Labels: name: bug

**Describe the bug**

The inference code in the ASR tutorial 01_ASR_with_NeMo doesn't put the model in training mode before inference. If copied and used in someone's code, this would cause augmentation like SpecAugment to be inadvertantly be applied to the test set resulting in higher WER than the model actually produces using `transcribe`. 

**Expected behavior**

The inference loop should 
1. Store the current mode (as transcribe does) with `mode=first_asr_model.training` 
2. Call `first_asr_model.eval()` before running the pytorch inference loop
3. Reset the mode with `first_asr_model.train(mode=mode)` upon completion



Fixed in main branch, thanks for pointing it out !
Text classification label added.
State: closed | Pull request: yes


@VahidooX to discuss:
1. should we make class_labels_file mandatory? and it so, then:
2. could we get rid of num_classes and get it as len(labels)  when loaded from class_labels_file?

The PR looks good and we can absorb it to TLT->JARVIS the way it is , but I am just thinking loud here...

2nd set of thoughts:

1) in fine-tuning we are effectively changing the dataset, hence should change the "class labels" too. We are using the `setup_*_ds`  for that purpose. Do you think those methods should re-load/overwrite those sections in model?
2) should we leverage that information in inference (`classifytext`) ?

(in other words - should we move that part of "post-processing" from TLT inference to `model.classifytext`?)
Update conformer with bug fixes.
State: closed | Pull request: yes


This pull request **introduces 1 alert** and **fixes 2** when merging 58871c29e0990a1d316eecda9ad54c8a2fc23c10 into 203633d43fc997c2b7d089d94a0e113ebf2e0381 - [view on LGTM.com](https://lgtm.com/projects/g/NVIDIA/NeMo/rev/pr-cb8227a1d7c811a936f38ca9ad1de026d5f9a1b6)

**new alerts:**

* 1 for Unused import

**fixed alerts:**

* 1 for Signature mismatch in overriding method
* 1 for Unused import

> 
> 
> Could you revert the changes related to WER for the time being? That needs further discussion and is not necessary for bug fixes to Conformer model itself. Everything else related to conformer modules looks good to me.

Dropped that part but keep in mind that predictions_len comes from the output of the decoder. So if a model return that, it is used otherwise it is ignored. For some models like encoder/decoder with cross attention it should return None for instance. So it would just used for the models who returns the valid length of their outputs.

> Dropped that part but keep in mind that predictions_len comes from the output of the decoder. So if a model return that, it is used otherwise it is ignored. For some models like encoder/decoder with cross attention it should return None for instance. So it would just used for the models who returns the valid length of their outputs.

All CTC based encoders return this length value, which would affect WER calculations wrt QuartzNet/Jasper as well. In the future, yes we can add this but there needs to be a special flag passed by only certain models - say Attention based models - which then submits the lengths for masking to the WER decoding. This is so that unrelated models such as QN/Jasper are not affected by this masking during wer.

This pull request **fixes 2 alerts** when merging 11007248b28ccfad5e63b1a8e75956a35abbb5c7 into 21a55cdb7a2150114a28b84e9521920ceb56181b - [view on LGTM.com](https://lgtm.com/projects/g/NVIDIA/NeMo/rev/pr-267ca61f8e59ff7c78621434ff4ba2b5c4e7af12)

**fixed alerts:**

* 1 for Signature mismatch in overriding method
* 1 for Unused import

This pull request **fixes 2 alerts** when merging 206fe29d77d91cb3495cee49915e1528a0bea652 into 2ced0eaca3e8a781aee48548914c610c432764e3 - [view on LGTM.com](https://lgtm.com/projects/g/NVIDIA/NeMo/rev/pr-c6b142b51b4a9a190753cdbb46e9c83eea3f0867)

**fixed alerts:**

* 1 for Signature mismatch in overriding method
* 1 for Unused import