renwei2024/nemo-github-issues huggingface dataset:
Adding RNN encoder for LSTM-Transducer and LSTM-CTC models
State: closed | Pull request: yes

# What does this PR do ?

This PR adds RNN-based encoder to NeMo. It enables us to have LSTM-Transducer (RNN-T) and LSTM-CTC models. 

# Changelog 
- Added RNN encoder for ASR models (LSTM-T and LSTM-CTC models)
- Added stacking downsampling
- Added support for proj_size to Transducer decoders
- Added skip_nan_grad support for ASR models which skips the gradients when there is a nan or inf in the gradients

# Usage

```python
# Add a code snippet demonstrating how to use this 
```

**PR Type**:
- [x ] New Feature
- [ ] Bugfix
- [ ] Documentation
Populate the GPT/BERT `list_available_models` with uploaded models
State: closed | Pull request: yes

# What does this PR do?


Populate the GPT/BERT `list_available_models` with uploaded models. So, user can see all the models available by
```python
MegatronBertModel.list_available_models()
```
User can easily load the model by
```python
MegatronBertModel.from_pretrained('megatron_bert_345m_cased', trainer)
```
Dialogue task
State: closed | Pull request: yes

# What does this PR do ?

Add various functionalities to dialogue domain for NeMo

**Collection**: NLP

# Changelog 

1. Support Zero Shot Intent Recognition
2. Further refactored Dialogue module
3. Implement Dialogue GPT Generation Model 
4. Support MS Marco Data Processor
5. Implement Dialogue S2S Generation Model (HF fully supported, Megatron training supported, inference pending integration of common generation API) 
6. Support System Response Generation using user utterance and system slots based on SGD dataset
7. Support Design Data Processor
8. Implement HF BART based classifier into zero shot intent model
9. Implement Dialogue Nearest Neighbour Model
10. Refactor Dialogue SGD Data Processor to make interface with models cleaner
11. Update Nearest Neighbour Model and ZeroShotIntentModel to support SGD dataset and ZeroShot Datasets
12. Support Mellon QA Data Processor
13. Add Documentation and Tutorial

See details in [NVIDIA only dev log](https://docs.google.com/document/d/1IUerv_PLPLgzm5rLEgvHITz_RmK4cKdJKJCV7BXTJ9c/edit?usp=sharing)

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# Before your PR is "Ready for review"
**Pre checks**:
- [x] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [x] Did you write any new necessary tests?
- [x] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [x] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
Fixing order of exported inputs in NLP models
State: closed | Pull request: yes

Signed-off-by: Boris Fomitchev <bfomitchev@nvidia.com>

# What does this PR do ?

Fixing remaining issues with exported order of inputs of NLP models:
* Except Punct&Capital, exported order of inputs of BERT networks was wrong
* There were discrepancies between Bert and Roberta/CamamBert forward() argument orders in nlp/modules/common/huggingface

Also cleaned up unused input_types()/output_types() overloads

**Collection**: [Note which collection this PR will affect]

# Changelog 
- Add specific line by line info of high level changes in this PR.

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
nemo2riva Can't instantiate abstract class ModelPT with abstract methods
State: closed | Pull request: no
Labels: name: bug

**Describe the bug**

I'm trying to convert a CitriNet model I trained with NeMo v1.7.0 to the Riva format using the `nemo2riva` CLI tool.
Everytime I run:

```sh
nemo2riva --out my-model.riva my-model.nemo
```
I get the following error:

```sh
ERROR: Failed to restore model from NeMo file : my-model.nemo. Please make sure you have the latest NeMo package installed with [all] dependencies.
Traceback (most recent call last):
  File "/home/allabana/.virtualenvs/nm-debug/bin/nemo2riva", line 8, in <module>
    sys.exit(nemo2riva())
  File "/home/allabana/.virtualenvs/nm-debug/lib/python3.8/site-packages/nemo2riva/cli/nemo2riva.py", line 50, in nemo2riva
    Nemo2Riva(args)
  File "/home/allabana/.virtualenvs/nm-debug/lib/python3.8/site-packages/nemo2riva/convert.py", line 42, in Nemo2Riva
    raise e
  File "/home/allabana/.virtualenvs/nm-debug/lib/python3.8/site-packages/nemo2riva/convert.py", line 35, in Nemo2Riva
    model = ModelPT.restore_from(restore_path=nemo_in)
  File "/home/allabana/.virtualenvs/nm-debug/lib/python3.8/site-packages/nemo/core/classes/modelPT.py", line 270, in restore_from
    instance = cls._save_restore_connector.restore_from(
  File "/home/allabana/.virtualenvs/nm-debug/lib/python3.8/site-packages/nemo/core/connectors/save_restore_connector.py", line 136, in restore_from
    instance = calling_cls.from_config_dict(config=conf)
  File "/home/allabana/.virtualenvs/nm-debug/lib/python3.8/site-packages/nemo/core/classes/common.py", line 473, in from_config_dict
    instance = cls(cfg=config)
TypeError: Can't instantiate abstract class ModelPT with abstract methods list_available_models, setup_training_data, setup_validation_data
```

I've tried every combination of `nemo2riva` and `nemo_toolkit` from 1.7.0-1.10.0 and 1.3.0-1.7.0 respectively.
I would install using:

```sh
# X = [7, 8, 9, 10]
pip install  nemo2riva-1.X.0b0-py3-none-any.whl
# X = [3, 4, 5, 6, 7]
pip install 'nemo_toolkit[all]==1.X.0'
```

**Steps/Code to reproduce bug**

Use any nemo model, including the pretrained citrinet models and try to convert it

```sh
ngc registry resource download-version "nvidia/riva/riva_quickstart:1.10.0-beta"
cd riva_quickstart_v1.10.0-beta
pip install nemo2riva-1.10.0b0-py3-none-any.whl
nemo2riva --out my-model.riva my-model.nemo
```

**Expected behavior**

`nemo2riva` converts the model succesfully.

**Environment overview (please complete the following information)**

 - Environment location: Docker and Bare metal
 - Method of NeMo install: `pip install 'nemo_toolkit[all]==1.7.0'`
 - If method of install is [Docker], provide `docker pull` & `docker run` commands used
 Using Docker image `nvcr.io/nvidia/riva/riva-speech:1.10.0-beta-servicemaker`

**Environment details**

Docker and local
Otherwise, please provide:
- OS version: Ubuntu 20.04
- PyTorch version: 1.10.0
- Python version: 3.8

**Additional context**

GPU model: RTX 3090
Jenkinsfile test changes
State: closed | Pull request: yes

# What does this PR do ?

Fixing a Jenkins Test error from the previous commit 

**Collection**:  NLP and Jenkinsfile

# Changelog 
- Renamed folder in Jenkinsfile in line 1026

# Before your PR is "Ready for review"
**Pre checks**:
- [x] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [x] Did you write any new necessary tests?
- [x] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
- [x] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [x] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
Conformer model support for Speaker Diarization
State: closed | Pull request: no

I tried offline Diarization with ASR using the following models
1. stt_en_quartznet15x5 
2. stt_en_conformer_ctc_large
3. stt_en_citrinet_1024
But running the sample code for diarization with asr works for the first and third models. The second model conformer ctc large is giving blank output with the following message: 
->Only EncDecCTCModelBPE models trained with per_feature normalization are supported currently. 
I tried with the example notebook of the NeMo repository as well. 
<img width="978" alt="Screenshot 2022-03-23 at 11 37 01 PM" src="https://user-images.githubusercontent.com/10113553/159766811-04288452-46a5-4531-a9a2-f162a49fd21b.png">

[[Ref source link]
](https://github.com/NVIDIA/NeMo/tree/main/examples/speaker_tasks/diarization)
<img width="702" alt="Screenshot 2022-03-23 at 11 33 45 PM" src="https://user-images.githubusercontent.com/10113553/159766263-b5e3ab3b-dcac-4e5f-8204-f71d3577e0b0.png">




Please suggest a solution if I am missing something or is it not supported yet by the NeMo stack yet.

Thanks,
Rafi
Continuous prompt refactor
State: closed | Pull request: yes

# What does this PR do ?

Refactor to combine p-tuning and prompt tuning  code as well as integrate it with the new evaluation code/api. 
Prompt tuning and p-tuning are now jointly called 'prompt learning' within NeMo. 

**Collection**: BigNLP

# Changelog 
- Added MegatronGPTPromptLearningModel class
- Added GPTPromptLearningDataset class, merging p-tuning and prompt-tuning dataset classes
- Removed p-tuning dataset and model classes
- Removed prompt tuning code from the MegatronGPTModel, GPTModel, and TransformerLanguageModel classes
- Edited inference/generation code to accommodate p-tuning/prompt-tuning 

# Usage

```python
python megatron_gpt_prompt_learning.py --config-name=megatron_gpt_prompt_learning_config.yaml
python megatron_gpt_eval.py --config-name=megatron_gpt_inference.yaml
```

# Before your PR is "Ready for review"
**Pre checks**:
- [x] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [x] Did you write any new necessary tests?
- [x] Did you add or update any necessary documentation? (**Will come in a different PR**)
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
- [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [x] New Feature
- [ ] Bugfix
- [x] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.

## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
Jenkinsfile test changes
State: closed | Pull request: yes

# What does this PR do ?

Fixing a Jenkins Test error from the previous commit 

**Collection**:  NLP and Jenkinsfile

# Changelog 
- Renamed folder in Jenkinsfile in line 1014

# Before your PR is "Ready for review"
**Pre checks**:
- [x] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [x] Did you write any new necessary tests?
- [x] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
- [x] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [x] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
Question about volume mounts while using Nemo container in Kubernetes
State: closed | Pull request: no
Labels: name: bug

**Describe the bug**

When using nemo container `nvcr.io/nvidia/nemo:1.4.0` on a kubernetes environment,  there seems to be increase ephemeral storage usage. I had mounted the experiment directory and data dir as volumes for the pod running the training on, was just wondering if there are other directories that are expected to grow for eg , storage for base model and intermediate models (?) , model dependencies like vocabularies etc (?) . Really not sure any help would be appreciated.  

**Steps/Code to reproduce bug**

On a strict k8s cluster which enforces ephemeral storage limits, I was trying to run the token classification task. 


**Expected behavior**

Hopefully if the directory that is exepcted to grow can be converted into a volume. 

**Environment overview (please complete the following information)**

 - Environment location: Bare-metal
 - Method of NeMo install: Running `nvcr.io/nvidia/nemo:1.4.0` as a Pod with 10Gi ephemeral storage
fix bug in intent/slot model reloading
State: closed | Pull request: yes

Signed-off-by: Carol Anderson <carola@nvidia.com>

# What does this PR do ?

Addresses the problem raised in #2769, where models cannot be reloaded if the data directory specified in the config is missing, such as when a model is reloaded on a different machine from the one on which it was trained.

**Collection**: NLP

# Changelog 
- Added a line to the IntentSlotClassificationModel and MultiLabelIntentSlotClassificationModel `__init__` methods to check whether `_is_model_being_restored()` is False before setting it to True. 
- Also fixed minor errors in the Zero Shot Intent Recognition model code: changed the display name of the tutorial notebook and corrected an incorrect type hint in the inference dataset class. 

# Before your PR is "Ready for review"
**Pre checks**:
- [x] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [x] Did you write any new necessary tests?
- [x] Did you add or update any necessary documentation?
- [x] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [x] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [x] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to  #2769
Add async grad allreduce to GPT training with O2 mixed precision
State: closed | Pull request: yes

# What does this PR do ?

When adding pipeline parallelism implementation we had to disable async allreduce in the case of tensor parallelism only. This causes a perf regression when training small models. With this PR we are able to add back async grad allreduce for O2 training.

A subsequent PR will implement async grad allreduce for O1 training.

**Collection**: NLP

# Changelog 
- Update `training_step` with async handler from O2 recipe

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# Before your PR is "Ready for review"
**Pre checks**:
- [x] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [x] Did you write any new necessary tests?
- [x] Did you add or update any necessary documentation?
- [x] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [x] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
Converting FastPitch to TensorRT engine.
State: closed | Pull request: no
Labels: name: TTS

name: stale

I wanted to convert my TTS system with TensorRT, HifiGAN models weren't problematic at all, but I've been unable to get the TensorRT engine from Fastpitch models.

I've exported both the HifiGAN and FastPitch models to ONNX, thanks to _Exportable.export()_ method, but the FastPitch ones fail to even get parsed by the ONNX parser from TensorRT.

That's the error I get:
```
[03/22/2022-09:23:02] [TRT] [E] [shuffleNode.cpp::symbolicExecute::387] Error Code 4: Internal Error (Reshape_1126: IShuffleLayer applied to shape tensor must have 0 or 1 reshape dimensions: dimensions were [-1,2])
[03/22/2022-09:23:02] [TRT] [E] parsers/onnx/ModelImporter.cpp:780: While parsing node number 1137 [Pad -> "1682"]:
[03/22/2022-09:23:02] [TRT] [E] parsers/onnx/ModelImporter.cpp:781: --- Begin node ---
[03/22/2022-09:23:02] [TRT] [E] parsers/onnx/ModelImporter.cpp:782: input: "1655"
input: "1680"
input: "1681"
output: "1682"
name: "Pad_1137"
op_type: "Pad"
attribute {
  name: "mode"
  s: "constant"
  type: STRING
}

[03/22/2022-09:23:02] [TRT] [E] parsers/onnx/ModelImporter.cpp:783: --- End node ---
[03/22/2022-09:23:02] [TRT] [E] parsers/onnx/ModelImporter.cpp:785: ERROR: parsers/onnx/ModelImporter.cpp:179 In function parseGraph:
[6] Invalid Node - Pad_1137
[shuffleNode.cpp::symbolicExecute::387] Error Code 4: Internal Error (Reshape_1126: IShuffleLayer applied to shape tensor must have 0 or 1 reshape dimensions: dimensions were [-1,2])
In node 1137 (parseGraph): INVALID_NODE: Invalid Node - Pad_1137
[shuffleNode.cpp::symbolicExecute::387] Error Code 4: Internal Error (Reshape_1126: IShuffleLayer applied to shape tensor must have 0 or 1 reshape dimensions: dimensions were [-1,2])
```
So it clearly seems like the TensorRT can't handle that particular padding.
I've tried using different ONNX Opset versions (11, 12, 13), tried converting the models from INT64 to INT32 myself, since I suspected the problem might lie in the fact that TensorRT doesn't natively support INT64 and it was attempting to cast the model down to INT32 itself, but nothing seems to work. 

The machine I'm working on has **CUDA 11.4** which I can't change, so I'm using **TensorRT v.8.2.2.1**.


Has anyone been successful in using TensorRT with FastPitch models or are they just not compatible? So far it seems to be a problem specific to FastPitch only.

If someone succeeded, was there a need to change the model - e.g. the paddings? Or if it worked out of the box, what's NeMo/ONNX opset/TensorRT versions combination to use?
enable bias+dropout+add fusion with nvfuser at inference
State: closed | Pull request: yes

# What does this PR do ?
Make nvfuser work at bias+dropout+add fusion at GPT inference

# Changelog 
Apply _torch.enable_grad_ to the fusion at training pass

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [x] Bugfix
- [ ] Documentation
Make index mappings dir configurable
State: closed | Pull request: yes

# What does this PR do ?

Index mapping files are created during nemo megatron training. This PR makes the directory that they are saved in configurable.

**Collection**: NLP

# Changelog 
- Add specific line by line info of high level changes in this PR.

# Usage
* You can potentially add a usage example below

```bash
# add to your model config the directory to save the index mappings
model.data.index_mapping_dir=/raid/data/gpt_pile/index_mappings
```

# Before your PR is "Ready for review"
**Pre checks**:
- [x] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [x] Did you write any new necessary tests?
- [x] Did you add or update any necessary documentation?
- [x] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [x] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
Update webdataset version
State: closed | Pull request: yes

# What does this PR do ?

Attempts to update webdataset version to use better seeding that was introduced in more recent versions.

**Collection**: NLP

# Changelog 

- Change webdataset version requirements file
- Change NMT tarredataset to use webdataset pipeline

# Usage
N/A

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
I'm getting syntax error  trying loading nlp (webdataset  import)
State: closed | Pull request: no

from nemo.collections import nlp as nemo_nlp gives me an error 


```
File "/usr/local/lib/python3.7/dist-packages/webdataset/cache.py", line 43
    while data := stream.read(chunk_size):
                ^
SyntaxError: invalid syntax
```

that looks to originate in webdataset  import. 
The webdataset version 0.2.4
and the last version of Nemo. 

Anyone can help?
[Question] Loading trainer configs for resume training of ASR models
State: closed | Pull request: no

Hi,
I know that when one want to fine-tune an ASR model can easily load pre-trained models' wights by using [this link hints and commands](https://docs.nvidia.com/deeplearning/nemo/user-guide/docs/en/main/asr/configs.html#fine-tuning-configurations). Now, my questions is that, how I can load the trainer configs for **resume training**? Assume, I had a long training and for some reasons, the running server shut down before the training procedure has completed. So, I want to load the last checkpoint as well as the trainer last configs (_i.e._, last learning rate value, optimizer state, etc.).
Can you please guide me how I can do this (loading the last state of trainer configs)?
Best
Neural Machine Translation with Megatron Transformer Models (Tensor Parallel and Tarred Datasets Only)
State: closed | Pull request: yes

# What does this PR do ?

Adds support to train Neural Machine Translation models with Megatron Transformer models that support tensor parallelism.

**Collection**: NLP

# Changelog 
- Add a `MegatronNMT` class that inherits from `MegatronLMEncoderDecoderModel` and overrides appropriate functions and calls classmethods from `MTEncDecModel`
- Change appropriate methods in `MTEncDecModel` to classmethods so they can be called in `MegatronNMTModel`.
- Add a config file and training script.

# Usage

```python
examples/nlp/machine_translation/megatron_nmt_training.py -cn aayn_base_megatron model.tensor_model_parallel_size=2
```

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
radtts to Nemo porting
State: closed | Pull request: yes

# What does this PR do ?

It's ports ADLR implementation of RADTTS to NeMo.

It affects TTS collection.  Specifically, TTS/Torch.

# Changelog 
- Added modules, model, loss functions, and helper functions for RADTTS

# Usage
Their is two stage training which requires two config files that are found under /examples/tts/conf/
- for each stage of the training different learning rates are used 
      - first stage learning rate (le4, 1e4, 5e5, 1e6(with fp= 32)) , config file: rad-tts_dec
      - second stage learning rate (1e3, 1e4, 1e5), config file: rad-tts_feature_pred


```python
# Add a code snippet demonstrating how to use this 
Once the config files are filled out. Run examples/tts/radtts_train.py

# Before your PR is "Ready for review"
**Pre checks**:
- [ x] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ x] Did you write any new necessary tests?
- [ x] Did you add or update any necessary documentation?
- [ x] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ x] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ x] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.

- onnx exportability is still not added
- not tested with multi-speaker TTS training  
## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
Adding P_voiced and voiced_mask to data.py will conflict with the othre TTS repos. Let's discuss about them.
WIP: Megatron support
State: closed | Pull request: yes

# What does this PR do ?

Add a one line overview of what this PR aims to accomplish.

**Collection**: [Note which collection this PR will affect]

# Changelog 
- Add specific line by line info of high level changes in this PR.

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
RADTTS-NeMo Porting 
State: closed | Pull request: yes

# What does this PR do ?

It's ports ADLR implementation of RADTTS to NeMo.

It affects TTS collection.  Specifically, TTS/Torch.

# Changelog 
- Added modules, model, loss functions, and helper functions for RADTTS

# Usage
Their is two stage training which requires two config files that are found under /examples/tts/conf/
- for each stage of the training different learning rates are used 
      - first stage learning rate (le4, 1e4, 5e5, 1e6(with fp= 32)) , config file: rad-tts_dec
      - second stage learning rate (1e3, 1e4, 1e5), config file: rad-tts_feature_pred


```python
# Add a code snippet demonstrating how to use this 
Once the config files are filled out. Run examples/tts/radtts_train.py

# Before your PR is "Ready for review"
**Pre checks**:
- [ x] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ x] Did you write any new necessary tests?
- [ x] Did you add or update any necessary documentation?
- [ x] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ x] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ x] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.

- onnx exportability is still not added
- not tested with multi-speaker TTS training  
## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
Adding P_voiced and voiced_mask to data.py will conflict with the othre TTS repos. Let's discuss about them.
Added Hindi and Marathi Models in Nemo pretrained ASR_CTC_BPE models …
State: closed | Pull request: yes

Signed-off-by: Megh Makwana [mmakwana@nvidia.com](mailto:mmakwana@nvidia.com)
# What does this PR do ?
Add new pretrained Hindi and Marathi ASR models (Conformer CTC Medium) to list_available_models and to documentation

Collection: [Note which collection this PR will affect]
ASR

# Changelog 
Add new pretrained Hindi and Marathi ASR models (Conformer CTC Medium) to the method list_available_models in EncDecCTCModelBPE 


# Usage

Import pretrained Hindi and Marathi Conformer-CTC-Medium Models
```python
asr_model = nemo_asr.models.EncDecCTCModelBPE.from_pretrained(model_name="stt_hi_conformer_ctc_medium")
asr_model = nemo_asr.models.EncDecCTCModelBPE.from_pretrained(model_name="stt_mr_conformer_ctc_medium")  
```

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
Wrong number of optimisation steps with tarred dataset
State: closed | Pull request: no
Labels: name: bug

I noticed that when training a `TransformerLMModel` using a `TarredSentenceDataset` the number of optimisation steps is incorrect in the following case:
* Training for a maximum number of epochs (not maximum number of steps)
* Using more than 1 GPU worker
* Using `shard_strategy: scatter`

In the above case, the dataset length should be `self.metadata['num_batches'] / num_workers` below.

https://github.com/NVIDIA/NeMo/blob/fe30166a3122aefe651052e2bbdd950a842e239b/nemo/collections/nlp/data/language_modeling/sentence_dataset.py#L273

If the dataset length is not divided by the number of workers (as is now), PyTorch Lightning has a too large number of steps/epoch and no validation is performed at the end of one epoch. (PTL does not divide the total dataset length by the number of workers for `IterableDataset`s).

However, when the dataset length is calculated as `self.metadata['num_batches'] / num_workers` then there is a problem when calculating the total number of optimisation steps. When the number of optimisation steps is calculated, the dataset length is divided by the number of workers, resulting in a too low number of optimisation steps in this case.

https://github.com/NVIDIA/NeMo/blob/fe30166a3122aefe651052e2bbdd950a842e239b/nemo/core/optim/lr_scheduler.py#L857

To calculate the correct total number of optimisation steps, it should not be divided by the number of workers when we have a dataset of type `IterableDataset`, use `shard_strategy: scatter`, more than one GPU worker, and train for a maximum number of epochs.

I noticed this problem only with `TarredSentenceDataset` but I think it is also the same for other `Tarred*Dataset`s.
RADTTS-NeMo Porting
State: closed | Pull request: yes

# What does this PR do ?

It's ports ADLR implementation of RADTTS to NeMo.

It affects TTS collection.  Specifically, TTS/Torch.

# Changelog 
- Added modules, model, loss functions, and helper functions for RADTTS

# Usage
Their is two stage training which requires two config files that are found under /examples/tts/conf/
- for each stage of the training different learning rates are used 
      - first stage learning rate (le4, 1e4, 5e5, 1e6(with fp= 32)) , config file: rad-tts_dec
      - second stage learning rate (1e3, 1e4, 1e5), config file: rad-tts_feature_pred


```python
# Add a code snippet demonstrating how to use this 
Once the config files are filled out. Run examples/tts/radtts_train.py

# Before your PR is "Ready for review"
**Pre checks**:
- [ x] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ x] Did you write any new necessary tests?
- [ x] Did you add or update any necessary documentation?
- [ x] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ x] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ x] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.

- onnx exportability is still not added
- not tested with multi-speaker TTS training  
## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
Adding P_voiced and Voiced_mask might create conflicts with the other TTS repos so let's discuss about it.
RADTTS-NeMo Porting 
State: closed | Pull request: yes

# What does this PR do ?

It's ports ADLR implementation of RADTTS to NeMo.

It affects TTS collection.  Specifically, TTS/Torch.

# Changelog 
- Added modules, model, loss functions, and helper functions for RADTTS

# Usage
Their are two stages of training which requires two config files that are found under /examples/tts/conf/
- for each stage of the training different learning rates are used 
      - first stage learning rate (le4, 1e4, 5e5, 1e6(with fp= 32)) , config file: rad-tts_dec
      - second stage learning rate (1e3, 1e4, 1e5), config file: rad-tts_feature_pred


```python
# Add a code snippet demonstrating how to use this 
Once the config files are filled out. Run examples/tts/radtts_train.py

# Before your PR is "Ready for review"
**Pre checks**:
- [ x] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ x] Did you write any new necessary tests?
- [ x] Did you add or update any necessary documentation?
- [ x] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ x] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ x] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.

- onnx exportability is still not added
- not tested with multi-speaker TTS training  
## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
SyntaxError with Colab Example
State: closed | Pull request: no
Labels: name: bug

I am running the following ipynp on my Colab:
https://github.com/NVIDIA/NeMo/blob/v1.0.2/tutorials/asr/01_ASR_with_NeMo.ipynb 

I only replaced the data set to speech commands from "an4" as it seems the link for an4 is not working. 

In the section "Using NeMo for Automatic Speech Recognition", the very first cell where I import nemo:

NeMo's "core" package
import nemo
NeMo's ASR collection - this collections contains complete ASR models and
building blocks (modules) for ASR
import nemo.collections.asr as nemo_asr

I get the following output:
[NeMo W 2022-03-17 16:03:28 optimizers:46] Apex was not found. Using the lamb optimizer will error out.

  File "/usr/local/lib/python3.7/dist-packages/webdataset/cache.py", line 43
    while data := stream.read(chunk_size):
                ^
SyntaxError: invalid syntax

Could anyone please point at a fix for this?
yuya/megatron t5 xnli eval
State: closed | Pull request: yes

# What does this PR do ?
Add megatron t5 xnli eval only script - it takes in finetuned checkpoint and output test results on test dataset.

**Collection**: [Note which collection this PR will affect]

# Changelog 
- Add specific line by line info of high level changes in this PR.

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [x] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
Ranking refactor e
State: closed | Pull request: yes

# What does this PR do ?

Add a one line overview of what this PR aims to accomplish.

**Collection**: [Note which collection this PR will affect]

# Changelog 
- Add specific line by line info of high level changes in this PR.

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
Disable nvfuser for gpt
State: closed | Pull request: yes

# What does this PR do ?

Disables NVFUSER and gets global batch size from config in build dataset

**Collection**: NLP

# Changelog 
- Comment out call that enables nvfuser in ngc container
- Get global batch size from config

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# Before your PR is "Ready for review"
**Pre checks**:
- [x] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [x] Did you write any new necessary tests?
- [x] Did you add or update any necessary documentation?
- [x] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [x] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
Change container
State: closed | Pull request: yes

Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca>

# What does this PR do ?

Changes the container for NeMo CI testing to support T5 PP CI tests. It uses the Pytorch 22.02 container with Apex built from a particular commit.

**Collection**: all

# Changelog 
N/A

# Usage
N/A

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
Notes in tutorial notebooks on deploying with Riva
State: closed | Pull request: yes

# What does this PR do ?

Added quick notes to the ends of several of the tutorial notebooks (asr, nlp, and tts) on deploying with Riva.

**Collection**: [Note which collection this PR will affect]

# Changelog 
- Add specific line by line info of high level changes in this PR.

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
Adding a RegEx tokenizers
State: closed | Pull request: yes

# What does this PR do ?

This PR adds a regex tokenizer (currently required by MegaMolBART).

**Collection**: `nemo.collections.common.tokenizers`

# Changelog 
- `nemo/collections/common/tokenizers/regex_tokenizer.py`
- `tests/collections/nlp/test_regex_tokenizer.py`

# Usage

In order to build a regexp tokenizer model use the following command.
The script will create:
* `.vocab` file - with learned vocabulary
* `.model` file - with provided regex

To build vocabulary from text files:
```
python -- scripts/nlp_language_modeling/build_regex_tokenizer.py \
  --regex '\[[^\]]+]|Br?|Cl?|N|O|S|P|F|I|b|c|n|o|s|p|\(|\)|\.|=|#|-|\+|\\\\|\/|:|~|@|\?|>|\*|\$|\%[0-9]{2}|[0-9]' \
  --input_type text \
  --output_file regex_tokenizer -- \
  data_file1.txt data_file2.txt
```

To build vocabulary from CSV files ("smiles" column):
```
python -- scripts/nlp_language_modeling/build_regex_tokenizer.py \
  --regex '\[[^\]]+]|Br?|Cl?|N|O|S|P|F|I|b|c|n|o|s|p|\(|\)|\.|=|#|-|\+|\\\\|\/|:|~|@|\?|>|\*|\$|\%[0-9]{2}|[0-9]' \
  --input_type csv \
  --input_csv_col smiles \
  --output_file regex_tokenizer -- \
  data_file1.csv data_file2.csv
```