renwei2024/nemo-github-issues huggingface dataset:
Raise error if bicleaner is not installed in NMT Data preprocesing notebook
State: closed | Pull request: yes

Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca>

# What does this PR do ?

Fixes NMT preproc notebook.

**Collection**: [Note which collection this PR will affect]

# Changelog 
- Raise error if bicleaner is not installed.

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
Prompt learning notebook bugfix
State: closed | Pull request: yes

# What does this PR do ?
Fixes notebook typo, changes branch to r1.9.0, improves manual dataset download message

**Collection**: BigNLP

# Changelog 
- Prompt Learning Notebook

# Usage
N/A

# Before your PR is "Ready for review"
**Pre checks**:
- [x] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
- [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [x] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to https://nvbugs/3639143 [ML-NeMo][A30] Multitask_Prompt_and_PTuning.ipynb branch name not 'r1.9.0' and failed with ImportError: cannot import name 'VirtualPromptStyle' as shown below if manually changed to the r1.9.0 branch
Update YAML with trainer.benchmark=False for NLP
State: closed | Pull request: yes

Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca>

# What does this PR do ?

Sets `trainer.benchmark=False` for all NLP yaml configs.

**Collection**: NLP

# Changelog 
Sets `trainer.benchmark=False` for all NLP yaml configs.


# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
Fixing the speed slow-down for speech models.
State: closed | Pull request: yes

Signed-off-by: vahidoox <vnoroozi@nvidia.com>

# What does this PR do ?

There was a slow down in speech models when upgraded to PTL 1.6. They changed the default of trainer.benchmark to True which slows down model with variable input lengths including speech models. This PR set all the trainer.benchmark to false for all speech models.

# Changelog 
Fixed the speed slow-down for speech models by setting benchmark=false for all speech configs.

**PR Type**:
- [ ] New Feature
- [x] Bugfix
- [ ] Documentation
updating PretrainedModelInfo and benchmark sheet for ASR models
State: closed | Pull request: yes

Signed-off-by: Krishna Puvvada <kpuvvada@nvidia.com>

# What does this PR do ?

Updating PretrainedModelInfo and benchmark sheet for ASR models

**Collection**: ASR

# Changelog 
- Added conformer RNNT xlarge, xxlarge info in PretrainedModelInfo, benchmark sheet
- Added conformer CTC xlarge info in PretrainedModelinfo, benchmark sheet


# Before your PR is "Ready for review"
**Pre checks**:
- [x] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [x] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [ ] Bugfix
- [x] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
fix broken link in the tutorial
State: closed | Pull request: yes

# What does this PR do ?

Fix broken link

**Collection**: [Note which collection this PR will affect]

# Changelog 
- Add specific line by line info of high level changes in this PR.

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
add 'relative positional embedding (RPE)' feature - re-creating after…
State: closed | Pull request: yes

… re-basing to 'main' branch from r1.9.0 - all previous review comments are in.

Signed-off-by: Hoo Chang Shin <hshin@nvidia.com>

# What does this PR do ?

Add 'relative positional embedding (RPE)' feature to T5 model.

**Collection**: NLP

# Changelog 
Adds an option to use 'relative positional embedding' other than 'absolute position embedding' or 'fixed position encoding'.

# Usage

In the 'megatron_t5_config.yaml' file change the 'position_embedding_type' option to 'relative'.

```python
python megatron_t5_pretraining.py
```

# Before your PR is "Ready for review"
**Pre checks**:
- [x] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [x] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [x] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
Support load_adapters with just adapter_name
State: closed | Pull request: yes

Signed-off-by: smajumdar <smajumdar@nvidia.com>

# What does this PR do ?

Adds support for load_adapters to load any module adapter with just the adapter_name

**Collection**: [Core, ASR]

# Changelog 
- Update code and tests

# Usage
```python
model = ModelWithAdapterSupport()
model.add_adapters("module:adapter_name", adapter_cfg)
model.save_adapters("adapters.pt")

new_model = ModelWithAdaptersSupport()
new_model.load_adapters("adapters.pt", name="adapter_name")
```

**PR Type**:
- [x] New Feature
- [ ] Bugfix
- [ ] Documentation
Add 'relational positional embedding (RPE)' feature to T5
State: closed | Pull request: yes

# What does this PR do ?

Add 'relative positional embedding (RPE)' feature to T5 model.

**Collection**: NLP

# Changelog 
Adds an option to use 'relative positional embedding' other than 'absolute position embedding' or 'fixed position encoding'.

# Usage

In the 'megatron_t5_config.yaml' file change the 'position_embedding_type' option to 'relative'.

```python
python megatron_t5_pretraining.py
```

# Before your PR is "Ready for review"
**Pre checks**:
- [x] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [x] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
Fastpitch TTS in other languages
State: closed | Pull request: no
Labels: name: feature

Hello!

I would like to train some models with fastpitch.

One model in Spanish and other model in Catalan.

Each language has its own alphabet, symbols, punctuations, tildes, phonemes, etc

1. How to define and set all of this properties to get started?

2. If there are other properties that need to be defined, which are?

Thank you!

David Martin Rius
Not sure how to train TTS model
State: closed | Pull request: no
Labels: name: feature

Hello!

In this post I would like to ask some questions.
As a premise I want to tell you what kind of machine I am working.

I have got an MSI Creator P100X -637
The specs are 
GPU RTX 3080 
64GB RAM
Intel Core i7-11700K
CPU 3.6 GHz

I work with trial an error and it is very confusing. Depending on the size of the dataset the GPU only accepts a limited max number in batch size.

For example, if I am going to train a dataset of 20 hours it will only accept a max batch size of 6. If I try a greater number it always raises an CUDA out of memory. (I supose it is the most common error when training)

On the other hand, if I am going to train a dataset of 5 hours aprox I can work with a batch size of 14 or 15.

### **Question 1/4.**
Is there a way to estimate the batch size previous to train?

Each time that I start training, if there is too much batch size it will crash. From I start training to crash it can pass like 15-20 minutes. This is because before start training the program needs to load all data previously. So  i wasted a lot of time because until I don't find the good max batch size I can spend hours.

### **Question 2/4.**
Is there a way to estimate how many epochs to train a model based on the size of the dataset and the GPU batch size?

Now I set like 1000 epochs and I let the model training until the training loss is near to 0. But I am sure there is a cleaner way to do it.

### **Question 3/4.**
How to know how much time to train a model?
I supervised the trainings in tensorboard.
I deduced that I should evaluate the training loss of the model to know if is enough trained. But I am not sure of this.

### **Question 4/4.**
How to evaluate when a model is enough trained?

I spend a lots of hours trying to train a model without success. I am sure a lot of people that start with NeMo has this kind of doubts. Also  I read a lot of discussions, issues, etc but I still did not solve this doubts.

So I would apreciate some help. I really want to train a TTS model with my own datasets.

Thank you!
Refactored path to manifest
State: closed | Pull request: yes

Signed-off-by: Evgeniy Shabalin <baah1999@yandex.ru>

# What does this PR do ?

Updates various files with the new path for the LJSpeech whitelist file

**Collection**: TTS
  
**PR Type**:
- [ ] New Feature
- [x] Bugfix
- [ ] Documentation
fixing bug 3642622
State: closed | Pull request: yes

Signed-off-by: Ghasem Pasandi <gpasandi@nvidia.com>

# What does this PR do ?

Add a one line overview of what this PR aims to accomplish.

**Collection**: [Note which collection this PR will affect]

# Changelog 
- Add specific line by line info of high level changes in this PR.

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [x] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
Fixing bug #3642622
State: closed | Pull request: yes

# What does this PR do ?

Add a one line overview of what this PR aims to accomplish.

**Collection**: [Note which collection this PR will affect]

# Changelog 
- Add specific line by line info of high level changes in this PR.

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [x] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
Fixes for Megatron Base Model Artifacts
State: closed | Pull request: yes

# What does this PR do ?

Fixes defaults for Megatron Base Model Artifacts.

**Collection**: NLP

# Changelog 
- Changes defaults in megatron base model artifacts.

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
Replace conditional imports in WFST based TN/ITN by importlib
State: closed | Pull request: yes

# What does this PR do ?
Replace conditional imports in WFST based TN/ITN by importlib

Add support for 
**Collection**: [Note which collection this PR will affect]

# Changelog 
- Replace conditional imports in WFST based TN/ITN by importlib.
- Fix deleted support for lm and audio based text normalizer
Replace conditional imports in WFST based TN/ITN by importlib
State: closed | Pull request: yes

Signed-off-by: Anand Joseph <anajoseph@nvidia.com>

# What does this PR do ?

Replaces conditional imports in WFST based TN/ITN by importlib

**Collection**: [Note which collection this PR will affect]

# Changelog 
- Replace conditional imports in normalize.py and inverse_normalize.py 


**PR Type**:
- Code enhancement

## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
How to reduce the vocabulary for a QuartzNet model?
State: closed | Pull request: no
Labels: name: feature

For a simple script like this:
```python

#!/usr/bin/env python
import nemo.collections.asr as nemo_asr
import ctc_decoders

import numpy as np
import librosa

import os, sys, pdb

## Instantiate model
DetailsDict = nemo_asr.models.EncDecCTCModel.list_available_models()
asr_model = nemo_asr.models.EncDecCTCModel.from_pretrained(model_name='QuartzNet15x5Base-En', strict=False)

## Test audio clip
AUDIO_FILENAME = 'test.wav'
signal, sample_rate = librosa.load(AUDIO_FILENAME, sr=None)

## Convert our audio sample to text
files = [AUDIO_FILENAME]
transcript = asr_model.transcribe(paths2audio_files=files)[0]; print(f'Transcript: "{transcript}"')
```

It performs well
However if i change the problem of just predicting fixed number of words, for example `yes, no`
How should i augment the language model part of it ?
Replace conditional imports in WFST based TN/ITN with importlib
State: closed | Pull request: yes

# What does this PR do ?
Replace conditional imports in WFST based TN/ITN with importlib

# Changelog 
- Replace conditional imports in WFST based ITN with importlib in inverse_normalize.py .
- Replace conditional imports in WFST based TN with importlib in normalize.py .

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
Replaces conditional imports in WFST TN/ITN with importlib
State: closed | Pull request: yes

# What does this PR do ?

Replaces conditional imports in WFST TN/ITN with importlib

# Changelog 
- Replaces conditional imports in WFST TN/ITN with importlib
- Support for en_in number system in ITN that supports both international as well as Indian (subcontinent) number system

# Usage
```python
from nemo_text_processing.inverse_text_normalization.inverse_normalize import InverseNormalizer
inverse_normalizer = InverseNormalizer(lang='en_in', overwrite_cache=True)
inverse_normalizer.inverse_normalize('twelve crores twenty five lakhs thirty nine thousand six hundred and eighty four', verbose=True)
```

  
**PR Type**:
- [*] New Feature
Use importlib for TN/ITN languages
State: closed | Pull request: yes

# What does this PR do ?

Replaces the conditional imports in TN/ITN with importlib. 

**Collection**: [Note which collection this PR will affect]

# Changelog 
- Added staticmethods in inverse_normalize.py and normalize.py to check if tagger/classifier exists for that language
- Delete all the language conditional imports
- Use importlib to load the taggers and classifiers for the specified language.

# Usage
* no change in usage
Fixing bug  3642622 
State: closed | Pull request: yes

# What does this PR do ?

Add a one line overview of what this PR aims to accomplish.

**Collection**: [Note which collection this PR will affect]

# Changelog 
- Add specific line by line info of high level changes in this PR.

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [x] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
Prompt Learning Typo Fixes
State: closed | Pull request: yes

# What does this PR do ?
Fixes Typos in Prompt Learning notebook and documentation 

**Collection**: BigNLP

# Changelog 
- Prompt Learning Notebook
- Prompt Learning Docs

# Usage
N/A

# Before your PR is "Ready for review"
**Pre checks**:
- [x] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [x] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
- [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [ ] Bugfix
- [x] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue) https://nvbugs/3642570 [ML-NeMo][A30] Multitask_Prompt_and_PTuning.ipynb contains typos
Save Virtual Prompt Weights Only
State: closed | Pull request: yes

Removes all GPT/Frozen model configs and weights from Prompt Learning Model's .nemo file. After training has ended, the prompt learning model now only saves prompt table parameters. During training, intermediate checkpoint files may also have prompt encoder parameters. 

**Collection**: BigNLP
- Adds custom state_dict and load_state_dict methods for prompt learning model class
- Updates prompt learning inference example script
- Updates prompt learning documentation

# Usage
The final .nemo checkpoint file contains

1. model_config.yaml
2. model_weights.ckpt

where model_weights.ckpt only has prompt table params

# Before your PR is "Ready for review"
**Pre checks**:
- [x] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [x] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
- [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [x] New Feature
- [x] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
GPT config options
State: closed | Pull request: yes

# What does this PR do ?

Adds missing configs to GPT pre-training.

**Collection**: NLP

# Changelog 
- Upates the GPT pretraining yaml config and propages the args through to ParallelTransformer.

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
Speaker Diarization: Known Number of Speakers
State: closed | Pull request: no

Hello NeMo team

I evaluated NeMo clustering diarizer on VoxConverse dataset. And the results are wonderful. But when I am providing the actual number of speakers through manifest, results get worse. I noticed the same behavior for NIST SRE 2000 dataset too. I am wondering why is this happening? Is it something related to dataset (like overlap speech)?


<img width="873" alt="Screen Shot 2022-05-23 at 23 07 41" src="https://user-images.githubusercontent.com/47416614/169889294-ff7b4100-5f9c-4b58-b695-06a686c34c52.png">
Update TTS Configs from LAMB to AdamW
State: closed | Pull request: yes

Signed-off-by: Jocelyn Huang <jocelynh@nvidia.com>

# What does this PR do ?

Replaces LAMB optimizer (unstable, hangs during multi-GPU training) with AdamW for FastPitch and Mixer-TTS configs.
Second attempt based off of `main` instead of `r1.9.0` (see #4219)

**Collection**: TTS

# Changelog 
- Replace LAMB defaults with AdamW
- Update `fastpitch.py` example script to use existing config

**PR Type**:
- [ ] New Feature
- [x] Bugfix
- [ ] Documentation
NeMo Megatron: Add sequence parallelism and selective activation checkpointing
State: closed | Pull request: yes

# What does this PR do ?

Add sequence parallelism and selective activation checkpointing.

**Collection**: NLP

# Changelog 
- Update GPT config
- Update parallel_lm_logits
- Add CoreAttention
  - This is the block that gets recomputed when using selective activation checkpointing
- Update ParallelAttention to work with CoreAttention
- Update ParallelTransformerLayer
  - Layer norm calls to use sequence parallel
- Update ParallelTransformer
  - Update args
  - Update final layer norm
  - Add rng contex to forward when using sequence parallel
  - Remove transposes
- Update Pooler
  - Gather from sequence parallel regions
- Update embedding
  - Add sequence_parallel flag
  - add transpose in forward
  - scatter for dropout
- Update TransformerLanguageModel
- Update GPTModel
  - Add transposes
- All-reduce layer norm params when using sequence parallel
- Add sequence parallel arg to apex calls
- Update configs and arg checking for activation checkpointing

# Usage

```
# enable sequence parallelism
model.sequence_parallel=True

# enable selective activation checkpointing
model.activations_checkpoint_granularity=selective

# all transformer layers have CoreAttention checkpointed when using selective
model.activations_checkpoint_method=null
model.activations_checkpoint_num_layers=null
```

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
Distributed training via Horovod does not make learning faster
State: closed | Pull request: no
Labels: name: bug

**Describe the bug**
I run [text classification with bert example](https://github.com/NVIDIA/NeMo/blob/main/examples/nlp/text_classification/text_classification_with_bert.py). Training is done well. I got approximately %90 accuracy after first epoch. Then I tried to run the same example with [horovod](https://github.com/horovod/horovod). My change is written below:



Change [line 116](https://github.com/NVIDIA/NeMo/blob/c03f87d47fc57abc89c0ebf859fccba397dd0f8e/examples/nlp/text_classification/text_classification_with_bert.py#L116):
`trainer = pl.Trainer(plugins=[NLPDDPPlugin()], **cfg.trainer)` to `trainer = pl.Trainer(strategy="horovod", **cfg.trainer)`. 

Change [config file](https://github.com/NVIDIA/NeMo/blob/main/examples/nlp/text_classification/conf/text_classification_config.yaml): 
model.train_ds.file_path = path/to/train
model.validation_ds.file_path = path/to/test
model.train_ds.batch_size = 8
model.validation_ds.batch_size = 8
model.dataset.num_classes = 2
trainer.devices = 1
trainer.max_epochs = 1

Then I run 2 training with this commands:
`horovodrun -np 1 -H localhost:1 --verbose --start-timeout 300 python text_classification_with_bert.py`
and
`horovodrun -np 2 -H localhost:2 --verbose --start-timeout 300 python text_classification_with_bert.py`

For second one, I also changed `trainer.devices` to 2 in config file.

First training took 15:06 min and second training took 22:00 min.

**Expected behavior**
I expect that second training should take less time.

**Environment overview**
horovod: 0.24.3
pytorch-lightning: 1.6.3
pytorch: 1.11.0
nemo-toolkit: 1.8.2
fix syntax error in ipynb-file
State: closed | Pull request: yes

Signed-off-by: Alexandra Antonova <aleksandraa@nvidia.com>

# What does this PR do ?

Fix syntax error in ipynb file

**Collection**: [Note which collection this PR will affect]

# Changelog 
- Add specific line by line info of high level changes in this PR.

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
Correcting spellings for BPE CTC conformer Models. 
State: closed | Pull request: no

Hi guys I fine-tuned a ctc large model on my custom dataset. Results are great but hv noticed some spellings issues. 

- How do I solve the spelling issues? 
- Do I need to use a lm for this?
- Is the tokeniser size the problem .  Uses same spe, unigram model with 256 tokens (added punts + numbers from 0 - 9)
unable to train asr
State: closed | Pull request: no
Labels: name: bug

**Describe the bug**

trying to build an ASR with min-Libri speech data

**command**
python3 speech_to_text_ctc_bpe.py   model.train_ds.manifest_filepath=data/mini_libri/train_clean_5.json     model.validation_ds.manifest_filepath=data/mini_libri/dev_clean_2.json     model.tokenizer.dir=data/tokenizer_libri/tokenizer_spe_bpe_v1024/     model.tokenizer.type="bpe"     trainer.devices=-1     trainer.accelerator="gpu"     trainer.strategy="ddp"     trainer.max_epochs=100     model.optim.name="adamw"     model.optim.lr=0.001     model.optim.betas=[0.9,0.999]     model.optim.weight_decay=0.0001     model.optim.sched.warmup_steps=2000     exp_manager.create_wandb_logger=True     exp_manager.wandb_logger_kwargs.name="Libri_1"     exp_manager.wandb_logger_kwargs.project="Asr_demo"
**Error**

Traceback (most recent call last):
  File "speech_to_text_ctc_bpe.py", line 95, in <module>
    main()
  File "/data/supreet/NeMo/nemo/core/config/hydra_runner.py", line 104, in wrapper
    _run_hydra(
TypeError: _run_hydra() missing 1 required positional argument: 'args'
fix load_from_checkpoint for MTEncDecModel
State: closed | Pull request: yes

Signed-off-by: Iztok Lebar Bajec <ilb@fri.uni-lj.si>

# What does this PR do ?

MTEncDecModel yaml configs do not declare `model.tokenizer`, hence without this fix restoring from a checkpoint will fail with
```
omegaconf.errors.ConfigAttributeError: Missing key tokenizer
    full_key: cfg.tokenizer
    object_type=dict
``` 

**Collection**: NLP/MT

# Changelog 
- Add specific line by line info of high level changes in this PR.

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [x] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
Dialogue tutorial fix
State: closed | Pull request: yes

# What does this PR do ?

Add a one line overview of what this PR aims to accomplish.

**Collection**: [Note which collection this PR will affect]

# Changelog 
- Add specific line by line info of high level changes in this PR.

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
Add RETRO indexed dataset and `set_inference_key_value_memory` inference 
State: closed | Pull request: yes

# What does this PR do ?

1. Added the RETRO index dataset for both data and retrieval data
2. Added the KNN Map index dataset
3. Added unit tests for all the index dataset
4. Added RETRO dataset that uses index dataset
5. Added the scripts to preprocess, build Faiss index and generate KNN map index for training
6. Added the `set_inference_key_value_memory` capability to RETRO modules so it can run inference efficiently 
7. Added unit tests for efficient inference of all RETRO modules.
Update configs from LAMB to AdamW
State: closed | Pull request: yes

Signed-off-by: Jocelyn Huang <jocelynh@nvidia.com>

# What does this PR do ?

Replaces LAMB optimizer (unstable, hangs during multi-GPU training) with AdamW for FastPitch and Mixer-TTS configs.

**Collection**: TTS

# Changelog 
- Replace LAMB defaults with AdamW
- Update `fastpitch.py` example script to use existing config

**PR Type**:
- [ ] New Feature
- [x] Bugfix
- [ ] Documentation
Dialogue tutorial fix
State: closed | Pull request: yes

# What does this PR do ?

Restore previously deleted files from conflicting PR

**Collection**: nlp

# Changelog 
- Add specific line by line info of high level changes in this PR.

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# Before your PR is "Ready for review"
**Pre checks**:
- [x] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [x] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
Dialogue tutorial fix
State: closed | Pull request: yes

# What does this PR do ?

Dialogue tutorial fix (path of a file has been shifted by a conflicting PR)

**Collection**:  nlp

# Changelog 
- Add specific line by line info of high level changes in this PR.

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [x] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
Fuse bias with geglu in ParallelMLP
State: closed | Pull request: yes

# What does this PR do ?

Fuse bias with geglu in ParallelMLP

# Changelog 
fused bias with geglu using TorchScript, fused function is in the newly added file fused_bias_geglu.py
in ParallelMLP, changed bias_gelu_fusion to biad_activation_fusion for generality, also fixed the way of this arg is propagated

# Usage
By default, bias_activation_fusion=True. If you want to disable it, simply set it as False in your config file.

# Before your PR is "Ready for review"
**Pre checks**:
- [x] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [x] Did you write any new necessary tests?
- [x] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [x] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
Prompt Learning Model Saving Changes
State: closed | Pull request: yes

Signed-off-by: Virginia Adams <vadams@nvidia.com>

Save all gpt + model weights to the model_weight.ckpt file removing the gpt model's .nemo file artifact. Saves GPT model's config file as an artifact. 

**Collection**: BigNLP

# Changelog 
Updated PromptLearningModel

# Usage
* You can potentially add a usage example below

Same as before

# Before your PR is "Ready for review"
**Pre checks**:
- [x] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
 - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [x] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
fix bugs for dialogue tutorial
State: closed | Pull request: yes

# What does this PR do ?

Fix typos in dialogue tutorial

**Collection**: [Note which collection this PR will affect]

# Changelog 
- Add specific line by line info of high level changes in this PR.

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)