renwei2024/nemo-github-issues huggingface dataset:
removed fp16 converting in complete method
State: closed | Pull request: yes

Signed-off-by: Dmytro Pykhtar <dpykhtar@nvidia.com>

# What does this PR do ?

Removed log_probs fp16 converting in complete method.

**Collection**: [Note which collection this PR will affect]

# Changelog 
- Add specific line by line info of high level changes in this PR.

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [x] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [x] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
Speaker diarization with ASR ouputs
State: closed | Pull request: no
Labels: name: feature

**Is your feature request related to a problem? Please describe.**

I am developing an API for a speaker diarization task with ASR (`/examples/speaker_tasks/diarization/offline_diarization_with_asr.py`). For my use case the script generates two useful outputs a `.json` that looks something like this:

```
{
	"status": "Success",
	"session_id": "example",
	"transcription": "thank you sunny day",
	"speaker_count": 4,
	"words": [
		{
			"word": "thank",
			"start_time": 0.0,
			"end_time": 0.6,
			"speaker_label": "speaker_1"
		},
		{
			"word": "you",
			"start_time": 0.7,
			"end_time": 1.1,
			"speaker_label": "speaker_1"
		},
		{
			"word": "sunny",
			"start_time": 1.5,
			"end_time": 2.1,
			"speaker_label": "speaker_2"
		},
		{
			"word": "day",
			"start_time": 2.2,
			"end_time": 2.3,
			"speaker_label": "speaker_2"
		}
	]
}
```

So we have the whole transcription which is very useful along with speaker labels for each of the spoken words. For API purposes this JSON is very handy however its contents are not very useful for practical applications. For diarization purposes and practical applications the script's `.txt` output is much more convenient:

```
[00:00.00 - 00:01.17] speaker_1: thank you
[00:01.54 - 00:02.33] speaker_2: sunny day
```

**Describe the solution you'd like**

Would it be possible to add the information from the `.txt` output to the JSON? E.g., something like:

```
	"diraization": [
		{
			"transcription": "thank you",
			"start_time": 0.0,
			"end_time": 1.17,
			"speaker_label": "speaker_1"
		},
		{
			"transcription": "sunny day",
			"start_time": 1.54,
			"end_time": 2.33,
			"speaker_label": "speaker_2"
		}
	]
```

**Describe alternatives you've considered**

I took a look at the code and I believe I would could code this myself, would you be interested in a pull request that modifies the output JSON?
Buffered speaker diarization
State: closed | Pull request: no
Labels: name: feature

**Is your feature request related to a problem? Please describe.**

For speaker diarization tasks tutorials and examples currently only offer scripts for offline diarization. I successfully got those running in combination with custom ASR models for Slovenian language. For ASR tasks the repository also contains scripts for buffered (streamed/live) inference, would such inference be possible for diarization?

**Describe the solution you'd like**

A tutorial or example on how to do buffered speaker diarization (if that is possible in the current framework). 

**Describe alternatives you've considered**

Some guidance about how I can implement this myself.
[TTS] Add cache_size to BetaBinomialInterpolator, fix bugs in TTS tutorials and FastPitch
State: closed | Pull request: yes

# What does this PR do ?

**Collection**: [TTS]

# Changelog 
- Change default speaker in FastPitch from `0` to `None` (i.e no speaker embedding is default now)
- Change call of `functools.lru_cache` (for python 3.7 support) and add `cache_size` to BetaBinomialInterpolator
- Add `pynini` install for TTS tutorials
  
**PR Type**:
- [x] New Feature
- [x] Bugfix
- [ ] Documentation

Closes #3627
Cannot export Fast Pitch models
State: closed | Pull request: no
Labels: name: bug

**Describe the bug**

I try to export fast pitch models, but it took a lot of RAM then process is killed. In my case, I use Mac OS to export and swap increase from 3 GB to 54 GB. I also used to try with google colab, but the kernel died after run export because memory issue.

**Steps/Code to reproduce bug**

```
from nemo.collections.tts.models import FastPitchModel

fastpitch = FastPitchModel.from_pretrained("tts_en_fastpitch")
fastpitch.export("fastpitch.onnx")
```

**Expected behavior**

The models can be exported to onnx

**Environment overview (please complete the following information)**

 - Environment location: Intel(R) Core(TM) i5-8279U CPU @ 2.40GHz
 - Method of NeMo install: pip install nemo_toolkit==1.6.2

**Environment details**

- OS version: MacOS 12.0.1
- PyTorch version: 1.10.1
- Python version: 3.8.9


Thank you
Audio based norm speed up
State: closed | Pull request: yes

Signed-off-by: ekmb <ebakhturina@nvidia.com>

# What does this PR do ?
Speeds.  up audio based normalization (from manifest)
Add a one line overview of what this PR aims to accomplish.

**Collection**: [Note which collection this PR will affect]

# Changelog 
- Add specific line by line info of high level changes in this PR.

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# Before your PR is "Ready for review"
**Pre checks**:
- [ X] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [ X] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
Enforce import tests on the three domains
State: closed | Pull request: yes

# What does this PR do ?

Enforce that all top level domain imports must be safe - and no optional dependency should break the base import.

**Collection**: [ASR, NLP, TTS]

# Changelog 
- Add an explicit import test to the three domains, independently of the others. 
- Follow the normal user style of installing nemo - but with individual domains.
- Error out if any domain breaks it, preventing merge of the PR.

# Usage
*  Part of CI test run on Github Actions

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [x] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [x] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [x] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [x] New Feature
- [ ] Bugfix
- [ ] Documentation
fix in clustering diarizer
State: closed | Pull request: yes

Signed-off-by: nithinraok <nithinrao.koluguri@gmail.com>

# What does this PR do ?

fix json -> join in clustering diarizer

**Collection**: speaker_tasks

# Changelog 
- change json -> join

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [x] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
Fixed Apex guard when imported classes are used for default values
State: closed | Pull request: yes

  1. Added ApexGuardDefaults class that provides None attributes when an Apex class is missing.
  2. Fixed Apex guard.

Signed-off-by: Micha Livne <mlivne@nvidia.com>

# What does this PR do ?

This PR fixes an Apex guard that leaves constructors with undefined constructor values.

**Collection**: [Note which collection this PR will affect]

# Changelog 
- Added `ApexGuardDefaults` class to `nemo/collections/nlp/modules/common/megatron/utils.py`
- Used `ApexGuardDefaults` to replace missing classes in `nemo/collections/nlp/modules/common/megatron/transformer.py`
typo in diarization class
State: closed | Pull request: no
Labels: name: bug

**Describe the bug**

In [diarization class (ClusteringDiarizer)](https://github.com/NVIDIA/NeMo/blob/main/nemo/collections/asr/models/clustering_diarizer.py#L395), there is `os.path.json()` function, but I think it is actually `os.path.join()`. 

Nemo version: 1.6.2
Add slot filling into DST Generative model
State: closed | Pull request: yes

# What does this PR do ?

See Design Doc and Milestones for Overarching project at [Nvidia-only access Google Docs](https://docs.google.com/document/d/1IUerv_PLPLgzm5rLEgvHITz_RmK4cKdJKJCV7BXTJ9c/edit?usp=sharing)

Add slot filling prediction capabilities and metrics

**Collection**: NLP

# Changelog 
- Integrated the capability of performing intent classification and slot filling jointly using a GPT-style model
- Include metrics for slot filling
- Edited dataset class to improve user customizability in using templates for generation
- Backported IntentSlotClassificationModel into DST module

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# Before your PR is "Ready for review"
**Pre checks**:
- [x] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [x] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
GEGLU activation for T5
State: closed | Pull request: yes

Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca>

# What does this PR do ?

Adds the GEGLU activation function for T5

**Collection**: NLP

# Changelog 
- Updates ParallelMLP to instantiate a new linear layer if the activation is GEGLU.
- Applies the appropriate activation function based on the provided `activation` arg.

# Usage
* You can potentially add a usage example below

Set `model.activation=gelu` or `model.activation=geglu` in `megatron_t5_config.yaml`

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
Megatron_LM checkpoint to NeMo checkpoint support 
State: closed | Pull request: yes

# What does this PR do ?

Convert Megatron LM checkpoint to NeMo checkpoint. Copy over the weights, optimizer states, LR scheduler states.
Fix the issue of weight decay parameter group assignment. 
Tested that the script converts the old Megatron GPT and BERT checkpoint to `.nemo` file fine.

Tested with 40B checkpoint file, the NeMo Megatron can resume from it and continue the training. Need to verify the training curve works fine.
Jasper mask/export fix
State: closed | Pull request: yes

# What does this PR do ?

Add a one line overview of what this PR aims to accomplish.

**Collection**: [Note which collection this PR will affect]

# Changelog 
- Add specific line by line info of high level changes in this PR.

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
lower bound PTL to 1.5.10 and remove last ckpt patch fix
State: closed | Pull request: yes

Signed-off-by: nithinraok <nithinrao.koluguri@gmail.com>

# What does this PR do ?

lower bound PTL to 1.5.10 and remove last ckpt patch fix in exp manager

**Collection**: core, exp_manager

# Changelog 
- lower bound PTL to 1.5.10
- Remove `_save_last_checkpoint` function from exp_manager. PTL fixed issue in 1.5.10 


# Before your PR is "Ready for review"
**Pre checks**:
- [x] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [x] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
TTS - Mixing datasets for FastPitch + HiFiGAN
State: closed | Pull request: no

### Discussed in https://github.com/NVIDIA/NeMo/discussions/3678

<div type='discussions-op-text'>

<sup>Originally posted by **gedefet** February 15, 2022</sup>
Hi All.
<br>
At the end of the FastPitch_Finetuning.ipynb tutorial:

https://github.com/NVIDIA/NeMo/blob/main/tutorials/tts/FastPitch_Finetuning.ipynb
<br>
it suggest two different ways to improve performance, one of which is:
<br>
_Mix new speaker data with old speaker data
We recommend to train fastpitch using both old speaker data (LJSpeech in this notebook) and the new speaker data. In this case, please modify the .json when finetuning fastpitch to include speaker information:_
<br>
but it isn't 100% clear how to do that.
<br>
I found another FastPitch_Finetuning.ipynb here: 

https://colab.research.google.com/github/NVIDIA/NeMo/blob/stable/tutorials/tts/FastPitch_Finetuning.ipynb
<br>
Which is different than the other. 
<br>
It seems that it has two functions defined to do the mixing, but it seems that this colab is abandoned. Nevertheless, I think that has very useful code, and that has tackled this part of mixing datasets.
<br>
Is this true? Or do I need something more that is not there?Why it is abandoned?
<br>
Thanks!</div>
fastpitch_finetuning.ipynb does not work from scratch
State: closed | Pull request: no
Labels: name: bug

Hi all. 

The tutorial https://github.com/NVIDIA/NeMo/blob/main/tutorials/tts/FastPitch_Finetuning.ipynb does not work from scratch.

First, I have to replace NeMo for NVIDIA in the first setup cell for enable it to run:

```
!python -m pip install git+https://github.com/NeMo/NeMo.git@$BRANCH#egg=nemo_toolkit[all]
```

for

```
!python -m pip install git+https://github.com/NVIDIA/NeMo.git@$BRANCH#egg=nemo_toolkit[all]
```

Then, I got the following error in the 11th cell:

First commands of the cell:

```
new_speaker_id = 6097
duration_mins = 5
mixing = False
original_speaker_id = "ljspeech"

_ ,last_ckpt = get_best_ckpt("./", new_speaker_id, duration_mins, mixing, original_speaker_id)
print(last_ckpt)
```

Error:

```
AttributeError                            Traceback (most recent call last)
[/usr/local/lib/python3.7/dist-packages/torch/serialization.py](https://localhost:8080/#) in _check_seekable(f)
    307     try:
--> 308         f.seek(f.tell())
    309         return True

AttributeError: 'NoneType' object has no attribute 'seek'

During handling of the above exception, another exception occurred:

AttributeError                            Traceback (most recent call last)
8 frames
[/usr/local/lib/python3.7/dist-packages/torch/serialization.py](https://localhost:8080/#) in raise_err_msg(patterns, e)
    302                                 + " Please pre-load the data into a buffer like io.BytesIO and"
    303                                 + " try to load from it instead.")
--> 304                 raise type(e)(msg)
    305         raise e
    306 

AttributeError: 'NoneType' object has no attribute 'seek'. You can only torch.load from a file that is seekable. Please pre-load the data into a buffer like io.BytesIO and try to load from it instead.
```

Best,
Comment out numba in 22.01 release
State: closed | Pull request: yes

Signed-off-by: smajumdar <titu1994@gmail.com>

# What does this PR do ?

Comment out numba install in 22.01 release. It contains numba 0.53.1 which works with RNNT.

**Collection**: [ASR]

# Changelog 
- Comment out numba due to numpy conflict.

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [x] Bugfix
- [ ] Documentation
Add pipeline support to eval methods
State: closed | Pull request: yes

# What does this PR do ?
Updates `.complete` and `.compute_logprobs` to work with pipeline parallelism.

**Collection**: [NLP]

# Changelog 
- Add specific line by line info of high level changes in this PR.

# Usage
```
python megatron_gpt_eval.py \
--model_file /raid/nemo_files/gpt_126m_tf32_pp2_tp1/megatron_gpt.nemo \
--prompt "Four score and seven years ago our fathers brought forth on this continent, a new nation, conceived in Liberty, and dedicated to the proposition that all men" \
--tensor_model_parallel_size 1 \
--pipeline_model_parallel_size 2 \
--tokens_to_generate 64 \
--stop_after_sentence False --precision 32
```

# Before your PR is "Ready for review"
**Pre checks**:
- [x] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [x] Did you write any new necessary tests?
- [x] Did you add or update any necessary documentation?
- [x] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [x] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
[Question] nemo container on ngc
State: closed | Pull request: no
Labels: name: bug

Is there any specific reason that the nemo:1.6.2 container is not yet available on ngc; (https://catalog.ngc.nvidia.com/orgs/nvidia/containers/nemo/tags). The info says the container was modified on Feb. 14, 2022, but the latest tag is still 1.5.1 from Dec. 15, 2021, even though 1.6.2 was released 11 days ago.
Update Librosa support to 0.9
State: closed | Pull request: yes

Signed-off-by: smajumdar <titu1994@gmail.com>

# What does this PR do ?

Adds support for Librosa 0.9

**Collection**: [ASR, TTS]

# Changelog 
- Update all librosa calls to required kwargs calls

  
**PR Type**:
- [ ] New Feature
- [x] Bugfix
- [ ] Documentation
Test HF online for SGD-GEN only
State: closed | Pull request: yes

Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca>

# What does this PR do ?

Sets `TRANSOFMERS_OFFLINE=0` for SGD-Gen tests only to avoid issues with model caching.

**Collection**: NLP

# Changelog 

- Edited Jenkins CI tests to set `TRANSOFMERS_OFFLINE` appropriately.

# Usage

N/A

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
[TTS] Update TTS tutorials, Simplification of testing Mixer-TTS and FastPitch
State: closed | Pull request: yes

# What does this PR do ?

Add a one line overview of what this PR aims to accomplish.

**Collection**: [TTS]

# Changelog 
- Update TTS tutorials
- Simplification of testing Mixer-TTS and FastPitch
- Small fix in FastPitch

**PR Type**:
- [x] New Feature
- [ ] Bugfix
- [ ] Documentation

Closes #3603 #3627 #3628 #3676
added num_workers param to all ASR conf files
State: closed | Pull request: yes

# What does this PR do ?

Add a one line overview of what this PR aims to accomplish.

**Collection**: [Note which collection this PR will affect]

# Changelog 
- Add specific line by line info of high level changes in this PR.

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [x] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
TypeError: Error instantiating 'nemo.collections.tts.torch.data.TTSDataset'
State: closed | Pull request: no
Labels: name: bug

**Describe the bug**

When I was trying to train Fastpitch from scratch using `fastpitch_align_v1.05.yaml` config, after it spending over 1 hour trying to `Loading dataset`, it threw the error:
```
Traceback (most recent call last):
  File "examples/tts/fastpitch.py", line 28, in main
    model = FastPitchModel(cfg=cfg.model, trainer=trainer)
  File "/root/NeMo/nemo/collections/tts/models/fastpitch.py", line 81, in __init__
    super().__init__(cfg=cfg, trainer=trainer)
  File "/root/NeMo/nemo/core/classes/modelPT.py", line 138, in __init__
    self.setup_training_data(self._cfg.train_ds)
  File "/root/NeMo/nemo/collections/tts/models/fastpitch.py", line 453, in setup_training_data
    self._train_dl = self.__setup_dataloader_from_config(cfg)
  File "/root/NeMo/nemo/collections/tts/models/fastpitch.py", line 440, in __setup_dataloader_from_config
    dataset = instantiate(
  File "/root/miniconda3/lib/python3.8/site-packages/hydra/_internal/instantiate/_instantiate2.py", line 180, in instantiate
    return instantiate_node(config, *args, recursive=_recursive_, convert=_convert_)
  File "/root/miniconda3/lib/python3.8/site-packages/hydra/_internal/instantiate/_instantiate2.py", line 249, in instantiate_node
    return _call_target(_target_, *args, **kwargs)
  File "/root/miniconda3/lib/python3.8/site-packages/hydra/_internal/instantiate/_instantiate2.py", line 64, in _call_target
    raise type(e)(
  File "/root/miniconda3/lib/python3.8/site-packages/hydra/_internal/instantiate/_instantiate2.py", line 62, in _call_target
    return _target_(*args, **kwargs)
  File "/root/NeMo/nemo/collections/tts/torch/data.py", line 228, in __init__
    librosa.filters.mel(
TypeError: Error instantiating 'nemo.collections.tts.torch.data.TTSDataset' : mel() takes 0 positional arguments but 2 positional arguments (and 3 keyword-only arguments) were given
```

**Steps/Code to reproduce bug**

` python examples/tts/fastpitch.py --config-name=fastpitch_align_v1.05.yaml train_dataset=/root/autodl-tmp/nancy_train.json validation_datasets=/root/autodl-tmp/nancy_val.json sup_data_path=/root/autodl-tmp/nemo-training/nancy_22k/sup_data exp_manager.exp_dir=/root/autodl-tmp/nemo-training/nancy_22k exp_manager.resume_if_exists=True exp_manager.resume_ignore_no_checkpoint=True model.train_ds.dataloader_params.batch_size=24 pitch_mean=199.37802124023438 pitch_std=55.59949493408203`


**Expected behavior**

A clear and concise description of what you expected to happen.

**Environment overview (please complete the following information)**

 - Environment location: Cloud
 - Method of NeMo install:  from source


**Environment details**

If NVIDIA docker image is used you don't need to specify these.
Otherwise, please provide:
- OS version: Ubuntu 20.04
- PyTorch version: 1.10
- Python version: 3.8

**Additional context**

Add any other context about the problem here.
Example: RTX 3090
Need Confirmation
State: closed | Pull request: no
Labels: name: feature

Dear NVIDIA Team,
I need a small confirmation from your end, please confirm that with the below command the model which is restored is from best checkpoints, as after that I am using the loaded model for predictions with the second command

model = TextClassificationModel.restore_from(restore_path='./trained-model-text-nvidia.nemo')

predicted_class = model.classifytext(queries=config.model.infer_samples, batch_size=2, max_seq_length=511)
Dialogue state tracking refactor/ SGDGEN patch 2 
State: closed | Pull request: yes

# What does this PR do ?

Fix bug that causes CI to break due to huggingface cache inconsistency by using local file location

**Collection**: NLP

# Changelog 
Previous patch provide a lm_checkpoint location for model to restore_weights, but this method still calls `AutoModel.from_pretrained('gpt2')` before restoring local weights, which can sometimes require cache.

Current patch overcomes this by using `AutoModel.from_pretrained(<local_gpt2_weights_folder>)`, which means that it searches for the local folder directly.

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# Before your PR is "Ready for review"
**Pre checks**:
- [x] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [x] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
Fixing the bug in the stateful rnnt decoder.
State: closed | Pull request: yes

# What does this PR do ?

Fixing the bug in the stateful rnnt decoder. The current code does not keep the last token properly from partial hypothesis to the next hypothesis. If The latest decoding does not produce any tokens, it would not propagate the non-blank last token and just passes blank token to the next hypothesis which is wrong.

# Changelog 
- Fixed the bug in the stateful rnnt decoder by keeping the last token and passing that along properly.

**PR Type**:
- [ ] New Feature
- [x ] Bugfix
- [ ] Documentation
Added speaker identification script with cosine and neural classifier…
State: closed | Pull request: yes

… backends

Signed-off-by: nithinraok <nithinrao.koluguri@gmail.com>

# What does this PR do ?

Added speaker identification script with cosine and neural classifier backends

**Collection**: speaker_tasks

# Changelog 
- replaced speaker_reco_infer.py with speaker_identification_infer.py script
- added conf/speaker_identification_infer.yaml 
- Updated documentation and README

# Usage


```bash
python speaker_identification_infer.py data.enrollment_manifest=<path/to/enrollment_manifest> data.test_manifest=<path/to/test_manifest> backend.backend_model=cosine_similarity
```

# Before your PR is "Ready for review"
**Pre checks**:
- [x] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [x] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [x] New Feature
- [ ] Bugfix
- [x] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
folder name update
State: closed | Pull request: yes

Signed-off-by: ekmb <ebakhturina@nvidia.com>

# What does this PR do ?

Fixes folder name for jenkins test.

**Collection**: [Note which collection this PR will affect]

# Changelog 
- Add specific line by line info of high level changes in this PR.

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [X ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
Parallel prompt tuning
State: closed | Pull request: yes

# What does this PR do ?

This PR fixes prompt tuning to work with the updated pipeline parallel code. In this PR a user can also prompt tune using tensor parallel > 1.  More work will need to be done for prompt tuning to be supported for pipeline parallel > 1. 

**Collection**: [Note which collection this PR will affect]
BigNLP? 

# Changelog 
- prompt tuning with tp=1 and pp=1 works again with updated pipeline parallel code
- prompt tuning with tp > 1 works
- prompt tuning works with the updated complete and compute log probs methods
- Added a prompt tuning config file with prompting specific defaults set
- prompt tuning tests are add back

# Usage
```python
python examples/nlp/language_modeling/megatron_gpt_prompt_tuning.py \
    --config-name=megatron_prompt_tuning_gpt \
    restore_from_path='/prompt-tuning/megatron_gpt.nemo' \
    trainer.val_check_interval=2 \
    trainer.max_steps=5 \
    model.new_prompt_tags=['Winogrande, BoolQ'] \
    model.new_prompt_init_text=['logic choose person name, None'] \
    model.new_prompt_init_methods=['text, random'] \
    model.data.train_ds='/prompt-tuning/wino_bool_prompt_tuning_train.json' \
    model.data.valid_ds='/prompt-tuning/wino_bool_prompt_tuning_val.json' \
    +model.data.test_ds='/prompt-tuning/wino_bool_prompt_tuning_val.json' \
    model.micro_batch_size=2 \
    model.global_batch_size=4 \
    model.optim.lr=2e-2 \
    model.optim.sched.min_lr=2e-3 \
    model.optim.sched.warmup_steps=2 \
    model.optim.sched.constant_steps=8 \
    model.encoder_seq_length=2048
```

# Before your PR is "Ready for review"
**Pre checks**:
- [X] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [X] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
- [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [x] New Feature
- [X] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
Add Buffered RNNT with LCS Merge algorithm
State: closed | Pull request: yes

Signed-off-by: smajumdar <titu1994@gmail.com>

Adds an optional research notebook discussing the Longest Common Subsequence merge algorithm as used for Streaming/Buffered transducer inference.

**Collection**: [ASR]

# Changelog 
- Add optional research notebook

**PR Type**:
- [x] New Feature
- [ ] Bugfix
- [ ] Documentation
Added export wrappers for LayerNorm and BatchNorm2d, cleanup
State: closed | Pull request: yes

Signed-off-by: Boris Fomitchev <bfomitchev@nvidia.com>

# What does this PR do ?

Add a one line overview of what this PR aims to accomplish.

**Collection**: [Note which collection this PR will affect]

# Changelog 
- Add specific line by line info of high level changes in this PR.

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
Dialogue state tracking refactor
State: closed | Pull request: yes

# What does this PR do ?

Unblock CI for SGDGEN by locally loading model checkpoint instead of calling from huggingface

**Collection**: NLP

# Changelog 
- Add specific line by line info of high level changes in this PR.

# Usage
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```

# Before your PR is "Ready for review"
**Pre checks**:
- [x] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [x] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
Megatron BART
State: closed | Pull request: yes

# What does this PR do ?

This PR adds support in BART Megatron model <https://arxiv.org/abs/1910.13461>.

**Collection**: `nemo.collections.nlp.models.language_modeling.megatron_bart_model`

# Usage

```
python /code/examples/nlp/language_modeling/megatron_bart_pretraining.py \
	--config-path=/code/examples/nlp/language_modeling/conf \
	--config-name=megatron_bart_config \
	trainer.num_nodes=1 \
	trainer.devices=1 \
	trainer.max_epochs=null \
	trainer.max_steps=${MAX_STEPS} \
	trainer.val_check_interval=${VAL_CHECK_INTERVAL} \
	trainer.accumulate_grad_batches=${ACCUMULATE_GRAD_BATCHES} \
	trainer.precision=16 \
	model.micro_batch_size=${MICRO_BATCH_SIZE} \
	model.tensor_model_parallel_size=${TENSOR_MODEL_PARALLEL_SIZE} \
	model.max_position_embeddings=${SEQ_LENGTH} \
	model.seq_length=${SEQ_LENGTH} \
	model.hidden_size=${HIDDEN_SIZE} \
	model.num_layers=${NUM_LAYERS} \
	model.num_attention_heads=${NUM_ATTENTION_HEADS} \
	model.init_method_std=0.015 \
	model.data.data_prefix=[1.0,/preproc_data/train/my-t5_test_bert_tokenizer_text_document] \
	model.data.seq_length=${SEQ_LENGTH} \
```
[NeMo Megatron] use worker processes for data preprocessing
State: closed | Pull request: yes

# What does this PR do ?

<!-- Add a one line overview of what this PR aims to accomplish. -->

Update Megatron style `BatchSampler`s and `GPTDataset` to prepare data samples in worker processes.


# Changelog 
- BatchSamplers return a list of `num_micro_batch x micro_batch_size` indices: https://github.com/NVIDIA/NeMo/compare/main...crcrpar:megatron/update_data_loading?expand=1#diff-a78cf82731c4e37f487037ab45cab0183aac6216ee96e08d3b81ff111495d3caR36 and https://github.com/NVIDIA/NeMo/compare/main...crcrpar:megatron/update_data_loading?expand=1#diff-a78cf82731c4e37f487037ab45cab0183aac6216ee96e08d3b81ff111495d3caR106
- A new arguments of `eos_id` is added to `GPTDataset` in order to change the callsite of `get_ltor_masks_and_position_ids` to `GPTDataset`. [`eos_id`](url) is an attribute of a `Tokenizer` instance.
- `process_micro_batch` is removed in favor of updated `GPTDataset.__getitem__`: https://github.com/NVIDIA/NeMo/compare/main...crcrpar:megatron/update_data_loading?expand=1#diff-e93ccae74f4b67d341676afc9f3c7e2c50f751ec64df84eb3b2a86b62029ef76L456
- `NLPDataConnector` isn't needed thanks to the updated `GPTDataset` and `BatchSampler`s: https://github.com/NVIDIA/NeMo/compare/main...crcrpar:megatron/update_data_loading?expand=1#diff-424bb19c0b7c9dba2aebb3459972c5a8bf8c902dca48a8762f12bc5cde5fea5fL68

# Usage

To utilize worker processes, it'd be better to set `model.data.num_workers` > 0.
See https://pytorch.org/docs/stable/data.html#torch.utils.data.DataLoader.

<!-- 
* You can potentially add a usage example below

```python
# Add a code snippet demonstrating how to use this 
```
-->

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [x] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
O2 style mixed precision training for T5
State: closed | Pull request: yes

# What does this PR do ?

Adds support for O2-style mixed-precision training for Megatron LM Encoder-Decoder Models.

**Collection**: NLP

# Changelog 
- This PR only changes `megatron_lm_encoder_decoder.py` to add O2 support based on the same changes for GPT.

# Usage

Set `megatron_amp_o2: True` in the `megatron_t5_config.yaml` file.

# Before your PR is "Ready for review"
**Pre checks**:
- [ ] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)
Fixed the bug in bucketing.
State: closed | Pull request: yes

# What does this PR do ?

Fix a bug in the bucketing which is caused by this PR: https://github.com/NVIDIA/NeMo/pull/3485
Training ASR models with bucketing feature were affected by this.

# Changelog 
Changed the way the length of a dataset is retrieved to fix the bug in the bucketing.

**PR Type**:
- [ ] New Feature
- [X] Bugfix
- [ ] Documentation
[TTS] Add speakerID to libritts/get_data.py
State: closed | Pull request: yes

Signed-off-by: Subhankar Ghosh <subhankar2321@gmail.com>

# What does this PR do ?

Adds SpeakerID to `process_transcript` method of libritts/get_data.py

# Changelog 
Gets speakerID from the path because the path to every manifest has the speakerID.
Adds the speakerID to the entry as another (key,value) pair. 

# Usage
Doesn't change how libritts/get_data.py will be called. The manifests will contain speaker field.


# Before your PR is "Ready for review"
**Pre checks**:
- [x] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [ ] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)